Search Docs
Loading...
Skip to content

Quickstart: OpenAI Agents SDK

Type a brief and get a finished PNG and an editable design file from a design agent on the OpenAI Agents SDK, in around 3 minutes, depending on the model and how hard the brief is.

12 mins
estimated time
Download
GitHub

To have a coding agent, such as Claude Code or Cursor, set up this kit for you, give it this prompt:

Open https://img.ly/docs/cesdk/codesign/starter-kits/quickstart-openai-agents-cdsqoa.md
and set up this starter kit for me.

What It Does#

You run one command with a brief, such as “A square Instagram post for a spring sale on ceramic mugs”. An AI model designs it, looks at what it made, and improves it, then the kit writes two files:

  • out/design.png, a picture of the design.
  • out/design.imgly, the design itself with its images and fonts, which you can open and edit in CE.SDK’s web editor ( Your Editor). CE.SDK is IMG.LY’s design engine and editor.

Start from this kit if your product runs a design agent on its servers: a chat, a job queue, or an API. Two things run on your machine:

  • The executor, a Docker container from the image imgly/codesign-executor, which IMG.LY publishes. It holds CE.SDK’s engine and runs the short JavaScript programs the model writes to change the design, in its own container, apart from your keys and your data. You write no code for it.
  • The agent runner, design.mjs, the kit’s code and later yours. It gets the design tools from the CoDesign SDK, @imgly/codesign-sdk, and hands them to the OpenAI Agents SDK (@openai/agents). A tool is a function the model can ask to call, such as one that changes the design or renders a preview. The Agents SDK runs the agent loop: it calls the model, runs each tool the model asks for, and sends the result back, until the model is done. Then the script exports the design.

The model runs through IMG.LY’s AI gateway, a service that forwards model calls to the model provider and charges them to your IMG.LY account, so you need no account with a model provider.

Pre-requisites#

  • Docker, to run the executor – Download
  • Node.js v22+ with npm, pnpm, or yarn – Download
  • A CoDesign license key. Contact sales to get a CoDesign license key. That one key starts the design engine, pays for each exported file, and pays for the model through the gateway.

Get Started#

Step 1: Clone the Repository#

Terminal 
git clone https://github.com/imgly/starterkit-codesign-quickstart-openai-agents-js-node.git
git clone https://github.com/imgly/starterkit-codesign-quickstart-openai-agents-js-node.git

What is inside:

starterkit-codesign-quickstart-openai-agents-js-node/
├── design.mjs           # The agent runner: prepares the tools, hands them to the Agents SDK, exports the design
├── .env.template        # The variables to copy into .env
├── README.md            # How to run the kit
├── LICENSE.md           # IMG.LY's license, the same as the CoDesign SDK's
├── package.json         # The SDK, the Agents SDK, and Zod, a schema library the Agents SDK loads, at exact versions; `npm run executor` starts the executor
├── tsconfig.json        # `npm run check` type-checks design.mjs
└── tsconfig.base.json   # The compiler settings tsconfig.json extends

Read design.mjs after your first run; How It Works below explains it.

Step 2: Install Dependencies#

Terminal 
cd starterkit-codesign-quickstart-openai-agents-js-node
npm install
cd starterkit-codesign-quickstart-openai-agents-js-node
npm install

package.json pins @imgly/codesign-sdk to the same version as the executor image’s tag in its executor script. Keep the two equal, because the SDK and the executor must come from the same build ( Versions and Upgrades).

Step 3: Set Your Keys#

Copy .env.template to .env and fill in one variable: IMGLY_CESDK_LICENSE holds your CoDesign license key (the variable keeps CE.SDK’s name). EXECUTOR_TOKEN, the token your runner and the executor share, is already filled in:

.env
IMGLY_CESDK_LICENSE=<your CoDesign license key>
EXECUTOR_TOKEN=codesign-quickstart-token

The executor script passes .env to the executor, and the start script to design.mjs, so the two always have the same token. Two optional variables are described in .env.template:

Variable What it is
MODEL The gateway’s model id. Default @openrouter/anthropic/claude-opus-5. The gateway lists its ids at https://gateway.img.ly/v1/models.
EXECUTOR_URL The executor’s address. Default http://localhost:8080.

Step 4: Start the Executor#

Terminal 
npm run executor
curl http://localhost:8080/health
npm run executor
curl http://localhost:8080/health

The executor script runs IMG.LY’s image in Docker with the variables from .env, on port 8080 of your machine only. The first start downloads the image. curl prints a line like this one:

{"result":{"sessions":0,"accepting":true,"bound":false,...}}

If /health does not answer, run docker logs codesign-executor to see why. The executor stops at start when the license key is missing or cannot be activated, or when it cannot reach api.img.ly.

Step 5: Design Something#

Terminal 
npm start – “A square Instagram post for a 20% spring sale on ceramic mugs. Calm, premium, lots of white space.”
npm start – “A square Instagram post for a 20% spring sale on ceramic mugs. Calm, premium, lots of white space.”

A run takes around 3 minutes, depending on the model and how hard the brief is. You see a line for each tool call as it happens, a ✓ and the tool’s name, as the model reads IMG.LY’s design rules, changes the design, and looks at a render, a picture of the design. Then you see the model’s one-sentence summary and a line with the number of steps, the seconds, and the tokens used. The last line is:

Wrote out/design.png and out/design.imgly

Open out/design.png to see the design. out/design.imgly opens in CE.SDK’s web editor, which you embed in a page of your product ( Your Editor). Use exactly the CE.SDK version the executor runs, which the SDK exports as CESDK_VERSION: an older or newer editor that saves the file drops what the other version wrote ( Versions and Upgrades).

To stop and remove the executor, run npm run executor:stop.

Run It Again#

The executor serves one conversation: your id for one chat or job, which design.mjs sets to quickstart on every run. At the end of a run, the script calls release(), which closes the design on the executor. Half a minute later the executor exits, and Docker starts it again, ready for your next run.

A run that starts in the few seconds while the container restarts fails with the executor at http://localhost:8080 is unreachable. Wait until curl http://localhost:8080/health answers again.

Before you run the executor anywhere but your machine, follow Host the Executor.

What It Costs#

Each run’s PNG export and the model’s tokens are charged to the IMG.LY account behind your key; the .imgly file, the edits, and the previews are free ( What It Costs).

How It Works#

design.mjs does six things. The excerpts below are from it, and the whole file follows them.

  1. It calls the model through IMG.LY’s gateway. The gateway at https://gateway.img.ly/v1 speaks OpenAI’s Chat Completions, the request format most model providers accept. Your license key is its API key. The Agents SDK reaches it with an OpenAIProvider pointed at that address. useResponses: false makes it send Chat Completions; by default the Agents SDK uses OpenAI’s newer Responses API, which the gateway does not offer.
// IMG.LY's AI gateway speaks OpenAI's Chat Completions; your CoDesign license key is its API key.
const imglyGateway = new OpenAIProvider({
  baseURL: 'https://gateway.img.ly/v1',
  apiKey: process.env.IMGLY_CESDK_LICENSE,
  useResponses: false // the gateway has Chat Completions, not OpenAI's Responses API
});
  1. It prepares the tools with prepareCoDesign.
    • conversation is your id for the work. Every design of a conversation opens on one executor that serves no other conversation, because the model’s code runs there; see Security.
    • executors: remoteExecutors({ acquire, token }) says where a conversation’s executor is. acquire is a function you write. Here it always answers the container on port 8080. In production, it answers a new container for each new conversation, and the same one while the conversation lasts.
    • plugins lists the CE.SDK plugins the engine applies. A plugin adds asset sources: searchable lists of what a design can use, such as photos, typefaces, and colors. googleFonts() is IMG.LY’s Google Fonts catalog, the smallest useful list.
    • design is the id of the design the tools work on. With no designs store, the design lives in the runner’s memory for the run.
    • gateway: false leaves out image generation. The model still runs through the gateway. The option is required, so you decide.
const codesign = prepareCoDesign({
  // Your id for this job. Every design of a conversation opens on one executor
  // that serves no other conversation. Every run here is yours, so all share one.
  conversation: 'quickstart',
  executors: remoteExecutors({
    // The executor for a conversation. Here there is one; a product starts a
    // new one for each conversation.
    acquire: () => ({
      url: process.env.EXECUTOR_URL || 'http://localhost:8080'
    }),
    token: process.env.EXECUTOR_TOKEN
  }),
  // The CE.SDK plugins the engine applies: everything the agent can search and
  // place. Google Fonts is the smallest useful list; your own plugins go here.
  plugins: [googleFonts()],
  design: `quickstart-${Date.now()}`, // any id; with no `designs` store, the design lives in memory
  gateway: false // no image generation; the model still runs through the gateway
});

  1. It hands the tools to the Agents SDK. Each SDK tool is { name, description, inputSchema, call }, and the Agents SDK’s tool() wraps each one. Three settings matter:

    • strict: false sends the SDK’s JSON Schema as it is. In its default strict mode, the Agents SDK rewrites a schema so that every property is required, which changes what the model is asked for.
    • A call the model can fix, such as one with a missing argument, comes back with isError and a sentence that says what to change. The kit returns that sentence to the model as the tool’s output.
    • errorFunction: null ends the run when a call fails in a way the model cannot fix, such as an executor that does not answer. Without it, the Agents SDK tells the model only that an error occurred, and the run goes on.

    The renders come back as image outputs, so the model gets them as pictures, not as text.

// The SDK's tools in the Agents SDK's shape.
const tools = codesign.tools.map((t) =>
  tool({
    name: t.name,
    description: t.description ?? '',
    // Plain JSON Schema, sent as it is: strict mode would make every property required.
    parameters:
      /** @type {{ type: 'object', properties: {}, required: [], additionalProperties: true }} */ (
        t.inputSchema
      ),
    strict: false,
    execute: async (args, _context, details) => {
      const result = await t.call(args ?? {}, { signal: details?.signal });
      console.log(`${result.isError ? '✗' : '✓'} ${t.name}`);
      if (result.isError) return textOf(result.content); // the refusal, in words the model can act on
      return result.content.map((p) =>
        p.type === 'text'
          ? { type: /** @type {const} */ ('text'), text: p.text }
          : {
              type: /** @type {const} */ ('image'),
              image: { data: p.data, mediaType: p.mimeType }
            }
      );
    },
    // A call the SDK rejects (the executor does not answer) ends the run with
    // its error; by default the model would get a generic message and go on.
    errorFunction: null
  })
);
  1. It lets the model see its renders. Chat Completions allows no images inside a tool result, and the Agents SDK drops them there with a warning. The kit’s imagesAfterToolResults runs as the Runner’s callModelInputFilter, before each model call, and moves each render into a user message after the tool results. Without that step nothing fails, but the model designs without seeing its work.
// Chat Completions has no images in a tool result: move each render into a
// user message after it, so the model sees its work. Left out, nothing fails;
// the model designs blind.
/** @type {import('@openai/agents').CallModelInputFilter} */
const imagesAfterToolResults = ({ modelData }) => {
  /** @type {import('@openai/agents').AgentInputItem[]} */
  const input = [];
  /** @type {{ type: 'input_image', image: string }[]} */
  let images = [];
  const flush = () => {
    if (images.length === 0) return;
    input.push({
      role: 'user',
      content: [
        { type: 'input_text', text: 'Attached image(s) from tool result:' },
        ...images
      ]
    });
    images = [];
  };
  for (const item of modelData.input) {
    if (item.type !== 'function_call_result' || !Array.isArray(item.output)) {
      if (item.type !== 'function_call_result') flush();
      input.push(item);
      continue;
    }
    const kept = item.output.filter((part) => part.type !== 'input_image');
    for (const part of item.output) {
      if (part.type === 'input_image' && typeof part.image === 'string')
        images.push({ type: 'input_image', image: part.image });
    }
    input.push({
      ...item,
      output:
        kept.length > 0
          ? kept
          : [
              {
                type: /** @type {const} */ ('input_text'),
                text: '(see attached image)'
              }
            ]
    });
  }
  flush();
  return { ...modelData, input };
};
  1. It runs the agent. The Agent gets your own system prompt with codesign.instructions after it, the model id, and the tools. The instructions tell the model how to use the tools, so your prompt names none of them. An Agents SDK Runner object holds the gateway, the filter, and tracingDisabled, which keeps the Agents SDK from sending a trace of every run to OpenAI’s platform, as it does by default, so only the model calls leave your machine:
const agent = new Agent({
  name: 'Designer',
  instructions: `You design marketing visuals. When you are done, say in one sentence what you made.\n\n${codesign.instructions}`,
  model: process.env.MODEL || '@openrouter/anthropic/claude-opus-5',
  tools
});


const runner = new Runner({
  modelProvider: imglyGateway,
  callModelInputFilter: imagesAfterToolResults,
  tracingDisabled: true // the Agents SDK would otherwise send a trace of every run to OpenAI
});
  1. It allows the model enough turns. runner.run(agent, brief, { maxTurns: 60 }) lets the model take up to 60 turns; a turn is one model call, which may ask for tools. The Agents SDK’s default is 10, which ends a design run before it is done, with MaxTurnsExceededError. When the model is done, the script exports the PNG and the .imgly file and calls release().
The whole file
design.mjs
// A brief in, a finished design out: the OpenAI Agents SDK runs the model through IMG.LY's gateway, the engine runs in one local executor.
import { mkdir, writeFile } from 'node:fs/promises';


import {
  googleFonts,
  prepareCoDesign,
  remoteExecutors
} from '@imgly/codesign-sdk';
import {
  Agent,
  MaxTurnsExceededError,
  OpenAIProvider,
  Runner,
  tool
} from '@openai/agents';


const brief = process.argv[2];
const missing = [
  ...(brief ? [] : ['a brief (the first argument)']),
  ...['IMGLY_CESDK_LICENSE', 'EXECUTOR_TOKEN'].filter((n) => !process.env[n])
];
if (missing.length > 0) {
  console.error(
    `Missing ${missing.join(', ')}.\nUsage: npm start -- "<brief>", with the variables of .env.template set in .env` +
      (process.env.IMGLY_CESDK_LICENSE
        ? ''
        : '\nContact sales (https://img.ly/forms/contact-sales/) to get a CoDesign license key.')
  );
  process.exit(1);
}


// IMG.LY's AI gateway speaks OpenAI's Chat Completions; your CoDesign license key is its API key.
const imglyGateway = new OpenAIProvider({
  baseURL: 'https://gateway.img.ly/v1',
  apiKey: process.env.IMGLY_CESDK_LICENSE,
  useResponses: false // the gateway has Chat Completions, not OpenAI's Responses API
});


const codesign = prepareCoDesign({
  // Your id for this job. Every design of a conversation opens on one executor
  // that serves no other conversation. Every run here is yours, so all share one.
  conversation: 'quickstart',
  executors: remoteExecutors({
    // The executor for a conversation. Here there is one; a product starts a
    // new one for each conversation.
    acquire: () => ({
      url: process.env.EXECUTOR_URL || 'http://localhost:8080'
    }),
    token: process.env.EXECUTOR_TOKEN
  }),
  // The CE.SDK plugins the engine applies: everything the agent can search and
  // place. Google Fonts is the smallest useful list; your own plugins go here.
  plugins: [googleFonts()],
  design: `quickstart-${Date.now()}`, // any id; with no `designs` store, the design lives in memory
  gateway: false // no image generation; the model still runs through the gateway
});


const textOf = (parts) =>
  parts
    .map((p) => (p.type === 'text' ? p.text : ''))
    .join('\n')
    .trim();


// The SDK's tools in the Agents SDK's shape.
const tools = codesign.tools.map((t) =>
  tool({
    name: t.name,
    description: t.description ?? '',
    // Plain JSON Schema, sent as it is: strict mode would make every property required.
    parameters:
      /** @type {{ type: 'object', properties: {}, required: [], additionalProperties: true }} */ (
        t.inputSchema
      ),
    strict: false,
    execute: async (args, _context, details) => {
      const result = await t.call(args ?? {}, { signal: details?.signal });
      console.log(`${result.isError ? '✗' : '✓'} ${t.name}`);
      if (result.isError) return textOf(result.content); // the refusal, in words the model can act on
      return result.content.map((p) =>
        p.type === 'text'
          ? { type: /** @type {const} */ ('text'), text: p.text }
          : {
              type: /** @type {const} */ ('image'),
              image: { data: p.data, mediaType: p.mimeType }
            }
      );
    },
    // A call the SDK rejects (the executor does not answer) ends the run with
    // its error; by default the model would get a generic message and go on.
    errorFunction: null
  })
);


// Chat Completions has no images in a tool result: move each render into a
// user message after it, so the model sees its work. Left out, nothing fails;
// the model designs blind.
/** @type {import('@openai/agents').CallModelInputFilter} */
const imagesAfterToolResults = ({ modelData }) => {
  /** @type {import('@openai/agents').AgentInputItem[]} */
  const input = [];
  /** @type {{ type: 'input_image', image: string }[]} */
  let images = [];
  const flush = () => {
    if (images.length === 0) return;
    input.push({
      role: 'user',
      content: [
        { type: 'input_text', text: 'Attached image(s) from tool result:' },
        ...images
      ]
    });
    images = [];
  };
  for (const item of modelData.input) {
    if (item.type !== 'function_call_result' || !Array.isArray(item.output)) {
      if (item.type !== 'function_call_result') flush();
      input.push(item);
      continue;
    }
    const kept = item.output.filter((part) => part.type !== 'input_image');
    for (const part of item.output) {
      if (part.type === 'input_image' && typeof part.image === 'string')
        images.push({ type: 'input_image', image: part.image });
    }
    input.push({
      ...item,
      output:
        kept.length > 0
          ? kept
          : [
              {
                type: /** @type {const} */ ('input_text'),
                text: '(see attached image)'
              }
            ]
    });
  }
  flush();
  return { ...modelData, input };
};


const agent = new Agent({
  name: 'Designer',
  instructions: `You design marketing visuals. When you are done, say in one sentence what you made.\n\n${codesign.instructions}`,
  model: process.env.MODEL || '@openrouter/anthropic/claude-opus-5',
  tools
});


const runner = new Runner({
  modelProvider: imglyGateway,
  callModelInputFilter: imagesAfterToolResults,
  tracingDisabled: true // the Agents SDK would otherwise send a trace of every run to OpenAI
});


const started = Date.now();
const seconds = () => Math.round((Date.now() - started) / 1000);
try {
  const run = await runner.run(agent, brief, { maxTurns: 60 }); // the default is 10
  console.log(`\n${run.finalOutput}`);
  const { inputTokens, outputTokens } = run.runContext.usage;
  console.log(
    `${run.rawResponses.length} steps, ${seconds()} s, ${inputTokens} input / ${outputTokens} output tokens`
  );
  await mkdir('out', { recursive: true });
  await writeFile('out/design.png', await codesign.export('png'));
  await writeFile('out/design.imgly', await codesign.export('imgly'));
  console.log('Wrote out/design.png and out/design.imgly');
} catch (err) {
  // The Agents SDK's errors carry the whole run's state; its message is enough.
  const what =
    err instanceof MaxTurnsExceededError
      ? 'The model stopped early'
      : 'The run failed';
  console.error(
    `${what} after ${seconds()} s: ${err instanceof Error ? err.message : err}; nothing exported.`
  );
  process.exitCode = 1;
} finally {
  await codesign.release(); // ends the design's session now; the engine closes about 30 s later
}

What to Replace#

Piece In the kit In your product
The agent runner design.mjs, run once from a terminal with a brief your server or job, with any agent framework, on the gateway with your license key
The tools codesign.tools the same
The executor one container from npm run executor, shared by every run IMG.LY’s executor image, a new container for each conversation, answered by acquire
The design the runner’s memory, then two files in out/ your database, through designs ( Connect Your Storage)
Fonts and media googleFonts() your own CE.SDK plugin in plugins ( Bring Your Brand)

To grow the script into your product:

  • To start from a customer’s file, such as a Photoshop file (PSD), a PDF, or an .imgly file, call codesign.load(bytes) before the model runs.
  • To let the model hand files to a person, pass uploadExport: a function that stores an exported file and answers where the person gets it.
  • To let the model generate images, pass gateway: { bearer }, a function that returns your key.

For each of these options, see the API Reference.

Troubleshooting#

The Executor Does Not Start#

  • docker: open .env: no such file or directory. Copy .env.template to .env in the kit’s folder and set your key.
  • The container name "/codesign-executor" is already in use. The executor is already running: check curl http://localhost:8080/health, or run npm run executor:stop first.
  • /health does not answer. Run docker logs codesign-executor. The executor stops at start when the license key is missing or cannot be activated, or when it cannot reach api.img.ly.

The Run Fails#

  • node: .env: not found. Copy .env.template to .env in the kit’s folder.
  • Missing a brief (the first argument). Pass the brief after the command, in quotes, as in step 5.
  • Missing IMGLY_CESDK_LICENSE, EXECUTOR_TOKEN. Set both in .env.
  • missing or wrong executor token. The executor answered 401: it was started with another EXECUTOR_TOKEN. After you change the token, run npm run executor:stop and npm run executor again, so the executor gets it.
  • The run failed after … s: … the executor at http://localhost:8080 is unreachable. The executor is not running, or the run started while it restarts after the last run. Wait until curl http://localhost:8080/health answers.
  • The run failed after … s: 402 …. The gateway refused the model call: the account behind the key is out of credits (insufficient_credits).
  • The model stopped early. The model reached the 60-turn limit of maxTurns. Nothing was exported.

For more failures and their fixes, see Troubleshooting.

Next Steps#