Search Docs
Loading...
Skip to content

Quickstart: LangChain.js

Type a brief and get a finished PNG and an editable design file from a design agent on LangChain.js, in around 3 minutes, depending on the model and how hard the brief is.

12 mins
estimated time
Download
GitHub

To have a coding agent, such as Claude Code or Cursor, set up this kit for you, give it this prompt:

Open https://img.ly/docs/cesdk/codesign/starter-kits/quickstart-langchain-cdsqlc.md
and set up this starter kit for me.

What It Does#

You run one command with a brief, such as “A square Instagram post for a spring sale on ceramic mugs”. An AI model designs it, looks at what it made, and improves it, then the kit writes two files:

  • out/design.png, a picture of the design.
  • out/design.imgly, the design itself with its images and fonts, which you can open and edit in CE.SDK’s web editor ( Your Editor). CE.SDK is IMG.LY’s design engine and editor.

Start from this kit if your product runs a design agent on its servers: a chat, a job queue, or an API. Two things run on your machine:

  • The executor, a Docker container from the image imgly/codesign-executor, which IMG.LY publishes. It holds CE.SDK’s engine and runs the short JavaScript programs the model writes to change the design, in its own container, apart from your keys and your data. You write no code for it.
  • The agent runner, design.mjs, the kit’s code and later yours. It gets the design tools from the CoDesign SDK, @imgly/codesign-sdk, and hands them to a LangChain agent. A tool is a function the model can ask to call, such as one that changes the design or renders a preview. LangChain’s agent runs the agent loop: it calls the model, runs each tool the model asks for, and sends the result back, until the model is done. Then the script exports the design.

The model runs through IMG.LY’s AI gateway, a service that forwards model calls to the model provider and charges them to your IMG.LY account, so you need no account with a model provider.

Pre-requisites#

  • Docker, to run the executor – Download
  • Node.js v22+ with npm, pnpm, or yarn – Download
  • A CoDesign license key. Contact sales to get a CoDesign license key. That one key starts the design engine, pays for each exported file, and pays for the model through the gateway.

Get Started#

Step 1: Clone the Repository#

Terminal 
git clone https://github.com/imgly/starterkit-codesign-quickstart-langchain-js-node.git
git clone https://github.com/imgly/starterkit-codesign-quickstart-langchain-js-node.git

What is inside:

starterkit-codesign-quickstart-langchain-js-node/
├── design.mjs           # The agent runner: prepares the tools, hands them to a LangChain agent, exports the design
├── .env.template        # The variables to copy into .env
├── README.md            # How to run the kit
├── LICENSE.md           # IMG.LY's license, the same as the CoDesign SDK's
├── package.json         # The SDK, LangChain, and its OpenAI chat model, at exact versions; `npm run executor` starts the executor
├── tsconfig.json        # `npm run check` type-checks design.mjs
└── tsconfig.base.json   # The compiler settings tsconfig.json extends

Read design.mjs after your first run; How It Works below explains it.

Step 2: Install Dependencies#

Terminal 
cd starterkit-codesign-quickstart-langchain-js-node
npm install
cd starterkit-codesign-quickstart-langchain-js-node
npm install

package.json pins @imgly/codesign-sdk to the same version as the executor image’s tag in its executor script. Keep the two equal, because the SDK and the executor must come from the same build ( Versions and Upgrades).

Step 3: Set Your Keys#

Copy .env.template to .env and fill in one variable: IMGLY_CESDK_LICENSE holds your CoDesign license key (the variable keeps CE.SDK’s name). EXECUTOR_TOKEN, the token your runner and the executor share, is already filled in:

.env
IMGLY_CESDK_LICENSE=<your CoDesign license key>
EXECUTOR_TOKEN=codesign-quickstart-token

The executor script passes .env to the executor, and the start script to design.mjs, so the two always have the same token. Two optional variables are described in .env.template:

Variable What it is
MODEL The gateway’s model id. Default @openrouter/anthropic/claude-opus-5. The gateway lists its ids at https://gateway.img.ly/v1/models.
EXECUTOR_URL The executor’s address. Default http://localhost:8080.

Step 4: Start the Executor#

Terminal 
npm run executor
curl http://localhost:8080/health
npm run executor
curl http://localhost:8080/health

The executor script runs IMG.LY’s image in Docker with the variables from .env, on port 8080 of your machine only. The first start downloads the image. curl prints a line like this one:

{"result":{"sessions":0,"accepting":true,"bound":false,...}}

If /health does not answer, run docker logs codesign-executor to see why. The executor stops at start when the license key is missing or cannot be activated, or when it cannot reach api.img.ly.

Step 5: Design Something#

Terminal 
npm start – “A square Instagram post for a 20% spring sale on ceramic mugs. Calm, premium, lots of white space.”
npm start – “A square Instagram post for a 20% spring sale on ceramic mugs. Calm, premium, lots of white space.”

A run takes around 3 minutes, depending on the model and how hard the brief is. You see a line for each tool call as it happens, a ✓ and the tool’s name, as the model reads IMG.LY’s design rules, changes the design, and looks at a render, a picture of the design. Then you see the model’s one-sentence summary and a line with the number of steps, the seconds, and the tokens used. The last line is:

Wrote out/design.png and out/design.imgly

Open out/design.png to see the design. out/design.imgly opens in CE.SDK’s web editor, which you embed in a page of your product ( Your Editor). Use exactly the CE.SDK version the executor runs, which the SDK exports as CESDK_VERSION: an older or newer editor that saves the file drops what the other version wrote ( Versions and Upgrades).

To stop and remove the executor, run npm run executor:stop.

Run It Again#

The executor serves one conversation: your id for one chat or job, which design.mjs sets to quickstart on every run. At the end of a run, the script calls release(), which closes the design on the executor. Half a minute later the executor exits, and Docker starts it again, ready for your next run.

A run that starts in the few seconds while the container restarts fails with executor_unreachable. Wait until curl http://localhost:8080/health answers again.

Before you run the executor anywhere but your machine, follow Host the Executor.

What It Costs#

Each run’s PNG export and the model’s tokens are charged to the IMG.LY account behind your key; the .imgly file, the edits, and the previews are free ( What It Costs).

How It Works#

design.mjs does five things. The excerpts below are from it, and the whole file follows them.

  1. It calls the model through IMG.LY’s gateway. The gateway at https://gateway.img.ly/v1 speaks OpenAI’s Chat Completions, the request format most model providers accept. Your license key is its API key. LangChain reaches it with its OpenAI chat model, ChatOpenAI, given the gateway’s address as baseURL.
// IMG.LY's AI gateway speaks OpenAI's Chat Completions; your CoDesign license key is its API key.
const model = new ChatOpenAI({
  model: process.env.MODEL || '@openrouter/anthropic/claude-opus-5',
  apiKey: process.env.IMGLY_CESDK_LICENSE,
  configuration: { baseURL: 'https://gateway.img.ly/v1' }
});
  1. It prepares the tools with prepareCoDesign.
    • conversation is your id for the work. Every design of a conversation opens on one executor that serves no other conversation, because the model’s code runs there; see Security.
    • executors: remoteExecutors({ acquire, token }) says where a conversation’s executor is. acquire is a function you write. Here it always answers the container on port 8080. In production, it answers a new container for each new conversation, and the same one while the conversation lasts.
    • plugins lists the CE.SDK plugins the engine applies. A plugin adds asset sources: searchable lists of what a design can use, such as photos, typefaces, and colors. googleFonts() is IMG.LY’s Google Fonts catalog, the smallest useful list.
    • design is the id of the design the tools work on. With no designs store, the design lives in the runner’s memory for the run.
    • gateway: false leaves out image generation. The model still runs through the gateway. The option is required, so you decide.
const codesign = prepareCoDesign({
  // Your id for this job. Every design of a conversation opens on one executor
  // that serves no other conversation. Every run here is yours, so all share one.
  conversation: 'quickstart',
  executors: remoteExecutors({
    // The executor for a conversation. Here there is one; a product starts a
    // new one for each conversation.
    acquire: () => ({
      url: process.env.EXECUTOR_URL || 'http://localhost:8080'
    }),
    token: process.env.EXECUTOR_TOKEN
  }),
  // The CE.SDK plugins the engine applies: everything the agent can search and
  // place. Google Fonts is the smallest useful list; your own plugins go here.
  plugins: [googleFonts()],
  design: `quickstart-${Date.now()}`, // any id; with no `designs` store, the design lives in memory
  gateway: false // no image generation; the model still runs through the gateway
});
  1. It hands the tools to LangChain. Each SDK tool is { name, description, inputSchema, call }, and LangChain’s tool() takes the function, the name, the description, and the input’s JSON Schema as it is. The function returns the result as LangChain content blocks, the pieces of a message, text and images, so the renders reach the model as pictures, not as text. A failed call is thrown, and the agent hands the message to the model as a failed call, so it can try again.
// The SDK's tools in LangChain's shape: a JSON Schema for the input, and the
// result as LangChain's text and image content blocks.
const tools = codesign.tools.map((t) =>
  tool(
    async (args, config) => {
      const result = await t.call(args, { signal: config?.signal });
      console.log(`${result.isError ? '✗' : '✓'} ${t.name}`);
      const content = result.content.map((p) =>
        p.type === 'text'
          ? { type: /** @type {const} */ ('text'), text: p.text }
          : {
              type: /** @type {const} */ ('image'),
              mimeType: p.mimeType,
              data: p.data
            }
      );
      if (result.isError)
        throw new Error(content.map((p) => p.text ?? '').join('\n')); // the model reads it as a failed call
      return content;
    },
    {
      name: t.name,
      description: t.description ?? '',
      schema: /** @type {Record<string, unknown>} */ (t.inputSchema)
    }
  )
);
  1. It lets the model see its renders. Chat Completions allows no images inside a tool result, and ChatOpenAI sends a tool result’s image blocks as they are, in a shape Chat Completions does not define. The kit’s imagesAfterToolResults runs in a LangChain middleware, a function that runs around each model call (its wrapModelCall hook). Before the call, it moves each render into a user message after the tool result, as OpenAI’s image_url with the image’s bytes. Without that step, the model does not see its work.
// Chat Completions has no images in a tool result: move each render into a
// user message after it, in OpenAI's image_url shape, so the model sees its work.
function imagesAfterToolResults(messages) {
  const out = [];
  let images = [];
  const flush = () => {
    if (images.length === 0) return;
    out.push(
      new HumanMessage({
        content: [
          { type: 'text', text: 'Attached image(s) from tool result:' },
          ...images
        ]
      })
    );
    images = [];
  };
  for (const message of messages) {
    if (
      !ToolMessage.isInstance(message) ||
      typeof message.content === 'string'
    ) {
      flush();
      out.push(message);
      continue;
    }
    const isImage = (block) => block.type === 'image';
    const kept = message.content.filter((block) => !isImage(block));
    if (kept.length === message.content.length) {
      out.push(message);
      continue;
    }
    for (const image of message.content.filter(isImage)) {
      images.push({
        type: 'image_url',
        image_url: { url: `data:${image.mimeType};base64,${image.data}` }
      });
    }
    out.push(
      new ToolMessage({
        tool_call_id: message.tool_call_id,
        name: message.name,
        status: message.status,
        content:
          kept.length > 0
            ? kept
            : [{ type: 'text', text: '(see attached image)' }]
      })
    );
  }
  flush();
  return out;
}


const showRenders = createMiddleware({
  name: 'ShowRenders',
  wrapModelCall: (request, handler) =>
    handler({ ...request, messages: imagesAfterToolResults(request.messages) })
});
  1. It runs the agent. createAgent from the langchain package takes the model, the tools, the middleware, and the system prompt, with codesign.instructions after your own: they tell the model how to use the tools, so your prompt names none of them. The brief goes in as the first message. recursionLimit: 120 lets the model take up to 60 turns, because LangChain counts a model call and the tool calls that follow it as two steps; its default of 25 ends a design run after about 12 turns, with an error. When the model is done, the script exports the PNG and the .imgly file and calls release().
The whole file
design.mjs
// A brief in, a finished design out: LangChain.js runs the model through IMG.LY's gateway, the engine runs in one local executor.
import { mkdir, writeFile } from 'node:fs/promises';


import {
  googleFonts,
  prepareCoDesign,
  remoteExecutors
} from '@imgly/codesign-sdk';
import { ChatOpenAI } from '@langchain/openai';
import {
  AIMessage,
  createAgent,
  createMiddleware,
  HumanMessage,
  tool,
  ToolMessage
} from 'langchain';


const brief = process.argv[2];
const missing = [
  ...(brief ? [] : ['a brief (the first argument)']),
  ...['IMGLY_CESDK_LICENSE', 'EXECUTOR_TOKEN'].filter((n) => !process.env[n])
];
if (missing.length > 0) {
  console.error(
    `Missing ${missing.join(', ')}.\nUsage: npm start -- "<brief>", with the variables of .env.template set in .env` +
      (process.env.IMGLY_CESDK_LICENSE
        ? ''
        : '\nContact sales (https://img.ly/forms/contact-sales/) to get a CoDesign license key.')
  );
  process.exit(1);
}


// IMG.LY's AI gateway speaks OpenAI's Chat Completions; your CoDesign license key is its API key.
const model = new ChatOpenAI({
  model: process.env.MODEL || '@openrouter/anthropic/claude-opus-5',
  apiKey: process.env.IMGLY_CESDK_LICENSE,
  configuration: { baseURL: 'https://gateway.img.ly/v1' }
});


const codesign = prepareCoDesign({
  // Your id for this job. Every design of a conversation opens on one executor
  // that serves no other conversation. Every run here is yours, so all share one.
  conversation: 'quickstart',
  executors: remoteExecutors({
    // The executor for a conversation. Here there is one; a product starts a
    // new one for each conversation.
    acquire: () => ({
      url: process.env.EXECUTOR_URL || 'http://localhost:8080'
    }),
    token: process.env.EXECUTOR_TOKEN
  }),
  // The CE.SDK plugins the engine applies: everything the agent can search and
  // place. Google Fonts is the smallest useful list; your own plugins go here.
  plugins: [googleFonts()],
  design: `quickstart-${Date.now()}`, // any id; with no `designs` store, the design lives in memory
  gateway: false // no image generation; the model still runs through the gateway
});


// The SDK's tools in LangChain's shape: a JSON Schema for the input, and the
// result as LangChain's text and image content blocks.
const tools = codesign.tools.map((t) =>
  tool(
    async (args, config) => {
      const result = await t.call(args, { signal: config?.signal });
      console.log(`${result.isError ? '✗' : '✓'} ${t.name}`);
      const content = result.content.map((p) =>
        p.type === 'text'
          ? { type: /** @type {const} */ ('text'), text: p.text }
          : {
              type: /** @type {const} */ ('image'),
              mimeType: p.mimeType,
              data: p.data
            }
      );
      if (result.isError)
        throw new Error(content.map((p) => p.text ?? '').join('\n')); // the model reads it as a failed call
      return content;
    },
    {
      name: t.name,
      description: t.description ?? '',
      schema: /** @type {Record<string, unknown>} */ (t.inputSchema)
    }
  )
);


// Chat Completions has no images in a tool result: move each render into a
// user message after it, in OpenAI's image_url shape, so the model sees its work.
function imagesAfterToolResults(messages) {
  const out = [];
  let images = [];
  const flush = () => {
    if (images.length === 0) return;
    out.push(
      new HumanMessage({
        content: [
          { type: 'text', text: 'Attached image(s) from tool result:' },
          ...images
        ]
      })
    );
    images = [];
  };
  for (const message of messages) {
    if (
      !ToolMessage.isInstance(message) ||
      typeof message.content === 'string'
    ) {
      flush();
      out.push(message);
      continue;
    }
    const isImage = (block) => block.type === 'image';
    const kept = message.content.filter((block) => !isImage(block));
    if (kept.length === message.content.length) {
      out.push(message);
      continue;
    }
    for (const image of message.content.filter(isImage)) {
      images.push({
        type: 'image_url',
        image_url: { url: `data:${image.mimeType};base64,${image.data}` }
      });
    }
    out.push(
      new ToolMessage({
        tool_call_id: message.tool_call_id,
        name: message.name,
        status: message.status,
        content:
          kept.length > 0
            ? kept
            : [{ type: 'text', text: '(see attached image)' }]
      })
    );
  }
  flush();
  return out;
}


const showRenders = createMiddleware({
  name: 'ShowRenders',
  wrapModelCall: (request, handler) =>
    handler({ ...request, messages: imagesAfterToolResults(request.messages) })
});


const agent = createAgent({
  model,
  tools,
  systemPrompt: `You design marketing visuals. When you are done, say in one sentence what you made.\n\n${codesign.instructions}`,
  middleware: [showRenders]
});


const started = Date.now();
let messages;
try {
  // A model call and its round of tool calls are two steps of the agent's
  // graph, so 120 lets the model take 60 turns. The default, 25, ends a design run early.
  ({ messages } = await agent.invoke(
    { messages: [new HumanMessage(brief)] },
    { recursionLimit: 120 }
  ));
} catch (err) {
  console.error(
    `The run ended with an error after ${Math.round((Date.now() - started) / 1000)} s: ${err instanceof Error ? err.message : err}; nothing exported.`
  );
  await codesign.release();
  process.exit(1);
}
const answers = messages.filter((m) => AIMessage.isInstance(m));
const last = answers.at(-1);
console.log(`\n${last?.text ?? ''}`);
const usage = (key) =>
  answers.reduce((sum, m) => sum + (m.usage_metadata?.[key] ?? 0), 0);
console.log(
  `${answers.length} steps, ${Math.round((Date.now() - started) / 1000)} s, ` +
    `${usage('input_tokens')} input / ${usage('output_tokens')} output tokens`
);
await mkdir('out', { recursive: true });
await writeFile('out/design.png', await codesign.export('png'));
await writeFile('out/design.imgly', await codesign.export('imgly'));
console.log('Wrote out/design.png and out/design.imgly');
await codesign.release(); // ends the design's session now; the engine closes about 30 s later

What to Replace#

Piece In the kit In your product
The agent runner design.mjs, run once from a terminal with a brief your server or job, with any agent framework, on the gateway with your license key
The tools codesign.tools the same
The executor one container from npm run executor, shared by every run IMG.LY’s executor image, a new container for each conversation, answered by acquire
The design the runner’s memory, then two files in out/ your database, through designs ( Connect Your Storage)
Fonts and media googleFonts() your own CE.SDK plugin in plugins ( Bring Your Brand)

To grow the script into your product:

  • To start from a customer’s file, such as a Photoshop file (PSD), a PDF, or an .imgly file, call codesign.load(bytes) before the model runs.
  • To let the model hand files to a person, pass uploadExport: a function that stores an exported file and answers where the person gets it.
  • To let the model generate images, pass gateway: { bearer }, a function that returns your key.

For each of these options, see the API Reference.

Troubleshooting#

The Executor Does Not Start#

  • docker: open .env: no such file or directory. Copy .env.template to .env in the kit’s folder and set your key.
  • The container name "/codesign-executor" is already in use. The executor is already running: check curl http://localhost:8080/health, or run npm run executor:stop first.
  • /health does not answer. Run docker logs codesign-executor. The executor stops at start when the license key is missing or cannot be activated, or when it cannot reach api.img.ly.

The Run Fails#

  • node: .env: not found. Copy .env.template to .env in the kit’s folder.
  • Missing a brief (the first argument). Pass the brief after the command, in quotes, as in step 5.
  • Missing IMGLY_CESDK_LICENSE, EXECUTOR_TOKEN. Set both in .env.
  • missing or wrong executor token. The executor answered 401: it was started with another EXECUTOR_TOKEN. After you change the token, run npm run executor:stop and npm run executor again, so the executor gets it.
  • executor_unreachable. The executor is not running, or the run started while it restarts after the last run. Wait until curl http://localhost:8080/health answers.
  • The gateway answers 402 insufficient_credits. The account behind the key is out of credits. The script stops with the error.
  • The run ended with an error. The message after it says why, such as the gateway refusing a call, or Recursion limit of 120 reached when the model took more than 60 turns. Nothing was exported.

For more failures and their fixes, see Troubleshooting.

Next Steps#