Search Docs
Loading...
Skip to content

Export Video

Let the agent turn a brief into a video or a sound file your users can download: MP4 or WebM video, and M4A, WAV, or WebM audio. Switch video on in the executor, and plan for length, export time, and cost.

This guide covers what the agent can make, how to switch video on, which codecs it uses, how long an export takes, and what it costs. No example product runs the code on this page; it is checked against the CoDesign SDK’s types, and each function marked Yours is yours to write. It assumes you run the executor as in Host the Executor.

What the Agent Can Make#

A video design is a design whose pages have a duration: each page is a timeline of clips, images, text, shapes, and animations, with music or speech under it. The agent builds one from a brief, or from an animation you import (a Lottie file opens as a video design; Import and Export Files). It renders one page to one file:

Format What the file holds Use it for
mp4 H.264 video and AAC audio Social posts, web pages, and players
webm VP9 video and Opus audio Web pages, and video with transparency
m4a The page’s sound as AAC Podcasts and players
weba The page’s sound as Opus, in WebM Web pages
wav The page’s sound, uncompressed Editing the sound elsewhere

The model picks the size and the frame rate, so a person can ask for “a 9:16 story, 15 seconds”. It can also render an image of any moment of a video, which is how it checks its own work. An MP4 always has a background; a WebM keeps a page without a background transparent when the model asks for it (alpha: true).

Every video or audio export has these limits:

  • At most 60 seconds long. A longer page is refused before it renders.
  • At most as much work as 60 seconds of 1080p at 30 frames per second. Work is frames times pixels, so a 4K page (3840 × 2160) can be 15 seconds long. A longer or larger export is refused with a message that says what to change: shorter, smaller, or fewer frames per second.
  • At most 150 MB per file, like every export.
  • Video and audio files that go into a design can be up to 1 GB each.

The encoder also refuses some very large frames. The agent then exports again at a size the message gives, with the same aspect ratio.

Switch Video On#

Video runs on the video image, imgly/codesign-executor:1.86.0-video, on x64 (amd64) hosts. It is the same executor, with video on and the codecs video needs, and it renders on an NVIDIA GPU when the container has one. On an arm64 machine, such as a Mac with Apple silicon, try it under emulation (--platform linux/amd64), which is far slower. The image without video, imgly/codesign-executor:1.86.0, refuses every video or audio export with video_unsupported. The memory video needs and the GPU setup are in Run It With Video.

At start, the executor encodes a tiny video to check that video works. When it does, the executor logs the formats it exports:

engine: video: mp4, webm, wav, m4a, weba

If it does not, the executor starts without video and logs one line, which starts with engine: video:, that says why.

Your runner can ask before the first turn. codesign.handshake() opens the conversation’s engine session and returns what the executor can do:

// Before the person types: does this conversation's executor encode video?
export async function videoFormats(codesign: CoDesign): Promise<string[]> {
const { capabilities } = await codesign.handshake();
if (!capabilities.video) return []; // the image without video, or one whose engine could not encode
// 'mp4', 'm4a', and 'wav'
return capabilities.exportFormats.filter((format) =>
['mp4', 'm4a', 'wav'].includes(format)
);
}

capabilities.video is true when the executor encodes video, and capabilities.exportFormats lists every format it can render. The model reads the same answer: on an executor without video, a video export it asks for is refused with a sentence that says video is not available here.

Nothing else changes in your runner. The model’s video exports reach your uploadExport function like any other file, with the format’s MIME type (video/mp4, video/webm, audio/mp4, audio/webm, or audio/wav) ( Connect Your Storage).

Store Large Files Directly#

A video can be tens of megabytes. With uploadExport, each file passes through your runner’s memory. If your runner has little memory, such as a serverless function, let the executor store the file itself:

  1. Pass uploadExportURL to prepareCoDesign instead of uploadExport. It answers, before the render, a presigned upload URL of your bucket and the link the person opens ( uploadExportURL).
  2. Pass maxExportBytes to remoteExecutors, such as 100_000_000. A file larger than that never comes back to your runner: a codesign.export() call that asks for the bytes fails with export_too_large, uncharged.
  3. In your own code, such as a batch job, export with codesign.export('mp4', { upload: true }). The executor stores the file through the same uploadExportURL, and the call returns its downloadURL and size, with the length of a video or audio file ( export(format)).
  4. Allow your bucket’s upload host in the executor’s egress ( Allow Only the Hosts the Executor Needs).

The executor charges the file once it is rendered, before the upload. So if the egress does not allow your bucket, every export fails with export_upload_failed and is charged anyway (charged: true, unless Metering was unreachable). Run one export with your upload URL before you go live. Exporting it again on the conversation’s same executor is not charged again, and within 5 minutes it uploads the file already rendered instead of rendering it again; on a new executor, after the old one ended, a second charge is possible.

Codecs#

The video image encodes MP4 and M4A with open-source H.264 and AAC encoders. At its first export, the engine writes this warning to the executor’s log:

engine: [!] IMPORTANT: Open Source Codec Usage Warning
engine: The open source codecs you're using are intended for TESTING and
engine: DEVELOPMENT purposes only.
engine: [X] IMG.LY provides NO support for these codecs
engine: [X] IMG.LY makes NO guarantees about their legal compliance
engine: [X] Use in production at your own risk

It means what it says: IMG.LY gives no support and no legal guarantee for these encoders, which come with no patent license for H.264 and AAC. Before you use MP4 or M4A in production, check the patent terms that apply to you. Encoders with a license are not supported yet ( Status). The video image also encodes royalty-free WebM (VP9 and Opus): the model exports webm and weba there, and so do your own codesign.export('webm') and codesign.export('weba') calls. An executor without WebM refuses it, and the model is told the nearest format it has. wav is uncompressed and needs no codec. The video image also contains FFmpeg’s libraries in their GPL build, with x264 and x265, to decode clips ( Codecs in the Video Image). The image without video contains no codecs at all.

Every H.264 file is Constrained Baseline, the profile every player decodes.

Clips in a design can be H.264 or H.265 video, 10-bit, 4:2:2, and interlaced footage included, with MP3 or AAC sound. The executor decodes them on the CPU, with or without a GPU.

Plan for Export Time#

Video exports take seconds, not milliseconds. With the 2 vCPUs a video executor gets, these are typical times for one 1080p MP4 at 30 frames per second:

Export 10 seconds long 30 seconds long
Text and shapes about 2 seconds about 8 seconds
A full-frame video clip under a blur, with text about 7 seconds about 20 seconds
An image of one moment under a second

Times vary with what is on the page. Each video clip in the design adds decoding work, and effects over footage, such as a blur, add rendering work. A page with a large photo can take several times as long as the same page with text and shapes, so plan no fixed time for photo-heavy videos. On an NVIDIA GPU, the video image exports 1.5 to 2.5 times faster, the most for long videos with effects over footage ( Video on an NVIDIA GPU). A typical design job takes around 3 minutes, depending on the model and how hard the brief is; a video adds its exports to that.

Each encode has a time budget: 30 seconds, plus more for each frame and each million pixels in it. An encode that runs past its budget is stopped, and so is one that reports no progress for 90 seconds once it has started rendering frames (an audio export reports none, so only its budget holds). The executor then reloads the design on a fresh engine and tries again, up to three times in all; a design with more than 256 MB of embedded media gets one attempt only. The call waits while it does. The largest export allowed has a budget of about 7 minutes, so in the worst case one call takes about 21 minutes, and up to 10 more when it uploads the file to your uploadExportURL. Let your runner’s HTTP client, and any proxy between it and the executor, wait 35 minutes for a call: one that gives up sooner reports a failure for an export that is still stored and charged. You can change the budget with the EXECUTOR_ENCODE_* variables ( The Executor Image’s Environment Variables).

When all three encodes run out of time, the export fails with encode_timeout. The model reads it and tries again shorter or smaller; your own call receives the error:

// Your own export, without a model: the design's first page as an MP4 video.
export async function exportClip(
codesign: CoDesign
): Promise<Uint8Array | undefined> {
try {
return await codesign.export('mp4');
} catch (err) {
if (!isCoDesignError(err)) throw err;
switch (err.kind) {
case 'export_refused': // no credits, or longer than the executor allows
case 'video_unsupported': // this executor runs without video
await tellThePerson(err.message);
return undefined;
case 'encode_timeout': // every encode ran out of time: the first on the design's engine, the rest on fresh ones
case 'encoder_busy': // the encoder did not come free
await tellThePerson('The video took too long. Try a shorter one.');
return undefined;
default:
throw err;
}
}
}
The whole file, video.ts
import { isCoDesignError, type CoDesign } from '@imgly/codesign-sdk';
// Before the person types: does this conversation's executor encode video?
export async function videoFormats(codesign: CoDesign): Promise<string[]> {
const { capabilities } = await codesign.handshake();
if (!capabilities.video) return []; // the image without video, or one whose engine could not encode
// 'mp4', 'm4a', and 'wav'
return capabilities.exportFormats.filter((format) =>
['mp4', 'm4a', 'wav'].includes(format)
);
}
// Your own export, without a model: the design's first page as an MP4 video.
export async function exportClip(
codesign: CoDesign
): Promise<Uint8Array | undefined> {
try {
return await codesign.export('mp4');
} catch (err) {
if (!isCoDesignError(err)) throw err;
switch (err.kind) {
case 'export_refused': // no credits, or longer than the executor allows
case 'video_unsupported': // this executor runs without video
await tellThePerson(err.message);
return undefined;
case 'encode_timeout': // every encode ran out of time: the first on the design's engine, the rest on fresh ones
case 'encoder_busy': // the encoder did not come free
await tellThePerson('The video took too long. Try a shorter one.');
return undefined;
default:
throw err;
}
}
}
// Yours: shows a message in your product.
async function tellThePerson(_message: string): Promise<void> {
throw new Error('yours to write');
}

What Video Costs#

Every video and audio export is charged to your IMG.LY account, like every other file. A video is charged by its length and its frame size, a sound file by its length. Images of one moment of a video, edits, and saving the design are free. For what each export sends to IMG.LY’s billing service, see Billing.

Generating video or sound with AI, such as a clip from a text prompt, is a separate charge, made through IMG.LY’s AI gateway when you pass gateway ( Image Generation).

Troubleshooting#

What you see Why What to do
video_unsupported on every video export The executor runs without video. Its log names why, in a line that starts with engine: video:. Run the video image, imgly/codesign-executor:1.86.0-video, on an x64 host (Switch Video On).
export_refused: “Video and audio exports are limited to 60 seconds” The page is longer than 60 seconds. Ask for a shorter video, or split it into pages.
encode_timeout Three encodes ran out of time. Retry once. If it repeats, export shorter or smaller, and check the container’s CPU (Plan for Export Time).
A container stops during an export It ran out of memory. Each 1080p clip in a design adds up to 480 MB. Give video executors more memory ( Run It With Video).

For every error kind, see Errors; for broken frames and slow exports, see Troubleshooting.

Next Steps#