Jan – IMG.LY Blog

How 600+ Teams Shipped Creative Editing in 14 Days

Jan — Tue, 28 Apr 2026 21:15:17 GMT

The IMG.LY Impact Report

Based on a self-reported customer survey, 28 structured customer interviews, and IMG.LY’s published customer case studies. External benchmarks cited in context.

600+ companies from startups, over Fortune 500s to government organizations run creative editing on IMG.LY, collectively producing 500 million creations every month. The median customer goes from first SDK access to a working prototype in 2 days and a production-ready feature shipped inside a single 14-day sprint. Those self-reported medians run roughly 13× faster than the 6-month industry baseline for building an equivalent editor in-house, and around 6× faster than stitching together an open-source toolkit.

The time-to-market number is the entry ticket. What those 600+ customers do after shipping, how the editor changes their pricing, funnel, team velocity, and competitive position is what this report is about.

Methodology note. Time-to-prototype and time-to-MVP figures are self-reported by IMG.LY customers in a recent customer survey. All named customer outcomes are drawn from IMG.LY’s published case studies, where the customer has reviewed and approved the specific figures cited. Quantitative ranges and anonymous quotes come from 30 structured customer interviews (2024–2026); individual customers are not identified.

What Customers Unlock

1. Ship in days, not quarters

A working prototype takes an average of two days, a production feature often just a single sprint. Customer interviews put the alternative, an in-house build, at 6–12 months with 2–3 senior developers, consistent with the cost bands documented in IMG.LY’s Build vs. Buy guide and independently with the Standish Group CHAOS Report, which finds only 31% of software projects succeed on time, on budget, and with full scope… and under 10% for large projects.

Plai went live in about a month where their founder had estimated “months and months” for an in-house build. Optimizely, serving 1,000+ enterprise customers, described the integration as “one of the quickest projects we’ve worked on.”

Time to a production-ready MVP

14 days vs. 6 months: how long it takes to ship creative editing

Median time from project start to a production-ready creative editor, by integration path.

Sources. CE.SDK figure (14 days) is the self-reported median from an IMG.LY customer survey; individual results vary with scope and team. In-house (180 days) and open-source (90 days) figures are midpoints of the ranges documented in IMG.LY's Build vs. Buy guide, cross-referenced with customer interviews (N=28).

2. Open new pricing and market tiers

Embedding an editor changes the customer’s own unit economics. It unlocks a self-serve tier where users previously needed a designer in the loop, lowers the price floor, and opens markets where manual production was the binding constraint.

Plai doubled annual revenue and now generates 30,000+ ad creatives monthly on the platform. Postbuddy measured a 4× A/B-test improvement after moving direct-mail creation in-app, a lift that funded expansion into Sweden and Norway. In an anonymized interview, one customer described the change plainly: “We can lower our prices for the base product, it’s cheaper for clients who can use self-service.”

3. Lift your own conversion and engagement

In-context editing keeps users inside the product, shortens time-to-value, and removes the “export to Photoshop” detour that quietly leaks trials. The effect shows up directly in sign-up and trial-to-paid numbers.

Omneky recorded a 10× month-over-month increase in new sign-ups after embedding CE.SDK. In a separate interview, one customer described 400–500 organic trials per month with roughly $65–70K of opportunity evaporating at an 8% trial-to-paid rate tied largely to editor friction, the number that made their own build-vs-buy case close itself.

4. Free your design, marketing, and support teams

The cost saved by an editor isn’t only on the engineering line. It shows up in designer queues that disappear, support tickets that drop, and marketing teams that stop waiting on the bottleneck.

ImageBank X compressed a recurring asset-prep task from 15 minutes to 2 minutes — an 87% reduction per task. Halio.ai enables financial advisors to produce 30 days of branded content in 30 minutes. Interview customers describe the same pattern in plainer language: “happier marketing teams,” and, of the post-launch support burden, “the things the sales team had to deal with all day are just gone.”

5. Compound instead of freeze

This is where the engineering-economics story earns its keep. An in-house editor is a snapshot: the team ships what’s feasible in the window, then expends time and money just to keep it alive. Atlassian’s 2024 State of Developer Experience Report (N=2,100+) found 69% of developers lose eight or more hours per week to inefficiencies — about 20% of their time — with technical debt as the primary cause. Stack Overflow’s 2024 Developer Survey (N=65,437) independently found 62% of developers rank technical debt as their single biggest frustration.

CE.SDK customers don’t pay that tax. New capabilities such as AI background removal, text-to-image, object add/remove, style transfer arrive as modular plugins, not re-architectures. One interviewed customer, who had spent a year building their own editor before switching, described maintaining it as “a full-time job and our main product isn’t editing.” Every month an in-house team spends on maintenance is a month IMG.LY customers get new capabilities shipped to them. The feature gap widens, not narrows.

Editor capability over 24 months

In-house builds freeze. CE.SDK customers compound.

An in-house editor ships at the snapshot the team can afford and then plateaus under the weight of maintenance. CE.SDK customers absorb new capabilities — AI background removal, text-to-image, object remove, automatic subtitles — as modular plugins each quarter.

Build in-house

CE.SDK customer

Illustrative. The CE.SDK trajectory reflects the cadence of major capability releases shipped to customers without re-architecture. The in-house trajectory reflects the typical build-then-maintain pattern documented in customer interviews and corroborated by the Atlassian 2024 State of Developer Experience Report (69% of developers lose 8+ hours per week to inefficiencies, primarily technical debt).

6. Become deal-eligible and privacy-credible

Some categories require an editor to win the deal at all. One interviewed customer received a written RFP rejection that read: “the other provider has an editor, you don’t.” That single moment approved their budget to buy.

For regulated buyers, fintech, healthcare, government, enterprise, client-side processing is a procurement requirement, not a feature. CE.SDK’s default client-side execution means customer data never leaves the browser, which makes the editor defensible in categories where server-side alternatives are blocked.

Swiss Post producing 1 million+ personalized postcards annually on the platform described CE.SDK as “the only solution allowing a specialized, on-brand UI.”

The economics, in one table

The cost gap between building and buying isn’t just wider on day one, it widens over time. Every month an in-house team spends maintaining the editor is a month IMG.LY customers compound new capability on top of what’s already shipped. The table below summarizes the 3-year picture. For the full cost methodology, including the open-source path and the senior-developer rate assumptions, see IMG.LY’s Build vs. Buy guide.

	Build In-House	Open Source + Glue	CE.SDK
Working prototype	2–6 months	1–2 months	2 days (self-reported median)
Production MVP	6–12+ months	3–6 months	14 days (self-reported median)
Team required	3–5 senior developers	1–2 senior developers	1 developer for integration
Year-1 development cost	$100K–$500K+	$150K–$400K	License + integration
Annual maintenance	$75K–$125K (15–25%)	$50K–$100K	Included
3-year TCO	$525K–$1.4M+	$300K–$700K+	Fraction of build cost
Cross-platform parity	Build per platform	Typically web-only	Web, iOS, Android, Desktop, Server

Source for build and open-source cost bands: IMG.LY’s Strategic Guide to Creative Editing: When to Build, When to Buy. Cross-referenced against BLS May 2024 wage data - median software-developer wage $133,080, 90th percentile $211,450 - and aggregated maintenance benchmarks (Pegotec, 2026). Time-to-prototype and time-to-MVP for CE.SDK are self-reported medians from an IMG.LY customer survey; individual results vary with scope, team, and platform target.

When building in-house still makes sense

This report defends one conclusion, so here is the honest counterweight.
Build your own creative editor if all of the following are true:

Creative editing is your core product, and your unique IP lives inside the editing experience itself.
Your requirements are genuinely non-standard, i.e. no off-the-shelf SDK supports your rendering or interaction model.
You have surplus senior engineering capacity that isn’t needed on differentiated work elsewhere.
Time-to-market is not competitive in your category, nobody else is shipping similar capability.

If any one of these is false, the economics above generally make buying both cheaper and faster.

Methodology

Time-to-prototype and time-to-MVP (2 days, 14 days) are self-reported medians from an IMG.LY customer survey. Individual results vary with team experience, integration scope, and platform target.

In-house and open-source timeline ranges are drawn from 28 structured customer interviews conducted between 2024 and 2026, cross-referenced against IMG.LY’s Build vs. Buy guide. Interview customers are not identified; quantitative ranges reflect the distribution of answers across the set.

Named customer outcomes (Plai, Omneky, Optimizely, Swiss Post, ImageBank X, Halio.ai, Postbuddy) come from IMG.LY’s published case studies, where the customer has reviewed and approved each cited figure.

External benchmarks are drawn from:

U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, Software Developers (May 2024)
Atlassian State of Developer Experience Report (2024) (N = 2,100+ developers and engineering leaders, in partnership with DX and Wakefield Research)
Stack Overflow 2024 Developer Survey (N = 65,437 developers across 185 countries)
Standish Group CHAOS Report
Aggregated software-maintenance benchmarks from Gartner, IEEE, and industry practitioners, summarized in Pegotec (2026) and Vention Teams.

Frequently asked questions

How long does it take to integrate CE.SDK?
The median customer in our survey reports a working prototype within 2 days and a production-ready MVP within a single 14-day sprint. Self-reported; varies with scope and team.

What do IMG.LY customers unlock beyond faster shipping?
Five additional outcomes: new pricing and market tiers enabled by self-serve editing; lifted trial-to-paid and sign-up conversion; reduced design, marketing, and support load; a roadmap that compounds instead of freezes as AI features ship as plugins; and deal eligibility in procurement categories that require an embedded editor or client-side processing.

Can AI coding agents like Claude Code accelerate CE.SDK integration further?
Yes. IMG.LY publishes a set of specialized CE.SDK Agent Skills portable knowledge packs that give AI coding assistants expert-level understanding of the SDK before they write a line of code. The skills cover framework-specific documentation lookup (React, Vue, Svelte, Angular, Next.js, Nuxt.js, SvelteKit, Electron, Node.js, Vanilla JS), feature implementation (/cesdk:build), conceptual explanations (/cesdk:explain), and a builder agent that autonomously scaffolds complete CE.SDK applications from a natural-language description.

How much does it cost to build a creative editor in-house?
Published industry benchmarks and customer-reported estimates converge on $100K–$500K+ initial development over 6–12 months with 3–5 senior developers, plus 15–25% of that cost annually in maintenance. 3-year TCO typically lands between $525K and $1.4M. See our Build vs. Buy guide for full methodology.

What percentage of developer time is spent on maintenance?
Atlassian’s 2024 State of Developer Experience Report found 69% of developers lose 8+ hours per week to inefficiencies, about 20% of their time, primarily to technical debt. Gartner reports that 55–80% of corporate IT budgets go to maintaining existing systems.

What is the failure rate of custom software projects?
The Standish Group CHAOS research finds only 31% of software projects succeed (on time, on budget, full scope); 50% are challenged, 19% cancelled outright. For large projects specifically, success rates fall below 10%.

Custom software project outcomes

Only 31% of custom software projects succeed

Across tens of thousands of tracked projects, the Standish Group CHAOS research finds the majority of custom software is late, over budget, reduced in scope, or cancelled outright.

31%

Successful

On time, on budget, with full scope.

50%

Challenged

Delivered late, over budget, or with reduced features.

19%

Failed

Cancelled before delivery.

~25% of IMG.LY customers failed at building a creative editor in-house

In our customer interviews (N=28), roughly one in four customers had attempted to build a creative editor in-house — or stitch one together from open-source — and abandoned the attempt before adopting CE.SDK.

Source. Standish Group CHAOS Report. For large projects specifically — the size band most in-house creative-editor builds fall into — success rates drop below 10%. IMG.LY customer figure is drawn from 28 structured customer interviews conducted between 2024 and 2026.

How many companies use IMG.LY CE.SDK?
600+ customers including startups, Fortune 500 companies, and government organizations collectively produce 500 million creations per month on the platform.

What platforms does CE.SDK support?
Web (React, Angular, Vue, Svelte, Next.js, Nuxt.js, SvelteKit, Vanilla JavaScript, Electron), Mobile (iOS/Swift, Android/Kotlin, React Native, Flutter, Ionic, Cordova), and Server (Node.js headless mode for batch automation).

Building a CapCut-Like Video Editor with CE.SDK and AI

Jan — Thu, 26 Feb 2026 22:50:34 GMT

Build this with AI in minutes

This tutorial is a great way to understand how CE.SDK works under the hood. But if you just want to get to the result, you can build this entire editor using IMG.LY Agent Skills, no manual setup required.

Install the skills, then run:

/cesdk:build Build a CapCut-like video editor with a dark theme, multi-track timeline, AI video/image/audio generation, background removal, and MP4 export.

Want to understand a specific concept in depth? Use /cesdk:explain — for example:

/cesdk:explain How does the video timeline and block hierarchy work?

Introduction

CapCut has set the standard for accessible video editing. Its dark UI, intuitive timeline, and AI-powered features make it feel like a professional tool that anyone can use. But what if you could build something similar embedded directly in your own web application in minutes? Yes, minutes!

In this tutorial, we’ll walk through how we built a CapCut-like video editor using IMG.LY’s CreativeEditor SDK (CE.SDK) for React. The finished editor includes:

A multi-track timeline for video, audio, text, and captions
Trim, split, and join operations on video clips
A CapCut-inspired dark theme built with CSS custom properties
AI video generation (text-to-video, image-to-video) via Minimax, Kling, and Pixverse
AI image generation (text-to-image, image editing) via RecraftV3, IdeogramV3, and GPT Image
AI audio generation (text-to-speech, sound effects) via ElevenLabs
AI text generation (copywriting, translation) via Anthropic Claude
Background removal — client-side, powered by WebAssembly/WebGPU
MP4 export directly from the browser

The entire editor is a single React component with roughly 180 lines of JavaScript and 100 lines of CSS.

Architecture Overview

CE.SDK has two main layers:

CreativeEngine: The headless core that manages scenes, blocks, assets, and rendering. It handles the video timeline, playback, and export entirely client-side.
CreativeEditor UI: A pre-built, customizable UI layer that wraps the engine with a dock, inspector, timeline, canvas, and navigation bar.

Plugins extend both layers. The AiApps plugin, for example, registers AI providers with the engine and injects UI components (dock buttons, canvas menu items, generation panels) into the editor.

Step 1: Project Setup

We scaffolded a React project with Vite and installed CE.SDK alongside the AI plugin packages:

npm create vite@latest capcut-like-editor -- --template react
cd capcut-like-editor
npm install

# Core SDK
npm install @cesdk/cesdk-js

# AI plugins
npm install @imgly/plugin-ai-apps-web
npm install @imgly/plugin-ai-video-generation-web
npm install @imgly/plugin-ai-image-generation-web
npm install @imgly/plugin-ai-audio-generation-web
npm install @imgly/plugin-ai-text-generation-web

# Background removal (runs locally via WASM/WebGPU)
npm install @imgly/plugin-background-removal-web onnxruntime-web@1.21.0

Why so many packages?

CE.SDK follows a modular plugin architecture. Each AI capability is a separate package with its own provider modules. This means you only bundle what you use if you don’t need audio generation, don’t install @imgly/plugin-ai-audio-generation-web. The @imgly/plugin-ai-apps-web package is the unifying layer that brings them all together into a single dock panel.

Step 2: The React Component

CE.SDK provides a first-class React wrapper via @cesdk/cesdk-js/react. The <CreativeEditor> component handles mounting, initialization, and cleanup:

import CreativeEditor from '@cesdk/cesdk-js/react';

const config = {
  // license: 'YOUR_CESDK_LICENSE_KEY',
};

const init = async (cesdk) => {
  // All setup happens here — theme, assets, plugins, UI customization
};

export default function VideoEditor() {
  return (
    <CreativeEditor config={config} init={init} width="100vw" height="100vh" />
  );
}

The config object is passed to the engine at creation time. The init callback fires once the cesdk instance is ready — this is where all our customization lives.

Pitfall: Silent init errors. The <CreativeEditor> component swallows errors thrown inside init. If something fails, the editor loads but appears broken with no console output. Always wrap init in a try/catch:

Step 3: Creating the Video Scene

A video editor needs three things at startup: asset sources, a video scene, and a timeline.

// Load built-in asset libraries (stickers, shapes, filters, typefaces, etc.)
await cesdk.addDefaultAssetSources();

// Load demo content (sample videos, images, audio) + enable upload slots
await cesdk.addDemoAssetSources({
  sceneMode: 'Video',
  withUploadAssetSources: true,
});

// Create a video scene with timeline
await cesdk.createVideoScene();

createVideoScene() sets up the scene hierarchy that powers the timeline:

Scene
 └── Page (represents the video canvas — 1920x1080 by default)
      ├── Track (video track — holds video clips in sequence)
      ├── Track (overlay track — text, stickers, images)
      ├── CaptionTrack (subtitles synced to playback)
      └── Audio (background music, voiceover, sound effects)

Each element on the timeline is a block with timing properties: timeOffset (when it appears) and duration (how long it plays). The engine handles rendering each frame, compositing layers, and synchronizing audio.

Browser support

Video editing relies on modern web codecs (WebCodecs API), which are available in Chromium-based browsers (Chrome, Edge, Brave). Safari and Firefox support is limited.

Step 4: The CapCut Dark Theme

CapCut’s visual identity is defined by its deep charcoal backgrounds, teal accent colors, and subtle elevation layers. CE.SDK’s theming system maps perfectly to this through CSS custom properties.

Setting the base theme

cesdk.ui.setTheme('dark');

This activates CE.SDK’s built-in dark theme. But we want to go further — we need CapCut’s specific color palette.

Custom CSS overrides

CE.SDK scopes all its UI under .ubq-public with data-ubq-theme and data-ubq-scale attributes. We override the CSS custom properties to inject our palette:

.ubq-public[data-ubq-theme='dark'][data-ubq-scale='normal'],
.ubq-public[data-ubq-theme='dark'][data-ubq-scale='modern'] {
  /* Deep charcoal backgrounds — darker than CE.SDK's default dark */
  --ubq-canvas: hsl(220, 15%, 8%) !important;
  --ubq-elevation-1: hsl(220, 13%, 12%) !important;
  --ubq-elevation-2: hsl(220, 12%, 15%) !important;
  --ubq-elevation-3: hsl(220, 10%, 18%) !important;

  /* High-contrast white text */
  --ubq-foreground-default: hsla(0, 0%, 100%, 0.92) !important;
  --ubq-foreground-light: hsla(0, 0%, 100%, 0.55) !important;

  /* CapCut's signature teal/cyan accent */
  --ubq-interactive-accent-default: hsl(190, 85%, 48%) !important;
  --ubq-interactive-accent-hover: hsl(190, 85%, 42%) !important;
  --ubq-interactive-accent-pressed: hsl(190, 85%, 36%) !important;

  /* Subtle borders — barely visible, like CapCut */
  --ubq-border-default: hsla(0, 0%, 100%, 0.08) !important;
}

The key design decisions:

Property	Value	Why
`--ubq-canvas`	`hsl(220, 15%, 8%)`	Near-black with a slight blue tint — matches CapCut’s canvas area
`--ubq-elevation-1/2/3`	12% → 15% → 18% lightness	Subtle elevation steps create depth without harsh contrast
`--ubq-interactive-accent-*`	`hsl(190, 85%, 48%)`	CapCut’s teal — used for buttons, selections, and progress bars
`--ubq-border-default`	8% opacity white	Nearly invisible borders that only appear on close inspection

Responsive scale

We also configure responsive scaling so the editor adapts to touch devices:

cesdk.ui.setScale(({ containerWidth, isTouch }) => {
  if ((containerWidth && containerWidth < 768) || isTouch) {
    return 'large'; // Bigger touch targets on small/touch screens
  }
  return 'normal'; // Standard desktop sizing
});

Step 5: AI Plugin Integration

This is where the editor transforms from a basic video tool into something that feels like CapCut’s AI-powered experience. CE.SDK’s plugin system lets us add all AI capabilities through a single unified AiApps plugin.

The unified AiApps approach

Instead of registering each AI plugin separately, @imgly/plugin-ai-apps-web provides a single entry point:

import AiApps from '@imgly/plugin-ai-apps-web';
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
import OpenAiImage from '@imgly/plugin-ai-image-generation-web/open-ai';
import Elevenlabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
import Anthropic from '@imgly/plugin-ai-text-generation-web/anthropic';

await cesdk.addPlugin(
  AiApps({
    dryRun: true, // Simulate for development — no API calls
    providers: {
      text2text: Anthropic.AnthropicProvider({ proxyUrl: PROXY_URL }),
      text2image: [FalAiImage.RecraftV3({ proxyUrl }) /* ... */],
      image2image: [FalAiImage.GeminiFlashEdit({ proxyUrl }) /* ... */],
      text2video: [FalAiVideo.MinimaxVideo01Live({ proxyUrl }) /* ... */],
      image2video: [
        FalAiVideo.MinimaxVideo01LiveImageToVideo({ proxyUrl }) /* ... */,
      ],
      text2speech: Elevenlabs.ElevenMultilingualV2({ proxyUrl }),
      text2sound: Elevenlabs.ElevenSoundEffects({ proxyUrl }),
    },
  })
);

Provider categories explained

Category	What it does	Models we configured
`text2text`	AI copywriting — improve text, translate, change tone	Anthropic Claude
`text2image`	Generate images from text prompts	RecraftV3, IdeogramV3, GPT Image
`image2image`	Transform existing images with AI	Gemini Flash Edit, GPT Image
`text2video`	Generate video clips from text descriptions	Minimax Video, Kling Video, Pixverse
`image2video`	Animate a static image into video	Minimax Video, Kling Video
`text2speech`	Convert text to spoken audio with voice selection	ElevenLabs Multilingual V2
`text2sound`	Generate sound effects from text	ElevenLabs Sound Effects

When multiple providers are configured in an array (like text2image), the UI automatically shows a provider/model selection dropdown so users can choose which AI model to use.

The proxy server requirement

Every provider takes a proxyUrl parameter. This is critical for production:

Browser  →  Your Proxy Server  →  AI Provider (fal.ai, ElevenLabs, etc.)
                ↑
          Injects API keys
          server-side

Your API keys should never be in client-side code. The proxy server receives requests from CE.SDK, attaches your API key, and forwards to the AI provider. During development, dryRun: true simulates generation without any API calls.

Background removal

Background removal is a separate plugin because it runs entirely client-side — no proxy needed:

import BackgroundRemovalPlugin from '@imgly/plugin-background-removal-web';

await cesdk.addPlugin(
  BackgroundRemovalPlugin({
    ui: { locations: ['canvasMenu'] },
  })
);

This uses ONNX Runtime (WebAssembly + WebGPU) to run an AI segmentation model directly in the browser. The first run downloads ~40MB of model weights, which are then cached. Select any image on the canvas and click “Remove Background” in the context menu.

Step 6: UI Customization with the Component Order API

CE.SDK’s UI is built from five customizable areas: Dock, Inspector Bar, Canvas Menu, Navigation Bar, and Canvas Bar. The Component Order API lets us insert, remove, and reorder components in each area.

Adding the AI button to the dock

We want the AI Apps button to be the first thing users see in the dock:

cesdk.ui.insertOrderComponent(
  { in: 'ly.img.dock', position: 'start' },
  'ly.img.ai.apps.dock'
);

insertOrderComponent takes a location specifier and the component(s) to insert. position: 'start' puts it at the top of the dock. Alternative positions include 'end', a numeric index, or relative placement with before/after matchers.

When a user selects a text or image block and right-clicks, we want AI options available:

cesdk.ui.insertOrderComponent({ in: 'ly.img.canvas.menu', position: 'start' }, [
  'ly.img.ai.text.canvasMenu',
  'ly.img.ai.image.canvasMenu',
]);

Passing an array inserts multiple components at once.

We add an Export button with a real click handler that triggers MP4 export:

cesdk.ui.insertOrderComponent(
  { in: 'ly.img.navigation.bar', position: 'end' },
  {
    id: 'ly.img.action.navigationBar',
    key: 'export',
    label: 'Export',
    icon: '@imgly/Download',
    onClick: async () => {
      const engine = cesdk.engine;
      const page = engine.block.findByType('page')[0];
      if (page) {
        const blob = await engine.block.exportVideo(page, {
          mimeType: 'video/mp4',
        });
        const url = URL.createObjectURL(blob);
        const a = document.createElement('a');
        a.href = url;
        a.download = 'video.mp4';
        a.click();
        URL.revokeObjectURL(url);
      }
    },
  }
);

The exportVideo method renders every frame of the timeline — compositing video tracks, overlays, text, captions, and audio — into an MP4 blob entirely in the browser.

Step 7: Wiring Generated Audio into the Asset Library

AI-generated audio (speech and sound effects) is stored in provider-specific history sources. To make this audio browsable alongside regular audio assets, we inject the history source into the audio asset library entry:

const audioEntry = cesdk.ui.getAssetLibraryEntry('ly.img.audio');
if (audioEntry != null) {
  const existingSourceIds = Array.isArray(audioEntry.sourceIds)
    ? audioEntry.sourceIds
    : audioEntry.sourceIds({});
  cesdk.ui.updateAssetLibraryEntry('ly.img.audio', {
    sourceIds: [...existingSourceIds, 'ly.img.ai.audio-generation.history'],
  });
}

Now when users open the Audio panel in the dock, they’ll see their AI-generated audio alongside the sample library.

Step 8: Custom Labels with i18n

CE.SDK’s i18n system lets us customize any UI string. We used it to make the AI prompt fields more inviting:

cesdk.i18n.setTranslations({
  en: {
    'ly.img.plugin-ai-video-generation-web.fal-ai/minimax/video-01-live.property.prompt':
      'Describe your video...',
    'ly.img.plugin-ai-image-generation-web.fal-ai/recraft-v3.property.prompt':
      'Describe your image...',
  },
});

Translation keys follow the pattern {plugin-id}.{provider-id}.property.{field}. You can also add multi-language support by including keys for de, fr, es, etc.

The Complete File Structure

capcut-like-editor/
├── src/
│   ├── App.jsx              # Root — imports VideoEditor + theme CSS
│   ├── VideoEditor.jsx      # The entire editor (single component, ~180 lines)
│   ├── capcut-theme.css     # CapCut dark theme overrides (~100 lines)
│   ├── index.css            # Global reset (margin/padding/overflow)
│   └── main.jsx             # React entry point
├── index.html
├── package.json
└── vite.config.js

That’s it. The entire CapCut-like editor is three files: one React component, one CSS file, and a thin App wrapper.

What We Get Out of the Box

Because CE.SDK’s video UI is pre-built, we didn’t write any code for these features — they come from createVideoScene() and the default UI:

Multi-track timeline with drag-to-reorder, drag-to-resize
Trim and split — drag clip edges or use the split tool
Join and arrange — drag clips between tracks
Transform — crop, flip, rotate via the inspector
Playback controls — play, pause, seek, scrub
Text overlays — add styled text with the text tool
Stickers and graphics — from the asset library
Filters and effects — LUT-based color grading
Undo/redo — full history stack
Zoom and pan — standard canvas navigation

The AI plugins add their own UI components (dock buttons, generation panels, context menu items) through the plugin system. We just positioned them where we wanted.

Going to Production

To take this from a prototype to production, you need three things:

1. License key

Get a free trial key at https://img.ly/forms/contact-sales and set it in the config (yes, if you want to ship to production you’ll have to talk to our lovely colleagues in sales):

const config = {
  license: 'YOUR_CESDK_LICENSE_KEY',
};

2. Proxy server

Set up a server that forwards AI requests with your API keys:

const PROXY_URL = 'https://your-server.com/api/ai-proxy';

See the CE.SDK Proxy Server guide for Express.js and other server examples.

3. Disable dry run

Remove dryRun: true from the AiApps configuration to enable real AI generation.

Key Takeaways

CE.SDK does the heavy lifting. The timeline, playback, rendering, and export are all handled by the engine. We wrote zero video processing code.
The plugin system is powerful. Six AI capabilities were added with a single addPlugin(AiApps({ ... })) call. Background removal was one more call.
CSS custom properties make theming painless. We matched CapCut’s aesthetic by overriding ~25 CSS variables. No forking, no patching.
The Component Order API is the customization backbone. insertOrderComponent with position-based placement is the cleanest pattern for adding UI elements.
Wrap init in try/catch. CE.SDK’s <CreativeEditor> swallows errors silently. This is the single most important debugging tip.
Pin your package versions. All @imgly/* plugins must match the @cesdk/cesdk-js version exactly. A version mismatch (like 1.68 vs 1.69) will cause peer dependency conflicts.

Resources

From Prompt to Editor: Running CE.SDK Inside ChatGPT with the Apps SDK

Jan — Fri, 19 Dec 2025 15:43:16 GMT

With the new ChatGPT Apps SDK and Model Context Protocol (MCP), chat interfaces are starting to look less like Q&A tools and more like places where work actually happens. To explore what that means for creative workflows, we built a small technical demo: CE.SDK running directly inside ChatGPT.

From a user’s perspective, the flow is almost trivial. You ask ChatGPT for something like an ecommerce template. ChatGPT searches our template catalog, selects a matching design, and opens it instantly in a fully interactive CE.SDK editor — right inside the chat interface. What looks like a preview is, in fact, a real editor loaded with a real template scene.

This isn’t meant as a product announcement. It’s a technical proof of concept showing how creative SDKs can plug directly into AI-native interfaces.

CE.SDK as a ChatGPT App

https://www.youtube.com/embed/AoDqUNVLLJ4?feature=oembed

The integration is built around a custom MCP server that exposes CE.SDK to ChatGPT as a tool. The server speaks OpenAI’s JSON-RPC–style MCP and implements the standard lifecycle methods (initialize, tools.list, tools.call, resources.read). It knows about our premium template catalog and emits structured payloads that the frontend understands.

On the client side, a Next.js app listens to tool output events streamed from ChatGPT, renders CE.SDK widgets, and hydrates them with the payloads returned by the tool — such as a scene URL, placeholder values, or export permissions. Templates are loaded via CE.SDK’s Template API, either from a URL or from a serialized scene string.

Under the hood, the stack is fairly conventional:

Next.js 15 (App Router)
CE.SDK Web / CreativeEngine
A custom MCP handler to normalize JSON-RPC
Vercel for hosting

What’s new is not the technology itself, but the context in which it runs.

Working with MCP in Practice

The hardest part of the demo wasn’t CE.SDK — it was MCP.

OpenAI’s MCP implementation is extremely strict. Even the smallest schema mismatch can trigger the infamous “TaskGroup 424” error, usually without any hint as to what went wrong. In many cases, the HTTP response is technically successful, but the JSON structure doesn’t match the expected schema closely enough.

The key lesson here is to treat MCP responses as hard contracts:

Validate every response against a schema (for example with zod).
Mirror OpenAI’s field names exactly, even for empty or optional capabilities.
Assume that a 424 almost always means “your JSON shape is wrong”.

Another important insight was how critical visual context is in chat-based tools. If your MCP responses don’t include thumbnails or preview images, ChatGPT will often fall back to rendering links. For creative tools, that immediately breaks the experience. In a chat UI, visuals aren’t an enhancement — they are the interface.

State handling also requires a shift in mindset. ChatGPT can replay tool calls, and each prompt effectively creates a new widget instance. You can’t rely on mutating an existing editor. The frontend needs to be idempotent: load scenes from serialized state first, then apply changes. Every tool call should be treated as a fresh render.

Why This Pattern Matters

This demo points to a broader change in how creative software may be accessed. Chat becomes a coordination layer, not just a conversational one. Instead of explaining how something could be designed, the AI opens the actual editor and lets the user continue from there.

For CE.SDK, this fits naturally. Editors become embeddable capabilities rather than standalone applications, and AI systems become the entry point into creative workflows. Prompting turns into doing.

Beyond OpenAI: The MCP UI Standard

Although this demo uses OpenAI’s MCP, the architecture maps cleanly to the new MCP UI standard recently introduced by Anthropic. That standard aims to make tool definitions and UI rendering more consistent across models and platforms.

Because this integration already separates tool logic from UI rendering and relies on structured, explicit payloads, transferring it to Anthropic’s MCP UI model is conceptually straightforward. CE.SDK can act as a reusable creative surface across ChatGPT, Claude, and future AI app ecosystems.

You can read more about Anthropic’s MCP UI direction here:

https://blog.modelcontextprotocol.io/posts/2025-11-21-mcp-apps/

This demo is intentionally small and technical, but it highlights a meaningful shift: AI systems that don’t just describe creative outcomes, but open the tools to actually create them.

Introducing CE.SDK Renderer: The Missing Piece of Creative Automation

Jan — Mon, 24 Nov 2025 11:05:15 GMT

For years, teams have used CE.SDK to power rich, customizable editing experiences across web, mobile, and desktop. But one critical component of true end-to-end creative automation remained challenging: server-side rendering.

Until now, export workflows depended on client devices, brittle FFmpeg scripts, or slow headless browser setups. While these approaches worked, they often fell short for enterprise scale workflows, AI pipelines, or high-volume creative automation.

Today, we’re excited to introduce the CE.SDK Renderer, the native, GPU-accelerated server-side rendering engine that completes the CE.SDK ecosystem.

The Renderer brings CE.SDK’s design engine to your backend with fast, compliant, enterprise-ready export for images, PDFs, and video, enabling organizations to generate media at scale with full fidelity and predictable performance.

Why the CE.SDK Renderer Matters

Rendering Has Been the Last Barrier to Full Automation

Teams could design and generate templates programmatically, but producing the final asset—especially video—was constrained by:

Lack of GPU acceleration
No native CE.SDK export on servers
Legal requirements for H.264/H.265
Unstable browser virtualization
Node.js memory and performance limits

The CE.SDK Renderer solves all of this in one fell swoop with a native Linux binary packaged as a Docker container, built explicitly for backend media generation.

Many of our SaaS customers have told us the same thing: they want a safe, supported alternative to FFmpeg that they don’t have to maintain themselves.

What the CE.SDK Renderer Delivers

The CE.SDK Renderer fully unlocks its value in the IMG.LY ecosystem by providing a creative automation infrastructure layer for the user facing portion of IMG.LY’s editor SDK. It enables two core capabilities that modern creative automation depends on: scalable rendering and fully compliant video export:

GPU-Accelerated Export Performance

Native code and GPU acceleration allow extremely fast rendering. Ideal for heavy scenes, 4K+ requests, and large batch pipelines.

Fully Licensed Video on the Backend (H.264/H.265)

You can render MP4 video entirely on your servers with full licensing compliance. Powered by Fluendo’s AV stack, the Renderer provides enterprise-grade legal protection for H.264/H.265, eliminating the risks associated with unlicensed or open-source encoding. This gives teams the confidence to run video-heavy pipelines at scale.

Built for Scale

Perfect for workloads generating hundreds or thousands of creative variations: ads, postcards, dynamic social assets, catalogs, and more on stable maintainable infrastructure supplanting FFmpeg or brittle pipelines based on headless browsers.

Perfect Visual Fidelity

Because it uses the same CE.SDK engine that powers the editing experience, the Renderer produces output that matches exactly what users see in the editor. This fidelity is critical for automated pipelines, ensuring every exported asset is accurate, consistent, and visually correct.

Together, these unlock a complete creative automation workflow: design → generate → render.

How It Works

Async Exports

A CE.SDK scene is created via browser editor, mobile app, or Node.js.
The scene is sent to the Renderer, running in Docker.
The Renderer outputs: PNG/JPEG, PDF, MP4 (H.264/H.265)
The rendered asset is returned to your application or batch pipeline.
Users can continue editing while export happens asynchronously.

This removes export load from the client entirely and opens the door for sophisticated backend media generation.

Batch Rendering

Multiple CE.SDK templates are prepared for automation.
An external data source (CSV, JSON, API) is connected to the templates to generate all design variants.
Each resulting scene is sent as a separate job to the Renderer running in Docker.
The Renderer outputs PNG/JPEG, PDF, or MP4 for every variant.
All assets are returned to your backend or stored in your pipeline, enabling large-scale automated creative production.

Example Use Case: Hyper-Personalized Video Ads at Massive Scale

One of our customers runs large demographic-targeted campaigns where a single master video must transform into tens of thousands of personalized variants. Instead of showing the same creative to everyone, they tailor each video to the viewer’s profile think German, mid-30s, dog owner.

From one base creative, their system dynamically generates up to 100,000 unique versions. Their backend assembles all scene variations in Node.js, then hands them off to the CE.SDK Renderer to produce the final video files.

Before adopting CE.SDK Renderer, this workflow was painful: brittle FFmpeg scripts, unpredictable render results across machines, codec licensing concerns, and processing times that stretched over days.

Maintaining this setup consumed engineering time that would have been better spent on improving the creative logic itself.

With CE.SDK Renderer, the entire pipeline became both scalable and predictable. They now render thousands of variants with consistent results, fully licensed codecs, and throughput high enough to keep up with real-world campaign demands. Instead of fighting their rendering stack, their team can focus on creative automation and campaign performance.

This is just one example of how CE.SDK Renderer enables production-grade, high-volume personalized media at scale something that previously required complex custom infrastructure.

Licensing Options

Open-Source Renderer

A lightweight version using open-source codecs. It’s fast, simple to run, and great for prototyping or early-stage products. Because it doesn’t include licensed H.264/H.265 encoding, it isn’t suitable for enterprise video production or legally compliant distribution.

AV-Licensed Renderer (Fluendo)

This edition includes fully licensed H.264/H.265 encoding powered by Fluendo, ensuring your video exports meet all patent and compliance requirements. It uses concurrency-based licensing and is designed for production workloads, regulated industries, and any platform generating video at scale.

Conclusion

By bringing fast, compliant, and scalable rendering to the backend, the CE.SDK Renderer removes one of the biggest engineering hurdles in creative automation. Whether you’re generating hundreds or hundreds of thousands of assets, it gives you the tools to build stable, modern media pipelines without the maintenance burden. The simplest path to production-grade creative automation.

Start your free trial today and get started in minutes.

How to Embed an Editable InDesign Template in Your Website

Jan — Tue, 28 Oct 2025 12:29:03 GMT

Why Editing InDesign Files in the Browser Matters Now

Adobe InDesign has long been the standard for high-fidelity layout and print design. Yet for many teams, its power comes with friction: it’s desktop-bound, collaboration-limited, and inaccessible to clients or non-designers who simply need to make minor edits.

Creative work requires ever shorter feedback loops and is becoming more and more accessible to the average users, hence organizations are looking for ways to bring InDesign workflows into the browser to make templates editable, collaborative, and automatable.

At the same time, businesses are under pressure to modernize creative production. Marketing teams need to localize campaigns at scale, SaaS platforms want to let users personalize assets, and agencies aim to deliver editable templates instead of static files. The question naturally arises:

“Can I edit InDesign files in a browser?”

Until recently, the answer was “not really.” Adobe offers Share for Review for commenting, InCopy on the Web for limited text changes, and Adobe Express for simplified exports, but none provide full-fidelity, browser-native editing or the ability to embed such functionality into your own product.

That’s where CE.SDK enters the picture.

CE.SDK is an embeddable creative editor that powers photo, video, and design workflows directly in the browser. It’s deeply customizable and extensible, enabling developers to tailor every aspect of the editing experience. The same SDK works cross-platform, Web, iOS, Android, Desktop, and Server, so teams can build consistent creative tools across environments.

By combining CE.SDK’s robust editing engine with the InDesign Importer, you can now bring InDesign templates (IDML files) into a fully fledged web-based design editor while preserving essential layout, style, and asset information. The result: true browser editing of InDesign content, without the limitations of traditional desktop software.

The Current Landscape: What’s Possible (and What Isn’t) with InDesign on the Web

While creative teams increasingly expect collaborative, browser-based tools, Adobe InDesign remains deeply rooted in its desktop heritage. Its powerful layout engine and proprietary file structure were never designed for real-time, cloud-native editing. As a result, teams who rely on InDesign often face friction when trying to make designs accessible to clients or other stakeholders online.

Adobe has made incremental steps toward the web, but these tools still serve limited purposes:

Share for Review – enables commenting and approval workflows in the browser, but doesn’t allow editing or layout changes.
InCopy on the Web (beta) – offers browser-based text editing within locked layouts. It’s useful for copy review, yet visual elements remain untouchable.
Adobe Express export – lets designers repurpose InDesign layouts as simplified templates for lightweight editing, but the process is one-way and loses much of InDesign’s fidelity and control.

For teams that need to deliver editable templates, enable client-side personalization, or embed creative workflows inside their own platforms, these options aren’t sufficient. They lack extensibility, API access, and the ability to maintain brand-level control in a web environment.

Beyond Adobe: Existing Alternatives

Several third-party tools have tried to fill the gap:

VivaDesigner mirrors parts of InDesign’s functionality in a browser, but operates as a self-contained product rather than an embeddable SDK.
Silicon Designer builds on InDesign Server to power web-to-print solutions, but depends on heavy backend infrastructure and costly licensing.
Photopea provides impressive browser editing for layered graphics and basic IDML files, yet lacks enterprise-grade extensibility or workflow integration.

Each of these solutions demonstrates what’s possible, but none offer a developer-friendly foundation for building editing workflows and experience on-top of InDesign in the browser.

Where IMG.LY’s CE.SDK Fits In

This is the gap that CE.SDK fills.

Instead of emulating InDesign’s desktop application, CE.SDK focuses on data translation and browser-native rendering. Its InDesign Importer converts the open IDML format into CE.SDK’s optimized scene format retaining layout structure, typography, and key design elements so users can edit and export designs directly in the browser and all other platforms supported by CE.SDK.

For developers, this approach unlocks a new level of flexibility:

Embed an editable InDesign experience in any web platform.
Integrate design editing into DAMs, CMSs, or creative automation workflows.
Customize UI, behaviors, and integrations to match existing systems.

In short, CE.SDK transforms what used to be static, desktop-bound InDesign files into interactive, web-based design templates, without compromising control or scalability.

Introducing the CE.SDK InDesign Importer

Moving professional InDesign layouts into the browser isn’t just about file conversion, it’s about accurately translating complex design data into a web-native format that can be rendered, edited, and automated.

That’s exactly what the CE.SDK InDesign Importer does.

The importer acts as a bridge between Adobe InDesign’s IDML format and CE.SDK’s scene model, transforming desktop-authored layouts into editable, browser-ready projects. Once an .idml file is exported from InDesign, the importer reconstructs its layers, assets, and properties, packaging them into a CE.SDK scene archive that can be opened instantly inside any CE.SDK instance.

Explore the InDesign Template Import Demo for a comprehensive example.

A Stand-Alone Module, Built for Integration

Unlike the core CE.SDK editor, the InDesign Importer is distributed as a stand-alone package that you can integrate into any workflow. It’s available via npm as

@imgly/idml-importer,

allowing developers to run imports in their own build systems, servers, or client-side applications before loading the resulting scene into CE.SDK.

This separation makes it easy to slot the importer into existing pipelines, for example, automated template ingestion systems, DAM integrations, or internal pre-processing tools — without requiring the full editor runtime.

How It Fits into the CE.SDK Ecosystem

CE.SDK (CreativeEditor SDK) is an embeddable creative editor powering photo, video, and design workflows across Web, iOS, Android, Desktop, and Server. It offers a modular, extensible engine and UI framework that teams can tailor to any brand or use case.

The InDesign Importer extends that ecosystem by unlocking compatibility with one of the most widely used layout tools in the world. Together, they enable a complete pipeline:

InDesign (IDML) → @imgly/idml-importer → CE.SDK Scene File → Browser Editing & Automation

This means existing InDesign templates can become live, editable browser experiences — without rebuilding designs manually or deploying heavy server infrastructure.

What the Importer Delivers

File-Format Translation – Converts IDML files into CE.SDK scene archives while preserving layout hierarchy, positioning, and grouping.
Asset Bundling – Packages fonts, embedded images, and color data for immediate use in CE.SDK.
Color Mapping – Converts CMYK values into RGB for web rendering (native CMYK support is in development).
Element Preservation – Maintains grouping, rotation, shapes (rectangles, ovals, polygons, lines), gradients, transparency, and strokes.
Developer Flexibility – Import locally or at scale, feed the resulting scene into CE.SDK’s API, or integrate into automated asset pipelines.

Real-World Use Cases & Workflows

Once an InDesign file becomes editable in the browser, entirely new workflows open up — from collaborative editing to automated content generation.

The CE.SDK InDesign Importer enables organizations to extend proven InDesign templates into scalable, web-native experiences:

Web-to-Print Platforms

Allow end users to personalize marketing collateral, business cards, or packaging directly in a browser editor while maintaining the designer’s original layout integrity.

Brand Template Portals

Empower distributed teams, agencies, or franchise partners to create on-brand materials without ever touching desktop software. Designers upload InDesign templates once; users edit and export variations on demand.

Creative Automation Systems

Combine CE.SDK with data pipelines to automatically generate localized or personalized assets at scale — replacing time-consuming manual layout work with programmable design workflows.

Client Collaboration

Deliver interactive proofing experiences where clients can adjust copy, swap images, or approve layouts in a controlled browser environment, eliminating the “export–review–revise” loop typical of InDesign-based projects.

Each of these use cases builds on the same foundation: reliable IDML translation plus CE.SDK’s flexible editing engine.

That combination makes the Importer not just a conversion tool, but a bridge to entirely new creative business models.

CE.SDK vs. Traditional InDesign Server & Other Alternatives

For teams exploring browser-based design editing, the landscape typically centers on three paths — InDesign Server, web-to-print middleware, or browser SDKs.

The CE.SDK InDesign Importer offers a modern alternative to all three.

Feature / Capability	InDesign Server	Third-Party Tools (VivaDesigner, Silicon Designer)	CE.SDK + InDesign Importer
Editing Fidelity	Full but desktop-rendered	Partial; varies by implementation	High; layout preserved via IDML
Web Accessibility	Limited; server-side only	Browser UI, but closed systems	Fully client-side, browser-native
Embeddable / SDK	No	Proprietary	Yes + modular npm packages
Infrastructure	Requires Adobe licensing & server setup	Vendor-hosted	Lightweight; deploy anywhere
Extensibility	Restricted scripting	Limited	Full API & UI customization
Cost / Licensing	High, per-instance	Varies; often enterprise-only	Predictable developer friendly licensing

CE.SDK’s approach eliminates the dependency on server-side rendering and proprietary hosting, providing a developer-first, browser-native foundation for creative editing.

By translating InDesign layouts into open CE.SDK scenes, it combines professional-grade fidelity with the flexibility of modern web architecture.

Conclusion – A New Era for InDesign Workflows

For years, creative teams have struggled to bridge the gap between InDesign’s print-grade precision and the web’s flexibility and scalability.

The CE.SDK InDesign Importer closes that gap — turning traditional .indd projects into browser-ready, editable templates that can live inside any modern application.

Whether you’re building a web-to-print platform, empowering clients through self-service editing, or connecting templates to creative-automation pipelines, CE.SDK provides the foundation to make it happen — with full developer control and a consistent experience across Web, Mobile, and Desktop.

Explore the live demo: InDesign Template Import Demo

Try the importer: @imgly/idml-importer on npm

Learn more about CE.SDK: https://img.ly/products/creative-sdk

Frequently Asked Questions

Can I edit an InDesign file directly in a browser?

Not with Adobe’s native tools alone — Share for Review and InCopy on the Web only allow commenting or text changes. With the CE.SDK InDesign Importer, however, you can convert an exported IDML file into a browser-editable format that retains layout, fonts, and key visual elements.

What file formats does the importer support?

The importer reads IDML files exported from Adobe InDesign and converts them into CE.SDK scene archives. These can then be opened in CE.SDK for full browser editing and exported again to formats such as PDF, PNG, or JSON.

Does it require Adobe InDesign Server?

No. The @imgly/idml-importer runs independently — it’s a standalone npm package and doesn’t depend on InDesign Server or any Adobe infrastructure. You only need an IDML export from InDesign.

Is CMYK color supported?

Currently, CMYK values are automatically translated into RGB for accurate web rendering. Native CMYK support is planned in future updates.

Can I automate bulk imports?

Yes. Because the importer is installable via npm, you can integrate it into scripts or pipelines to process large template libraries automatically before loading them into CE.SDK.

Do imported InDesign templates remain editable for non-designers?

Absolutely. Once loaded into CE.SDK, templates can be edited through a customizable browser interface — ideal for client portals, marketing platforms, or self-service brand editors.

Animate Between Images - AI-Native Video Workflows with CE.SDK and Veo 3

Jan — Wed, 22 Oct 2025 14:10:31 GMT

With the release of Veo 3.1, we wanted to show just how effortless it is to embed generative AI capabilities directly into creative workflows. In this quick demo, we integrated Veo 3 into CreativeEditor SDK (CE.SDK) enabling users to animate between two still images in just a few clicks.

From Still Images to Motion

In our demo, we start with two images of the same person, one wearing a hat, the other without. Inside the editor, users simply select both images, click the AI context button, and choose “Animate between images.”

The images are loaded into a side panel where users can optionally add a text prompt to guide the transition. Once generated, the resulting short video is placed directly on the canvas ready for editing, compositing, or export.

What’s particularly impressive is the generation speed. In this example, Veo 3.1 produced a smooth 8-second transition in just 9 seconds a major improvement compared to earlier versions. This speed makes iterative creative workflows feel fluid and responsive, bridging the gap between prompt-driven generation and real-time editing.

Try It Out

You can check out the implementation on GitHub and give Veo 3.1 a spin inside your CE.SDK instance (sign up for a free trial if you haven’t already).

A Glimpse of AI-Native Editing

This simple feature highlights how easy it is to make your editor AI-native, combining traditional editing tools with generative intelligence.

Practical use cases include:

E-commerce
Show products “in action” or animate between styles and configurations.
Marketing
Create quick product reveal animations from static assets.
Content creation
Generate short motion clips or “tween” between creative scenes.

And if users want longer clips, they can simply add another 8-second track and transition seamlessly into the next.

The Future of Creative Workflows

With Veo 3 integrated, CE.SDK becomes a powerful playground for AI-driven creativity from image-to-video to scene interpolation and contextual animation.

We already empowered over 600 innovative startups, government entities, and Fortune 500 companies to add powerful design, video, and photo editing workflows to their products. Get in touch, to see how we can do the same for you.

A Guide to Creative Automation with CE.SDK & Javascript

Jan — Fri, 05 Sep 2025 11:14:31 GMT

https://www.youtube.com/embed/GtORNyzjOq0?feature=oembed

What is Creative Automation?

Creative Automation is the process of generating visual content, images, videos, designs, programmatically, often in response to structured inputs like product data, user attributes, or campaign parameters.

Instead of designing each asset manually, creative automation enables you to define templates and rules that the system uses to produce content at scale.

This approach accelerates production workflows and enables personalization at scale, multivariate testing, and omni-channel consistency without stretching design resources.

Creative Automation & Generative AI

Of course, you cannot talk about creative automation in this day and age without also discussing the role of generative AI. Gen AI introduces intelligent content generation into the pipeline making it easy to generate headlines, product descriptions and entire design elements like illustrations, videos or voiceovers and background music.

When paired with a robust editing and rendering engine like CE.SDK, generative AI becomes even more powerful. A design editor that sits at the intersection between the human decision maker and raw AI generated creatives ensures that those creatives can be curated, refined and deployed in an orderly fashion. This usually works by employing design templates that provide extension points for dynamic data through variables and placeholders and that AI generated content can be merged into for a brand-safe final creative.

Our customer Omneky provides an excellent example of how these components can be orchestrated for maximum efficiency and productivity. Omneky is an ad tech platform that uses generative AI to automatically generate creatives and ad copy for a company based on their product and positioning, these are then interpolated into CE.SDK template to create ad variations. These variations are then tested across ad channels and the results used to inform further refinement and ad selection. This is a prime example of how to combine creative automation and editors to tame the chaotic fountain of AI creatives.

How CE.SDK Enables Creative Automation

Let us get into the meat and potatoes of creative automation and sketch out the requirements of a creative automation solution and an end to end example with CE.SDK. IMG.LY’s CreativeEditor SDK (CE.SDK) offers both a user interface for design-time flexibility and a graphics processing engine API for runtime automation.

Design Once, Automate Everywhere

With CE.SDK, you create design templates that define placeholders, variables, and lockable design elements. These templates ensure brand consistency while allowing dynamic content generation. Whether you’re building social ads, team cards, or product visuals, the structure remains stable while the content varies. The diagram below illustrates these components using a classical mail merge example. The user creates a design template within the editor and defines certain variables like first_name and address, a data source of address data can now be merged with this template using the engine API: variable: setString to create an individualized postcard for every data point in the set:

Under the Hood: CE.SDK Architecture

In order to fully understand how to implement creative automation workflows with CE.SDK we need a basic mental model of its architecture. CE.SDK is built on two distinct layers, giving you total flexibility and control.

The UI Layer is the customizable frontend allowing users to design and edit different media types from photos, design compositions such as collages, videos or mixed media.

The Engine Layer exposes the Creative Engine, CE.SDK’s core rendering and editing engine, through an API. The architecture of CE.SDK is built around a layered approach that separates concerns while ensuring cross-platform consistency:

**Engine Core (C++)**At the foundation lies the Creative Engine, a high-performance rendering and editing core written in C++. This is where all processing, layouting, and rendering logic happens, guaranteeing precision and fidelity.
API LayerOn top of the core sits the API Layer, which exposes the engine’s capabilities in a consistent, high-level interface. This abstraction allows developers to focus on creative logic without dealing directly with low-level rendering details.
Cross-Platform TargetsThe same engine and API are cross-compiled to iOS, Android, Web, and server environments, ensuring that rendering results remain identical regardless of platform. This uniformity is critical for automation workflows, where assets may be produced or consumed across multiple devices and services.
Headless ModeBeyond powering interactive UIs, the engine can also be run headlessly. In this mode, CE.SDK operates without a frontend, enabling automated scenarios such as bulk rendering of images or videos, dynamic template generation, and data-driven personalization at scale.

This modular architecture ensures that whether you are building a custom editing UI or running automated creative workflows on the server, every output is backed by the same reliable, cross-platform engine.

How to Integrate Creative Automation into Your Workflow

Step 1: Integrate CE.SDK

Even if you don’t plan to use the CE.SDK UI for end users, you still need to spin it up locally or within internal tooling to create templates. Let’s install CE.SDK and configure it to the advanced UI, in Creator mode.

CE.SDK by default comes with two UI flavors. The design or default UI config exposes all the essentials for productive design editing by most users, ideal for adapting templates. An advanced UI fine grained editing controls and the ability to define placeholder elements and control what users can change.

The “Creator” role allows setting constraints on template elements, while the “Adopter” role is focused on adapting these elements.

Creator: Set constraints and manage template settings.
Adopter: Edit elements within the bounds set by the Creator.

Install CE.SDK following one of our guides and explore the entire demo here.

You can find the complete demo here

Configuring CE.SDK for Template Creation

Below is a sample configuration. Notice how we set role: 'Creator' and view: 'advanced' to unlock template authoring features. We also configure an onSave callback to export complete template archives — these include all necessary resources (images, fonts, stickers, etc.) in a self-contained package. Archives are what you’ll deliver to adopters or use for data-driven automation later.

const config = {
  theme: 'dark',
  role: 'Creator',
  callbacks: {
    onExport: 'download',
    onUpload: 'local',
    onSave: async (sceneString: string) => {
      try {
        // assumes engine is available in scope
        const blob = await engine.scene.saveToArchive();
        const formData = new FormData();
        formData.append('file', blob);

        await fetch('/upload', {
          method: 'POST',
          body: formData,
        });
      } catch (error) {
        console.error('Save failed', error);
      }
    },
  },
  ui: {
    elements: {
      view: 'advanced',
      panels: {
        inspector: {
          show: true,
          position: 'right',
        },
      },
      dock: {
        iconSize: 'normal',
        hideLabels: true,
      },
      navigation: {
        action: {
          export: {
            show: true,
            format: ['image/png', 'application/pdf'],
          },
        },
      },
    },
  },
};

Step 2: Build Smart Templates

With CE.SDK running in Creator mode, we can now define editable templates. These templates can include:

Named blocks → Easy-to-reference design regions (e.g., PodcastCover, PodcastBadge).
Variables → Text placeholders enclosed in double curly braces (e.g., {{Message}}, {{PodcastName}}).

For example, here’s a podcast cover template with two text variables and named blocks. The variables make it possible to inject data programmatically later.

Podcast template:
- Variables: {{Message}}, {{PodcastName}}
- Named blocks: PodcastCover, PodcastBadge

Since we’re in Creator mode, we can also enforce brand constraints, for instance, locking background colors, logos, or fonts so that adopters (or automated processes) don’t accidentally override brand-critical assets.

Step 3: Automate Asset Creation

Once you have a template, the next step is automation. Using the headless Creative Engine, you can:

Populate placeholders with external data (e.g., from APIs).
Swap images, update text variables, or adjust styling dynamically.
Export the results at scale into formats like PNG, PDF, or even video.

Example: Dynamic Podcast Covers

In this demo, we connect to the Apple iTunes API to fetch podcast data. Whenever a podcast is selected (or user input changes), our fillTemplate function updates the CE.SDK scene.

Key concepts:

Blocks API → Query template blocks by name (engine.block.findByName('PodcastBadge')) and replace their fills.
Fills → Blocks can have images, solid colors, gradients, or videos. For podcast artwork, we set imageFileURI.
Variables API → Set text variables directly (engine.variable.setString('Message', 'Hello World')).

 const fillTemplate = (engine: CreativeEngine, page: number) => {
    let { r, g, b } = backgroundColorRGBA;
    engine.block.setColor(page, 'fill/solid/color', { r, g, b, a: 1 });
    if (podcast) {
      const photoBlocks = engine.block.findByName('PodcastCover');
      photoBlocks.forEach((photoBlock: number) => {
        const photoFillBlock = engine.block.getFill(photoBlock);
        engine.block.setString(
          photoFillBlock,
          'fill/image/imageFileURI',
          podcast.artworkUrl600
        );
      });
    }
    const colorTheme = getThemeColorFromBackgroundColor(backgroundColorRGBA);
    const [badgeBlock] = engine.block.findByName('PodcastBadge');
    engine.block.setString(
      engine.block.getFill(badgeBlock),
      'fill/image/imageFileURI',
      caseAssetPath(
        `/podcast-badge-${colorTheme === 'light' ? 'black' : 'white'}.png`
      )
    );

    // set text variables
    engine.variable.setString('Message', debouncedMessage || '');
    engine.variable.setString(
      'PodcastName',
      podcast ? podcast.collectionName : '')
      // set text colors based on background theme
      const [messageBlock] = engine.block.findByName('Message & Name');
      const rgb =
      colorTheme === 'dark' ? { r: 1, g: 1, b: 1 } : { r: 0, g: 0, b: 0 };
      engine.block.setTextColor(
      messageBlock,
      { ...rgb, a: 0.75 },
      0,
      "{{Message}}".length
    );
    engine.block.setTextColor(
      messageBlock,
      { ...rgb, a: 1.0 },
       "{{Message}}".length
    );
  };

At this point, you’ve created a data-driven creative automation pipeline:

Templates authored with placeholders and constraints.
Dynamic population of text and image fields via external data.
Export to high-quality assets at scale.

This workflow is the foundation for personalized marketing campaigns, automated content production, and AI-driven design pipelines with CE.SDK.

Creative Automation isn’t just about speed it’s about scalability, consistency, and customization. IMG.LY’s CE.SDK gives you the tools to build powerful, flexible automation workflows tailored to your brand and users.

Start you free trial today and integrate content automation into your application.

The Top 7 Video Editing SDKs in 2026

Jan — Tue, 02 Sep 2025 11:19:08 GMT

With dozens of video editing SDKs on the market, choosing the right one can feel like a daunting task.

To help you cut through the noise, we’ve put together a breakdown of the 7 best video editing SDKs in 2026: IMG.LY, Meishe SDK, Banuba, Shotstack, BytePlus Effects, Rendley, and Picsart. We’ll cover:

Key features and capabilities
Supported platforms and customization options
Ideal use cases and trade-offs

This guide is unbiased, but we’ll also highlight how IMG.LY compares and where it stands out as the only solution that combines a polished editor with scalable automation in a single package.

1. IMG.LY CreativeEditor SDK (CE.SDK)

Choosing IMG.LY means selecting a platform that offers both: a Canva-grade editor for your users and a powerful Engine API for automation. That dual focus is what sets CE.SDK apart.

Features & capabilities: CE.SDK works best when you see it as a bridge between what end users expect and what your product team needs behind the scenes. On the surface, users get familiar tools including multi‑track editing, ready‑made templates, and a growing set of AI features like text‑to‑image, text‑to‑video, and AI voiceovers. Behind the scenes, developers can extend and customise with a plugin system so the editor adapts as your product evolves.
Platforms & customization: This flexibility carries over to platforms. CE.SDK works across Web, iOS, Android, React Native, Flutter, Electron, and Node.js. It’s also fully white‑label, which means your customers see a seamless extension of your brand, not a third‑party tool.
Implementation & use cases: Getting started with our tool doesn’t require a huge lift. Teams can spin up a proof of concept quickly, then scale into enterprise deployment when ready. That’s why you’ll see CE.SDK inside SaaS platforms, e‑commerce personalization tools, DAM systems, and even social networks.
Future‑proofing, support & pricing: What reassures teams is the long‑term roadmap: regular feature releases, AI integrations, and direct support from solution engineers. Enterprise SLAs and SSO come standard. Pricing depends on the usage, which makes it easier to scale without committing to rigid licensing.
Who it’s for: CE.SDK is valuable for SaaS, MarTech, e-commerce, and DAM platforms. These teams need to give users a simple, branded editing experience while also managing complex automation in the background. Instead of stitching together multiple tools or sacrificing one capability for another, CE.SDK provides an all-in-one foundation: polished editing, scalable automation, enterprise-grade support, and ongoing AI-driven innovation.

Key differentiator

CE.SDK doesn’t force you to choose between usability and power. On one side, it delivers a polished, user-facing editor that feels familiar to end users. On the other hand, it provides a powerful automation engine that scales with your product. Most SDKs lean heavily in one direction. They either focus on end-user editing or specialize in backend automation. CE.SDK is different because it combines both in a single platform.

2. Meishe SDK (Meicam)

Meishe SDK, also known as Meicam, is a mature Chinese SDK that has carved out a space in many professional apps across Asia. Think of it as a toolkit built for industries that need stability and advanced editing power, though it comes with a few trade‑offs that teams should weigh carefully.

Features & capabilities: Meishe offers multi‑track timelines, real‑time effects, and plugin support across mobile, web, and PC. Industries like automotive, broadcasting, and education have already used Meishe successfully, showing how it handles complex and demanding video workflows.
Platforms & customization: It supports full UI customization, white‑labeling, and plugins. And while customization depth is solid, integration tends to be heavier compared to other tools. Documentation can feel dense, which sometimes slows teams down as they work toward launch.
Implementation & use cases: Meishe SDK is beginning to layer in AI integrations like intelligent editing and motion tracking. For large organizations in Asia with strong internal dev resources, these capabilities can be appealing.
Future‑proofing, support & pricing: Meishe continues to expand with new AI features like facial recognition and virtual anchors. It also offers custom enterprise support agreements. However, teams often struggle with the integration process, and the developer experience feels less polished. Setting up complex projects can take longer, and the documentation isn’t as approachable, which leaves some developers hesitant to adopt it widely. Pricing typically follows an enterprise licensing model, which can be less flexible for fast‑scaling SaaS companies.
Who it’s for: Meishe SDK is best suited for large-scale apps in Asia and companies with internal dev resources.

How it compares to IMG.LY

Meishe delivers solid editing capabilities and is a proven choice in professional and regional contexts. But the trade‑offs become clear when you look at integration and support. It can feel heavy to implement, and documentation isn’t as approachable.

IMG.LY pulls ahead here with stronger web support, compatibility with creative file types like PSD, AI, and INDD, and a faster onboarding process that makes it easier for teams to move from idea to production.

Learn more about how IMG.LY compares to Meishe SDK here.

3. Banuba

Banuba is a mobile‑first SDK with a strong focus on augmented reality (AR). It has built its reputation on powering social and UGC apps with effects that mimic the TikTok experience. If your product relies on immersive filters, beauty effects, or virtual try‑on, Banuba often appears at the top of the evaluation list.

Features & capabilities: The SDK provides a TikTok‑like editor with AR masks, beauty filters, face tracking, background removal, and even virtual try‑on features. These tools are designed to keep users engaged with interactive and shareable content.
Platforms & customization: Banuba supports iOS, Android, Flutter, and React Native. It offers modular components with a preset UI, which makes implementation faster for mobile. That said, customization options are limited when compared to solutions designed for enterprise-level flexibility.
Implementation & use cases: Integration is relatively quick for mobile teams, and the SDK has proven popular among social media platforms, beauty apps, and e‑commerce companies offering AR try‑on. However, support for web or large‑scale enterprise environments is limited, which narrows its scope.
Future‑proofing, support & pricing: The company continues to expand its AR and AI features, focusing on personalization and beauty‑driven effects. Banuba operates with a smaller team of about 70 employees, which can affect the pace of enterprise‑grade support. Licensing costs are typically high and not publicly disclosed, which may be a barrier for smaller companies.
Who it’s for: Banuba is best suited for mobile apps that need advanced AR features like beauty filters, face tracking, or virtual try‑on to keep users engaged.

How it compares to IMG.LY

Banuba leads when it comes to AR‑driven capabilities, making it a natural fit for beauty and social platforms. But when teams look beyond filters to enterprise readiness, automation, and cross‑platform support, other options such as IMG.LY offer broader possibilities.

Learn more about how IMG.LY compares to Banuba here.

4. Shotstack

Shotstack focuses squarely on automation. It’s a cloud‑based video editing API designed for bulk rendering rather than end‑user editing. Teams often evaluate it when they need to generate thousands of videos quickly and at scale.

Features & capabilities: Shotstack runs on a JSON‑driven API that lets developers programmatically create and assemble video content. With this approach, teams can handle automated ad generation, personalization, or large‑scale marketing campaigns. While powerful for automation, it doesn’t include a native editor, which means you must build a custom UI if you want end‑user editing.
Platforms & customization: Shotstack is built as a REST API in the cloud, so it fits naturally into modern development stacks. It allows teams to design workflows programmatically, though the absence of prebuilt UI components makes it harder to use the tool right away as compared to other SDKs.
Implementation & use cases: Developers appreciate the smooth onboarding experience and clear API structure. Typical use cases include bulk rendering of ads, dynamic personalization for campaigns, and other scenarios where scaling output matters more than in‑app editing.
Future‑proofing, support & pricing: Shotstack continues to invest in automation and AI‑powered generation, offering tools that personalize video creation at scale. The vendor provides commercial SLAs, but support resources remain limited given the team size. Pricing follows a pay‑as‑you‑go model ($0.30 per minute) or subscription plans, which can work well for predictable, high‑volume usage.
Who it’s for: Shotstack is best suited for teams that need scalable rendering pipelines and automation rather than in‑app editing features.

How it compares to IMG.LY

Shotstack gives you powerful rendering automation and programmatic workflows. IMG.LY takes it further by combining those automation strengths with a complete, user‑facing editor, so you can address both your development needs and your end‑users’ experiences in one solution.

Get a detailed comparison of IMG.LY CreativeEditor SDK and Shotstack here.

5. BytePlus Effects

BytePlus Effects is modelled after TikTok and CapCut. It’s designed to give app developers the same kind of short‑form, social‑first experience that has made those platforms successful. As a result, it appeals to teams that want to replicate TikTok‑like engagement inside their own apps.

Features & capabilities: The SDK offers more than 80,000 effects and filters, including AR stickers and beautification tools. These features aim to keep users engaged by giving them endless ways to personalize and enhance their content.
Platforms & customization: BytePlus integrates with iOS, Android, React Native, and Flutter, making it a strong option for mobile‑first development. You can customise the UI to match your brand, but customization beyond theming is limited, which can be a constraint if you need deeper control.
Implementation & use cases: Since it’s optimized for mobile integration, you can embed it relatively quickly into short‑form consumer video apps. Common use cases include social platforms, beauty apps, and any consumer‑facing product that thrives on AR‑enhanced video.
Future‑proofing, support & pricing: With the backing of TikTok’s ecosystem, BytePlus is evolving rapidly, adding new AI‑driven AR and beauty effects frequently. However, commercial support feels less transparent, and licensing comes through enterprise negotiations with annual fees. This structure can work for large consumer platforms, but may be harder for smaller teams.
Who it’s for: BytePlus is best suited for apps that want to recreate TikTok‑style user experiences and rely heavily on prebuilt AR effects to keep users engaged.

How it compares to IMG.LY

BytePlus takes a step ahead when it comes to prebuilt effects and AR‑driven experiences. But if you need more flexibility in customization, automation capabilities, or enterprise‑grade workflows, IMG.LY offers a broader solution.

For a detailed comparison, take a look at our guide on IMG.LY vs BytePlus Video Editor.

6. Rendley

Rendley is a newer player in the SDK space and has gained attention because of its browser‑only approach. It’s JavaScript‑based and runs entirely in the browser, which means you don’t need a server to get started. This makes it attractive for teams that want a lightweight, client‑only setup without managing infrastructure.

Features & capabilities: The SDK supports multi‑track editing, keyframes, and even integrates with After Effects (AE). These features give frontend developers enough flexibility to build interactive editing experiences directly in the browser.
Platforms & customization: Rendley runs entirely in the browser, which makes it a natural fit for lightweight web apps and UGC platforms. Developers get flexibility through an open, in‑browser approach. However, without enterprise‑level frameworks, scaling to larger organizations becomes more difficult.
Implementation & use cases: Setting up Rendley is straightforward for frontend teams, which makes it appealing for lightweight UGC applications or smaller web projects. That ease of entry, however, comes with limits: AI integrations aren’t a core focus, so its potential for more advanced future applications remains restricted.
Future‑proofing, support & pricing: Rendley is still early‑stage and hasn’t yet proven itself at enterprise scale. Support is provided by a small vendor team, which can make long‑term stability less certain. Pricing details are not publicly available, leaving prospective users to negotiate directly.
Who it’s for: Rendley fits developers who want a browser‑native, client‑only editing tool without the need for server infrastructure.

How it compares to IMG.LY

Rendley works well as a lightweight, browser‑native solution. But if you need enterprise‑grade robustness, automation, or dedicated support, IMG.LY offers more comprehensive coverage.

Learn more about how IMG.LY is a great alternative for Rendley.

7. Picsart Video SDK

Picsart Video SDK is an embeddable video editor SDK that brings Picsart’s creative editing tools into third-party apps or websites.

Features & capabilities: This SDK brings the company’s well‑known creative editing tools into third‑party products. Out of the box, you get features like trimming, multi‑scene editing, transitions, dynamic audio, subtitles, and text overlays, enough to cover the basics of video creation. AI‑driven tools extend these capabilities with smarter image and video editing options.
Platforms & customization: The SDK is React Native‑based but optimized for web embedding, making it appealing for web‑first platforms. You can configure the UI, apply theming, white‑label the experience, toggle features on or off, and integrate your own asset library so the editor matches your product’s design and workflow.
Implementation & use cases: One of Picsart’s strengths is ease of setup: installation is fast, with a React editor installation process that reduces complexity, documentation is detailed, and the editor is designed to embed with minimal lift. This makes it a good choice for advertising or marketing platforms, edu‑tech products, and media platforms that want to give users creative tools quickly.
Future‑proofing, support & pricing: The SDK is backed by Picsart’s large ecosystem and its AI research arm (PAIR), so teams can expect regular updates, new features, and scalable performance. Support comes through developer documentation, a support team, and enterprise collaboration options. Pricing isn’t public, but it’s likely offered as enterprise or usage‑based licensing.
Who it’s for: Picsart’s Video SDK is best for teams that need a web‑based video editor to power product features or educational video production without investing heavily in custom development.

How it compares to IMG.LY

Picsart offers a fast, embeddable editor with AI effects and a strong ecosystem. However, IMG.LY provides deeper customization, more automation capabilities, cross‑platform support, and enterprise‑grade integrations.

Overview Table (Use Cases vs. Solutions)

Conclusion

When you line these SDKs up side by side, it’s clear that each one has its own niche strengths and limitations. Here’s a quick overview:

Banuba / BytePlus: These are the strongest options for AR and social apps, with beauty filters, effects, and TikTok-like workflows.
Meishe: A professional, Asia-centric SDK with heavy native capabilities, but a steeper implementation curve.
Shotstack / Rendley: Both shine in automation pipelines or lightweight browser-based setups, but they don’t offer polished editor UIs.
Picsart: A strong web-first, prebuilt editor with AI effects that’s simple to embed, though limited in customization and automation.

Where does that leave you?

If you’re evaluating SDKs, you need to balance two priorities: delivering a great editor UX for your users and scaling automation behind the scenes. Only IMG.LY covers both ends: offering a polished, enterprise-ready editor and a scalable automation engine.

That makes it the go-to option for teams planning not only their current projects but also long-term growth. With IMG.LY, you can give users a polished editing experience today while building the automation backbone that will let your product scale tomorrow

If you’d like to see how IMG.LY can fit into your platform, get in touch with our experts today for a deeper look and tailored guidance.

What is Visual Prompting?

Jan — Tue, 29 Jul 2025 13:06:11 GMT

A New Paradigm for Creative AI, Built by IMG.LY

To say it’s trite to refer to the impact of AI in this or that domain as disruptive or groundbreaking would be an understatement. Yet, few areas have been as profoundly affected as the creative process. With just a text prompt, anyone can produce stunning images, remix visual styles, and explore design possibilities at a scale and speed never seen before. AI has inserted itself so quickly into this process that its gone from curious novelty to an essential part of the creator toolchain.

The more serious adoption we see, however, the more key limitations of today’s AI tooling come into focus: the prompt itself.

Text alone, for all its expressive power, struggles to capture the essence of visual intent. Most creative work doesn’t begin with a sentence it begins with a sketch, a layout, a mood board, or an arrangement of elements. Visual ideas are shared by pointing, placing, showing.

At IMG.LY, we have begun to think about better ways to direct AI for visual generation, the term we use is Visual Prompting.

Visual Prompting: the practice of composing a visual scene or layout as input for a generative model.

Instead of describing what you want with paragraphs of text, you show it directly using a canvas of images, text, spatial cues, and annotations. This visual composition then becomes the prompt for the AI to generate new content in return. It’s a more natural, intuitive, and powerful way to collaborate with AI, especially when integrated directly into the creative process.

Problem: the Chat Disconnect

The current generation of AI tools has largely been shaped by language-first interfaces. Whether it’s ChatGPT for writing or Midjourney for image generation, the assumption is the same: the user will type a descriptive prompt, and the AI will generate a result based on it.

But when it comes to design, this workflow quickly runs into friction. Visual ideas are inherently spatial and non-linear. Trying to express layout, balance, mood, or specific spatial relationships through text can feel like trying to describe a painting over the phone. It’s possible but unnecessarily cumbersome.

A designer might want to:

Indicate that a certain area in the image should be blue.
Replace a background with a texture sample.
Position a character precisely in a composition.
Annotate which parts of a scene to preserve or modify.

All of these are difficult to express fluently in text. But they’re effortless in a visual interface. The truth is: an image is worth more than a thousand words when prompting an image.

What Is Visual Prompting?

Visual Prompting is a multimodal approach to generative AI, where the input to the model is not just text, but a full visual composition: images, text, annotations, and layout.

Rather than prompting AI in isolation, the user builds their intent on a canvas. This might include:

Reference images that communicate mood or style.
Text blocks indicating desired copy or instructions.
Annotations pointing to specific areas with notes like “make this glow” or “replace this object.”
Spatial composition: where elements are arranged meaningfully to convey intent.

The visual prompt is then interpreted by a multimodal model such as OpenAI’s gpt-image-1 to generate new visual content that reflects not only the textual description, but also the visual context.

How Visual Prompting Works in CE.SDK

About time for an example. As part of our recent AI released we demoed how to use OpenAIs gpt-image-1 model to build visual prompting into CreativeEditor SDK (CE.SDK).

Here’s what the process looks like inside CE.SDK:

Compose Visually: The user creates a layout with reference content, uploaded images, icons, color schemes, design elements, placeholder text, and annotations. This composition represents the “prompt” in visual form.
Add AI Layers: With a single click, the user can trigger image generation using CE.SDK’s built-in AI plugin. The plugin sends the visual context (alongside any optional text input) to a multimodal model capable of interpreting both.
Refine and Iterate: Users can adjust the layout, reposition elements, change annotations, or layer in new references, then prompt again. Because the canvas is interactive and editable, the feedback loop is tight.
Build Up Complexity: Over time, users can layer generated images with manually designed components or other generated outputs, creating rich compositions that blend AI creativity with human direction.

This workflow turns the traditional prompt/response cycle into a conversation between the designer and the model, with the canvas acting as the shared language.

Who Is Visual Prompting For?

The use cases for Visual Prompting extend across industries:

Creative teams can go from reference to generation in seconds, iterating visually instead of wrangling prompts.
Marketing teams can generate regionalized or personalized creative variants from a shared layout.
Product designers can prototype in context, turning layouts into realistic screens without leaving the editor.
Storytellers and content creators can use annotated sketches to generate detailed illustrations or scene variations.
E-commerce platforms can give sellers the power to visually customize their brand materials with AI assistance.

In every case, Visual Prompting replaces friction with flow and text-based prompting with something more expressive, more reliable, and more fun.

Built for This: Multimodal Models and CE.SDK’s Plugin System

Visual Prompting is only possible because of two parallel advancements:

Multimodal AI models, such as OpenAI’s gpt-image-1, that can interpret both images and text, understand spatial relationships, and respond to annotated cues.
A flexible, composable editor SDK like CE.SDK, which enables the construction of visual prompts on a live canvas, and makes it easy to integrate AI models directly into the design flow.

Our SDK was built from the ground up to support AI-first creative workflows. Its plugin architecture allows you to add any model or API, image generation, video generation, captioning, text rewriting and use it natively inside the editor without the need to switch tools or copy/paste.

Generative AI’s full potential is only unlocked when it is embedded directly into the tools creatives use not siloed in chatbots or separate interfaces. Visual Prompting allows that embedding to go even deeper, aligning the mode of input (visual) with the desired output (visual).

Explore It Yourself

🎨 Try out Visual Prompting in our AI Editor demo
📘 Learn How to Integrate AI into CE.SDK
💬 Contact Us to Bring Visual Prompting to Your Product

Build vs. Buy: Is Fabric.js Right for You

Jan — Thu, 15 May 2025 09:31:43 GMT

When you’re building a design editor into your product whether for social content, marketing assets, video, or design workflows choosing the right foundation matters. Many teams start with Fabric.js, an open-source canvas library, attracted by its flexibility and permissive license. But as the project scope grows, so do the limitations.

This article explores the advantages, but also the pitfalls of building a creative tool with Fabric.js and why many product teams ultimately decide to switch or start with a commercial SDK like IMG.LY’s CreativeEditor SDK (CE.SDK).

Consult our comparison page of Fabric.js and IMG.LY for a feature by feature breakdown of the differences between our SDKs and Fabric.js.

Why Teams Switch from Fabric.js

We’ve spoken with dozens of companies, from startups to enterprise clients, who evaluated Fabric.js and in many cases opted for IMG.LY instead. Here’s what we consistently hear:

1. Time-to-Market vs. Reinventing the Wheel

Open-source is attractive at first. But once the excitement of building wears off, teams often face a sobering reality: stitching together a decent editing experience from Fabric.js is a months-long effort.

“We used Fabric.js before, but if IMG.LY’s SDK is cost-effective, we’d rather not reinvent the wheel.”

— B2B SaaS Prospect

They want a modern editor UI, fast iteration cycles, and a stable base not to spend their next quarter building the basics like zoom, text editing, object grouping, layer management, or responsive templates.

2. Lack of Advanced Features

Fabric.js excels at low-level canvas manipulation, offering a robust API for working directly with objects on the HTML5 canvas - shapes, images, text, and transformations. It’s a great starting point for developers who want fine-grained control over rendering and interactivity. However, it falls short when it comes to modern editing paradigms. Fabric.js provides no native concept of reusable templates, lacks support for layout and design constraints like alignment guides or snapping behavior, and does not include higher-level abstractions for content-aware or AI-powered editing. In short, Fabric.js gives you a raw toolkit, not a polished, plug-and-play editing solution. If you’re building a sophisticated design application, you’ll need to layer these capabilities yourself or integrate additional frameworks to bridge the gap.

“We needed a template system and advanced photo editing. Open-source libraries couldn’t do it out of the box.”

— Marketplace Platform Prospect

Teams chasing parity with Canva or Adobe tools quickly hit walls. What starts as a proof-of-concept becomes a rebuild of the tables stakes of design editing.

Real-World Use Cases & Vertical-Specific Challenges

We’ve seen teams from various industries reach the same conclusion: Fabric.js doesn’t scale to meet their creative, technical, or business needs. A few common themes:

E-commerce & Web-to-Print: Retailers building product customization flows need template constraints, consistent output quality, and export formats that go beyond canvas. They can’t afford rendering inconsistencies or unpredictable export fidelity.
MarTech & Social Media Tools: Marketers need batch generation, AI-assisted creative workflow, brand constraints and consistent creative output across devices. Fabric.js lacks built-in support for these workflows.
Mobile-first Apps: Teams building hybrid apps or web apps that need to be functional on mobile struggle with Fabric.js’ lacklustre mobile support. The inability to deliver a consistent UX across iOS, Android, and Web leads to dropped features or split tech stacks.
Video and Multimedia Platforms: Fabric.js has no native video support, track-based editing, or multi-frame logic. Prospects in this space frequently abandon their Fabric.js POCs once real constraints emerge.

A major pain point across all verticals is rendering consistency. CE.SDK renders identically across browser, server, and mobile thanks to a shared rendering core (CreativeEngine). This is critical for:

Workflows that begin on web and finish on mobile
Server-side generation (e.g., previews, PDFs, batch exports)
Feature parity and UX reliability across platforms

Check out our demo page to explore these use cases.

Technical Debt: The Hidden Cost of Fabric.js

On paper, Fabric.js seems like a fast way to get started. But its real cost emerges over time in performance bottlenecks, missing architecture, and maintenance drag.

Aging Codebase and Patchy Maintenance

As of 2025, Fabric.js has over 400 open issues on GitHub, some dating back years. Many feature requests such as better performance for large object sets, async rendering, or text-on-path improvements are either unresolved or uncertainly prioritized.

The core maintainers do their best, but progress is slow. You’re relying on volunteer effort for critical infrastructure.

Slow Issue Resolution

Many of the GitHub issues (e.g., #5885, #6582) highlight long-standing rendering bugs and performance problems. These are hard to fix, and updates can break your own hacks or workarounds.

Unlike a commercial SDK, Fabric.js doesn’t guarantee backward compatibility. No SLAs. Little roadmap visibility.

Customer Feedback: Why They Walked Away

Here are some soundbites from our customers and prospects on why they decided against building with Fabric.js:

“The quality and UX didn’t meet our standards.”
Product Customizer

“Developer experience was poor. We spent more time debugging than building features.”
B2B SaaS Ad Design

“We needed enterprise support, scalability, and documentation. Open source didn’t cut it.”
Web to print Customer

“Cross-platform support and minimum SDK version constraints were a major blocker.”
Claim Management Application

Voice of the Dev

Beyond enterprise evaluations, developers working with Fabric.js often share their struggles publicly on GitHub, Stack Overflow, and developer forums. Their commentary offers real-world insight into day-to-day frustrations:

“FabricJS doesn’t seem to work well on Mobile.”

Github issue #6980

“Fabric.js object controls don’t work until after a common selection is made. Had to hack around it with extra event listeners.”

StackOverflow

“Trying to integrate Fabric.js with Next.js + Rollup is a mess. Unexpected tokens, config rewrites — it doesn’t play well with modern bundlers.”

GitHub Issue #8444

“We’re blocked by the use of ‘unsafe-eval’ due to our content security policy. Fabric.js needs a rewrite to be CSP-compliant.”

GitHub Issue #9666

Concrete takeaways from developer feedback:

Basic UI control quirks: Common issues with selection handling and control behavior require manual workarounds.
Compatibility friction: Fabric.js does not play well with modern build tools like Next.js and Rollup without additional configuration or patching.
Security blockers: The reliance on unsafe-eval creates CSP conflicts, making it unsuitable for projects with strict security requirements.
Uncertain roadmap: Critical features like async rendering may sit open for years, and niche features important to your use case may never land if they don’t align with community priorities.

While every open source project faces these difficulties, they do represent systemic issues with performance, extensibility, and production-readiness. For teams building long-lived, customer-facing tools, this developer feedback is often a leading indicator of future roadblocks.

Back-of-the-Envelope Cost of Building with Fabric.js

Many teams underestimate the cost of building a design editor from scratch. Here’s a rough breakdown of the time investment we’ve heard from teams who tried:

Task	Estimated Engineering Time
Core canvas-based editor UI (zoom, drag, resize, selection, tool switching)	6–8 weeks
SVG support, snapping, grouping, and layer management	4–6 weeks
Template constraints	3–5 weeks
Cross-platform adaptation (ironing out issues on mobile browsers, defining fallbacks)	4–6 weeks
Bug triage, ongoing maintenance, and refactors (first 12–18 months)	6–9 weeks

Total: ~6–8 months of senior dev time just to get to feature parity with what CE.SDK offers out of the box.

This doesn’t include the cost of QA, product management, or future extensibility. Nor the opportunity cost of what your team could be building instead.

IMG.LY’s CE.SDK: A Ready-Made Solution Built for Growth

By contrast, IMG.LY offers a production-grade SDK built for cross-platform creative tools, backed by a dedicated engineering team and used by major apps across industries.

Category	IMG.LY (CE.SDK)	Fabric.js	Notes
Out-of-the-box UI	✅ Prebuilt modern UI	❌ Build from scratch	CE.SDK ships with full UI/UX patterns
Plugin System	✅ Native plugin architecture	⚠️ Manual code extension	IMG.LY supports formal plugin APIs
Free Drawing Tools	⚠️ Requires integration	✅ Built-in pencil + shapes	Fabric.js excels at shape-level control
Advanced Editing	✅ Vector + raster support, layering, filters	⚠️ Basic vector only	CE.SDK includes pro-grade editing capabilities
Templating System	✅ Dynamic placeholders + constraints	❌ Manual implementation	IMG.LY supports automation workflows
Video Editing	✅ Multi-track timeline editor	❌ Not supported	No video support in Fabric.js
Cross-Platform	✅ Web, iOS, Android, Node, Flutter	⚠️ Web only	IMG.LY has native SDKs and server tools
Asset Integrations	✅ Unsplash, Getty, Brandfolder	❌ Manual setup	CE.SDK integrates assets out-of-the-box
Enterprise Support	✅ SLAs, onboarding, deployment help	❌ Community support only	IMG.LY offers dedicated engineering help
Design File Import Support	✅ PSD, AI, INDD	⚠️ None	CE.SDK supports importing common design file formats
AI Editing	✅ Background removal, integrate any AI model for image/video/text/audio gen	❌ Not supported	AI-native editing pipeline available
Rendering Consistency	✅ Unified engine across platforms	⚠️ Browser dependent	CE.SDK ensures pixel-parity on all targets

When Does Fabric.js Still Make Sense?

This article might read like a put-down, but it’s not supposed to be that. Until now we were simply evaluating Fabric.js with respect to building fully-featured design editing tools. There are, however, a number of well-scoped use cases where Fabric.js excels. These include:

Building an internal tool, proof of concept, or educational app
Creating a custom, single-platform sketching or annotation layer
Developing a low-fidelity prototype with full code-level control
Building canvas collaboration tools such as drag and drop editors or interactive canvases, such as Miro.

An example of where Fabric.js worked well is a tool such as Mockola, a drag-and-drop diagram editor, where shape manipulation, lightweight rendering, and canvas-level control are more important than cross-platform design fidelity. Mockola’s is a good use case, because its emphasis on real-time collaboration and planning rather than high-fidelity design. This means that features like free drawing, drag-and-drop, and JSON-based serialization are a great fit, all of which Fabric.js supports out of the box.

For teams willing to invest in building and maintaining their own editor infrastructure and whose use case does not require cross-platform parity, advanced media editing, or enterprise scaling Fabric.js continues to be a compelling foundation.

However, for products that need to ship quickly, scale reliably, and support modern creative workflows across platforms, IMG.LY’s CE.SDK offers the infrastructure and polish that customers now expect.

The Bottom Line

As MIT CIO Symposium speaker Mark Holst-Knudsen put it:

“You shouldn’t build anything that’s available off the shelf because it’s not a source of competitive advantage if everybody else can avail themselves of it.”

For most companies, building an editor from scratch is not a competitive edge, it’s a distraction from shipping and scaling. And Fabric.js, while capable in the hands of specialists, isn’t a shortcut to the finish line.

IMG.LY’s CE.SDK gives you a head start with advanced editing features, AI automation, template support, plugin extensibility, rendering consistency, and battle-tested scalability across platforms. We put together a data report based on 600+ customers, 28 structured customer interviews, and a self-reported customer survey that speaks for itself when it comes to real-life outcomes.

So before you dive into Fabric.js and start reinventing core functionality, ask yourself: What competitive advantage are you going to build on top of a design editor and which functionality are you better served buying from a trusted vendor?

OpenAI GPT-4o Image Generation (gpt-image-1) API: A Complete Guide for Creative Workflows for 2025

Jan — Mon, 28 Apr 2025 07:55:48 GMT

Update: AI-first Visual Editing

A day after the release of the gpt-image-1 API, we took it for a spin and integrated it into CreativeEditor SDK. Users can now generate images, create variants and use the canvas to compose visual prompts with our design editor. See it in action:

Open AI Editor Demo Page

Introduction

The release of OpenAI’s gpt-image-1 model signals a pivotal shift in the creative developer landscape—one that moves beyond static, one-shot image generation and toward a more dynamic, multimodal interaction model. Until recently, most image APIs followed a predictable pattern: submit a prompt, receive a finished image. The process was useful, but flat. What’s changing now is not just image quality or style fidelity, but the shape of the workflow itself. With gpt-image-1, built on the GPT-4o foundation, developers can start designing creative tools that feel conversational and iterative. This evolution invites a new kind of interface where prompting, tweaking, and refining happen inside the canvas, not outside of it.

For teams building creative editing experience into their app, this moment coincides with the release of IMG.LY’s AI Editor SDK, a powerful, fully integrated toolkit designed for generative workflows. The SDK is already equipped to support interactive image generation, contextual editing, and multimodal inputs, and you can try it today through this live demo.

This guide is a comprehensive introduction to the gpt-image-1 API, but it also goes further. It’s not just about wiring up an endpoint, it’s about rethinking what image generation means in a user-centric product.

From prompt handling to interactive iteration, we’ll walk through how to design creative cycles, not just outputs. This guide explores how to make that shift, how to go from generating images to integrating gpt-image-1 into real creative cycles, where AI becomes a tool that bends to user intent, not the other way around.

Overview of `gpt-image-1`

OpenAI’s gpt-image-1 model, released in April 2025, is the latest evolution in the company’s generative image lineup and marks a turning point in how developers approach visual creation inside applications. Built on the same multimodal foundation as GPT-4o, this model allows applications to move beyond one-shot static generation and instead build toward more conversational, iterative image workflows.

Model Architecture and Capabilities

gpt-image-1 is rooted in GPT-4o’s ability to understand and generate across modalities. It is designed to produce high-resolution images—up to 4096×4096 pixels—based on natural language prompts. The model handles complex scenes with more fidelity than previous iterations and provides improved consistency in how it interprets detailed descriptions. This is particularly relevant for tools that need reliability when turning prompt inputs into design elements.

Parameter Control

Developers working with gpt-image-1 have access to a streamlined set of parameters, here is a subset of the most important ones:

prompt: The primary text input describing the desired image.
size: Choose between “1024x1024”, “1024x1536” (portrait), “1536x1024” (landscape), or “auto” (default, based on prompt).
n: Number of images to generate (default is 1).
response_format: Always returns b64_json. URL outputs are not supported.

Unlike DALL·E 3, gpt-image-1 does not accept style modifiers or quality settings. It is designed for straightforward, high-fidelity image creation driven purely by the text prompt and size selection.

Full documentation of these options is available via OpenAI’s official guide.

Style and Use Case Alignment

By supporting a wide range of stylistic templates, gpt-image-1 positions itself as a flexible backend for everything from marketing collateral to storyboarding tools. The output can be tailored to suit technical illustrations, concept art, or even photorealistic renderings, allowing developers to map visual outputs more directly to brand or product requirements.

Limitations and Future Direction

As of April 2025, gpt-image-1 supports only one image per request and does not offer fine-grained image editing or inpainting. However, its tight coupling with GPT-4o suggests that future iterations may embrace persistent context, conversational refinement, or even integrated image-plus-text exchanges within the same session. For developers building editors or multimodal workflows, the current model lays a strong foundation for these future capabilities.

API Setup and Usage

2.1 Get Access

To start using gpt-image-1, developers must first register for access via the OpenAI platform at platform.openai.com. Access requires an API key, which is tied to your OpenAI account and associated usage limits based on your billing tier. Be sure to confirm that your account is approved for image generation, as availability may differ by region and subscription level. Once authenticated, keys can be created in your dashboard and stored securely in your server or development environment.

2.2 First Image Generation (Node.js Example)

The image generation API for gpt-image-1 can be used directly via OpenAI’s official Node.js client. Below is a complete example showing how to send a prompt and receive an image URL in response:

import OpenAI from 'openai';
import fs from 'fs';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY, // make sure this is securely set
});

async function generateImage() {
  try {
    const prompt = `
    A studio ghibli style illustration of a cyberpunk girl holding a butterfly on her finger.
    `;

    const result = await openai.images.generate({
      model: 'gpt-image-1',
      prompt,
      size: '1024x1024', // or "1024x1536", "1536x1024", or "auto"
    });

    const image_base64 = result.data[0].b64_json;
    const image_bytes = Buffer.from(image_base64, 'base64');
    fs.writeFileSync('butterfly.png', image_bytes);
    console.log('Image saved as butterfly.png');
  } catch (err) {
    console.error('Error generating image:', err);
  }
}

generateImage();

Remember that all outputs from gpt-image-1 are delivered as base64-encoded JSON. Developers should decode this data for display, storage, or further processing within their applications. For complete parameter options and examples, consult the OpenAI Images API guide.

Integrating with CE.SDK

Embedding gpt-image-1 into a creative editor like CE.SDK is about more than just piping an image into a canvas. It reshapes how users interact with content creation, bridging manual design work and AI-driven generation within the same editing environment. Rather than operating as a standalone prompt generator, gpt-image-1 becomes a continuous creative partner inside your editor. For in in-depth technical guide on how to integrate gpt-image-1 stay tuned for our upcoming tutorial, sign up to our newsletter to be notified when it goes live.

Embedding Image Generation in a Creative Editing Workflow

The natural entry point for gpt-image-1 inside CE.SDK is through a dual-mode experience: offering users the option to start either from scratch or from existing context. In “from scratch” mode, a user might open a blank scene and initiate an image generation by writing a prompt for example, “Create a vibrant festival scene at sunset.” The result appears directly on the canvas, immediately editable like any other design element.

Where gpt-image-1 shows its real potential is in “in-context editing.” Here, users interact with existing content—a background, a product shot, or a decorative element and trigger AI enhancements based on that visual context. A user might select an image of a bird, as in the example below and ask for variants, initiate a background swap, or request a change like adding more birds in a conversational interface embedded in the editor. Because CE.SDK treats generated images as first-class canvas elements, context such as positioning, layering, and cropping is preserved throughout the process.

Let’s see what this might look like in practice. We positioned an image of a single bird on our canvas, opening the AI context menu we can now manipulate that image in place using the OpenAI API:

We edit the image and prompt the API to add more birds:

We see that the model correctly identified the type of bird in the picture (seagull) and filled it in with a swarm of flying seagulls.

We can now continue to work with the image, overlaying filters, changing the texture, cropping etc.

Switching Between Manual Edits and AI-Powered Enhancements

A critical design principle when integrating gpt-image-1 is giving users freedom to toggle between manual edits and AI suggestions. Manual edits should always remain possible after generation, e.g. cropping, masking, compositing while users can also seamlessly prompt gpt-image-1 for additional changes without losing prior work. Think of variant generation as a branch: a user picks a generated image and creates “forks” by asking for alternate styles, different lighting, or new thematic elements.

In this setup, the generated image serves as a stable node in the creative graph, while edits and regenerations can attach contextually. This workflow minimizes user frustration by avoiding the “start over” penalty typical of isolated generation APIs. It also opens up more complex creative behaviors, like blending user-drawn sketches with AI-augmented refinements, or iteratively developing an asset library around a consistent visual theme.

An upcoming in-depth tutorial will walk through implementing this multimodal workflow step-by-step, but the key takeaway is that gpt-image-1 shines brightest when it is embedded into a creative loop—not treated as a black-box generator, but as an interactive, iterative design companion.

Prompt Engineering Tips

One of the most overlooked but critical factors in successful image generation is prompt design. With gpt-image-1, prompt engineering isn’t just about describing an image—it’s about steering the model toward intent, tone, composition, and usability. Because the model is capable of rendering complex scenes and a wide range of styles, thoughtful phrasing and contextual hints can dramatically affect the outcome.

Writing for Visual Intent

Start by clarifying what the image is supposed to communicate. Are you looking for atmosphere, action, product detail, or narrative clarity? A prompt like “a city skyline at night” is a starting point, but it leaves too much to chance. Adding elements like “view from a rooftop bar, with glowing signage and overcast haze” gives the model anchors for both composition and mood.

Leveraging Artistic Language

You can further refine outputs by referencing mediums or artistic schools. Prompts that include terms like “in watercolor style,” “oil painting,” ”80s anime aesthetic,” or “studio photography” help the model lock onto a particular visual identity. These cues not only improve stylistic fidelity but also align the output with specific brand or genre expectations, which is especially important for products with a defined look and feel.

Creating Consistency in Branded Outputs

When generating a set of related images, such as social media creatives, campaign assets, or UI visuals, consistency becomes more important than variety. To achieve this, structure prompts with repeatable patterns and include brand elements such as color palettes, motifs, or reference characters. While gpt-image-1 doesn’t yet support persistent memory across requests, consistency can be enforced by prompting with the same style terms, layout descriptions, and constraints. Teams working within CE.SDK can even pair prompt templates with locked canvas layers to preserve composition between generations.

Ultimately, good prompt engineering is not about verbosity but about clarity and constraint. It’s less like writing poetry and more like drafting a product spec. The best prompts are focused, directive, and give the model just enough creative freedom within clear boundaries. However, effective prompting should not burden the user. In practice, the interface should abstract most of the complexity away. Users can be guided toward better outputs through simple UI choices—selecting predefined styles, choosing themes, or adjusting mood settings—while the system dynamically enhances and augments their input behind the scenes. By managing the technical depth invisibly, you enable a creative process that feels intuitive and powerful without ever making prompt engineering the center of the user experience.

Real-World Use Cases

The versatility of gpt-image-1 makes it especially impactful across a variety of industries where visual content creation is either a core product feature or a major operational need. Beyond isolated image generation, the model supports workflows that demand contextual awareness, brand consistency, and iterative refinement, key ingredients for modern digital products.

Web-to-Print

In web-to-print applications, customers expect to customize marketing materials, event invitations, signage, or packaging with minimal friction. By integrating gpt-image-1, developers can offer template-driven personalization where users simply select a theme or enter a few keywords, and receive ready-to-edit visual assets. Combined with CE.SDK’s layout and editing capabilities, this enables a highly interactive experience where generated backgrounds, graphical elements, or themed illustrations can be dynamically placed into editable templates.

Marketing teams rely on high-frequency content creation, often needing visually consistent, campaign-specific assets. gpt-image-1 can assist by automating the generation of background scenes, promotional visuals, and thematic graphics based on campaign briefs. Brands can define style presets aligned with their visual identity, making it easy for marketing teams to produce “on-brand” assets without heavy design overhead. Integrating image generation directly into campaign builders or social scheduling tools amplifies speed without sacrificing quality.

Digital Asset Management (DAM)

Asset libraries often suffer from gaps: missing variants, seasonal versions, or content tailored to different demographics. DAM systems can integrate gpt-image-1 to extend asset catalogs dynamically. Instead of manually commissioning variations, users can generate alternative backgrounds, localize visuals with region-specific elements, or adjust brand visuals for different markets—all from a single master file. With CE.SDK handling structured editing, teams maintain asset consistency while boosting creative flexibility.

E-Commerce

Product visualization remains a huge challenge in e-commerce, especially for smaller retailers. gpt-image-1 can be used to automatically create product lifestyle imagery, context backgrounds, or thematic campaigns without expensive photo shoots. For example, a single shoe photograph can be placed into a generated “urban,” “sporty,” or “luxury” background, customized according to target audiences. When tightly integrated into e-commerce platforms, this enables faster product launches, A/B tested visuals, and localized campaigns at scale.

E-Learning

Educational platforms can harness gpt-image-1 to generate explanatory diagrams, thematic illustrations, or scene-based visual storytelling assets. Instead of relying solely on static stock imagery, teachers, course designers, or even learners themselves can prompt the generation of custom visuals aligned with the curriculum. When embedded into authoring tools, this approach accelerates content creation and enables more engaging, visually enriched learning experiences tailored to specific topics and age groups.

Cost Optimization

While gpt-image-1 opens up impressive creative possibilities, it also introduces new cost considerations that developers and product teams must plan for carefully. Since image generation typically incurs higher API costs than text-based operations, structuring workflows efficiently becomes critical, especially at scale.

Balancing Price, Quality, and Resolution

The cost of generating an image with gpt-image-1 depends significantly on both the requested resolution and the selected quality setting. Higher resolutions like 4096×4096 produce sharper, more detailed results, but they also consume more compute resources-and therefore cost more. For many use cases, especially for previews, lower resolutions such as 1024×1024 or 2048×2048 strike an excellent balance between visual fidelity and API efficiency. Reserving the highest quality settings for final exports or premium workflows can help manage overall spend without compromising user experience.

Image Reuse and Smart Upscaling

One practical cost-saving approach is to design workflows that encourage image reuse. Instead of regenerating similar images for every small variation, applications can create high-quality master images and allow users to crop, edit, or layer additional design elements dynamically. Integrating smart upscaling techniques-for instance, using specialized image enhancement libraries after initial generation-also allows teams to work with smaller base images without sacrificing end-user quality.

Rate Limits and Batching Strategies

Every call to gpt-image-1 counts toward your usage quota, and OpenAI imposes rate limits depending on account tier. To optimize performance and cost, it’s helpful to batch generation requests thoughtfully where possible-for instance, combining multiple prompts into structured queues or allowing users to preview low-res draft versions before finalizing a high-res render. Building this logic into your app’s generation flow not only controls expenses but also improves perceived responsiveness, an important UX factor for creative applications.

By considering cost optimization as an early design constraint rather than a late-stage patch, developers can build scalable, sustainable creative tools powered by gpt-image-1.

Bonus: Starter Kit Repo

We are currently in the process of integrating the new GPT-4o-powered gpt-image-1 model into CE.SDK. As part of this effort, we are preparing a comprehensive Starter Kit will showcase a complete with CE.SDK integration, real-time prompt input, image generation workflows, and best practices for building an AI-powered creative editor.

Both a public GitHub repository and a live demo will be made available soon. If you want to be notified when the Starter Kit launches, you can subscribe to updates here.

This Starter Kit is designed to help developers move beyond simple image generation into building full creative cycles, where users can generate, edit, refine, and remix visuals seamlessly inside the editor.

FAQs

Choosing to work with gpt-image-1 raises a number of practical and strategic questions. Below, we address the most common topics for teams evaluating the model for integration into creative workflows.

How is `gpt-image-1` different from DALL·E 3?

While DALL·E 3 and gpt-image-1 both translate text prompts into images, the underlying architecture and integration paths are different. gpt-image-1 is built on GPT-4o’s multimodal framework, making it better suited for future conversational and iterative workflows. It also offers support for a wider range of styles, higher resolutions up to 4096×4096 pixels, and is positioned for deeper integration into dynamic user experiences rather than one-off generation tasks.

Can you fine-tune or train `gpt-image-1`?

As of April 2025, OpenAI does not allow fine-tuning of gpt-image-1. The model is optimized for broad creative use cases out of the box. Developers seeking more control typically customize the user-facing prompt engineering or combine outputs with structured editing tools like CE.SDK to achieve brand or project-specific consistency.

Is offline support available?

Currently, gpt-image-1 requires access to OpenAI’s cloud APIs. There is no offline inference mode or local deployment option. Teams requiring strict data residency, offline workflows, or private model hosting should consider hybrid architectures where images are generated securely via backend services and then edited locally using embedded tools like CE.SDK.

What about copyright and licensing?

Images generated by gpt-image-1 can be used commercially according to OpenAI’s usage policies, but developers are encouraged to review the latest terms. Outputs are not directly copyrighted by OpenAI or the user, and responsibility for ensuring compliance with branding, likeness, or content standards typically falls on the developer or platform operator. When deploying generation features to end-users, it is good practice to provide clear terms of use and, if needed, additional moderation or review layers.

By addressing these considerations early, teams can integrate gpt-image-1 more effectively and responsibly into creative products and workflows.

Conclusion

gpt-image-1 offers developers a significant opportunity to rethink what image generation can mean inside creative applications. It is not simply a tool for producing pictures on command, but a foundation for building interactive, iterative design workflows where users stay in control of the creative process. When combined with CE.SDK, it becomes even easier to move from static outputs to living, editable canvases that support real-world design needs. As we continue to integrate GPT-4o capabilities, the next wave of creative tooling will be about more than prompting images-it will be about shaping truly collaborative creative environments. Now is the time to start experimenting, iterating, and reimagining the user experience around this new generation of multimodal AI.

How OpenAI's Upcoming GPT-4o Image Generation API Will Change Creative Workflows

Jan — Mon, 14 Apr 2025 10:51:53 GMT

If you’ve been working with image-generation APIs over the past year, you’ve probably gotten used to a certain flow: send a prompt, wait a few seconds, and get a flat image back. It’s a one-shot deal. Useful? Definitely. But not exactly interactive. That’s what will change with OpenAI’s upcoming GPT-4o image-generation capabilities.
IMG.LY, which recently released a suite of AI features for its design editor, is eagerly awaiting the release to expand how users can interact with AI-driven creativity even further.

Update: AI-first Visual Editing

A day after the release of the gpt-image-1 API, we put the UX principles outlined in this post into practice and integrated it into CreativeEditor SDK. Users can now generate images, create variants and use the canvas to compose visual prompts with our design editor. See it in action:

Open AI Editor Demo Page

GPT-4o: Beyond the Prompt-to-Image Pipeline

GPT-4o isn’t just another version of DALL·E. It represents a shift in how developers will integrate AI into creative applications. While DALL·E 3 is powerful it is also somewhat siloed (you send a prompt, you get an image), GPT-4o looks like it will be part of a much more dynamic, conversational model one that accepts both text and image inputs, and could soon generate visual content in context, on the fly, and as part of a back-and-forth user interaction.

If you’ve used ChatGPT recently, you’ve already seen glimpses of this. You can drop an image into the chat, ask GPT to describe or edit it, and get a response that feels fluid and visual. Developers should expect the API version to follow a similar pattern. It likely won’t just be a /generate-image endpoint. Instead, we may be looking at an extension of the chat/completions endpoint that handles multimodal messages. That changes the way you integrate this capability into your application. Rather than simply placing an image generation step in your pipeline, you will have to build your app’s UX around this new user flow. This comes with its own set of unique challenges.

Rethinking the Interface: Prompting as a Conversation

So what does this mean if you’re planning to integrate multi-modal image generation into your own product? For starters, you’ll probably need to rethink how users initiate and refine prompts. In the DALL·E flow, you might offer a text box with a few style dropdowns and call it a day. But in a GPT-4o world, your UI needs to support image inputs, persistent context, and dynamic editing, image gen becomes more like a conversation than a command.

This is where the rubber meets the road. The tools that will benefit most from GPT-4o aren’t static generators but interactive editors. Think collaborative design apps, video editors with generative overlays, or product customizers that let users sketch or upload a photo and then iterate with AI. Put differently, the model output isn’t the endpoint but rather a checkpoint in the creation process.

A Typical Iteration Cycle in a Multimodal Workflow

Here’s a rough sketch of a workflow we might be seeing more of: The user starts with a prompt and an image, maybe a rough sketch or collage created inside an editor, a product photo, or a UI frame. GPT-4o returns a generated image based on that input. The user then edits or annotates the result, maybe adds new prompt text for refinement, and resubmits that combination to further develop the output. This cycle might loop several times: generate, tweak, refine, regenerate.

That’s a fundamentally different interaction model from past AI tooling. It’s less about one-off generation and more about a guided creative journey, where the user is in dialogue with the model. The result: better alignment with the original intent, more control, and more usable creative outputs.

There is an additional, more subjective benefit to this kind of workflow: it gives the user a sense of autonomy again; they are back in the driver’s seat and less at the whim of an inscrutable machine. In many contexts, that makes a difference. Most notably, as we discussed in our white paper on print personalization, the psychological benefit of personalization lies to a large extent in the investment, the sense of ownership that comes about when you create something. “Make it yours” is the common tagline attached to personalization campaigns in e-commerce. That only works if the user exerts more control over the output than iterating over a set of prompts.

The most pithy encapsulation of this paradigm that I have heard is Humans on top, AI on tap.

Persistent Elements and Visual Consistency

One particularly interesting frontier here is character and object persistence. If a user defines a character early in the workflow, either via prompt, image, or a combination, they’ll increasingly expect that character to appear consistently across assets. Think of it as visual continuity, whether you’re generating scenes in a story, slides in a deck, or frames in a video.

If the user of a creative marketing cloud creates a campaign avatar or mascot, that character needs to be consistent within and across campaigns.

Being able to reference earlier outputs, prompts, or style cues gives the user control over not just individual assets but the whole arc of the design narrative. GPT-4o’s ability to maintain that continuity is a game-changer for workflows that involve storytelling, brand identity, or serialized design work.

What to Expect from the API

Technically, if GPT-4o follows OpenAI’s recent design philosophy, you can expect a JSON-based API with a messages array, where content can include both text and image_url types. The output will likely be returned either as an image URL hosted by OpenAI or as base64-encoded image data, depending on the format you request.

That structure plays nicely with modern JavaScript front-end frameworks. React, Svelte, and Vue are all well-suited to async generation flows with visual previews. If you’re already using tools like Zustand or Jotai for local state or something like tRPC or GraphQL for structured calls, you’re in a good position to layer GPT-4o in without breaking the flow.

Trade-offs and Technical Considerations

There are trade-offs, of course. GPT-4o will probably cost more per call than a standard DALL·E 2 or 3 generation. Its latency is still an open question, and the multimodal input support will likely require more thoughtful UX decisions. What happens when a user drops an image and wants to undo just part of the generation? Where do you store prompt context for edits? How do you communicate what’s editable and what’s not?

This is where design and engineering need to work together. You’ll want to build an interface that makes AI feel like a creative partner, not just a backend service. That might mean giving users a visual prompt history or allowing partial re-generations of specific canvas elements. You’ll need sensible fallback states. What happens when generation fails or the result isn’t what the user wanted?

Where IMG.LY’s CE.SDK Fits In

We have already given the questions raised above some serious thought, and most of the complexities introduced by this new workflow are the table stakes for the Creative Editor. So, if you’ve already integrated IMG.LY’s CE.SDK, we have taken care of most of these problems, and you can seamlessly integrate with any AI model. We are actively working on an off-the-shelf integration of the GPT-4o image model once its public API launches.

In general, you can treat GPT-4o’s image outputs as just another layer in the editing canvas, positioned, styled, cropped, and ultimately editable in the same environment as everything else. That’s the real power of multimodal workflows: not just generating but integrating. And once GPT-4o’s API goes live, you’ll want your infrastructure ready to slot it in with minimal friction.

The Loop: Prompt, Generate, Refine

The era of single-shot generation is winding down. What’s coming next is a loop: edit, prompt, generate, refine, repeat. And this loop doesn’t just belong in the backend, it needs to live in the UI, in a way that invites user input, creativity, and correction.

We’ll be publishing more on how this integrates into IMG.LY’s upcoming AI workflows soon. Expect tools that don’t just generate visuals but help teams and individuals work through ideas in real time. Because especially as AI gets more potent, it needs humans on top.

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

Top 5 Generative AI APIs for Creative Apps in 2025: A Developer’s Guide (GPT-4o, Gemini, Firefly, and More)

Jan — Mon, 14 Apr 2025 07:57:51 GMT

If you’re working on creative tooling right now, anything from a lightweight design editor to a marketing automation suite, you’re probably already thinking about or actively working on bringing image generation into the mix. The tech is here, expectations are rising, and if your users can’t type a prompt and get a visual back in seconds, your app might feel like it’s lagging behind.

But choosing which AI model to integrate, and how, isn’t all that straightforward. There’s a growing ecosystem of APIs out there, and they don’t all behave the same way, some are designed for open-ended creativity, others for structured workflows. Some offer pixel-perfect fidelity with fine control, others lean toward rapid ideation. And Importantly in our content not all of them are equally accessible to developers.

This is a guide to help you make sense of it all. What models are available, how do they differ, and what should you consider when embedding them into your product. This isn’t supposed to be a hype piece or a leaderboard, just a clear-eyed look at what’s out there and what’s coming.

OpenAI GPT-4o

GPT-4o is OpenAI’s next-gen multimodal model, currently only available inside ChatGPT. It can take both text and images as input and is capable of generating image outputs in context.

The potential upside is significant. With GPT-4o, you may soon be able to create deeply interactive creative tools where users chat, sketch, and prompt all within a single UI. It’s likely to support richer input types and more natural iteration flows.

The main downside is availability. There’s no API yet, so you can’t build on it directly. It also remains to be seen how OpenAI will expose generation tools—whether through a dedicated endpoint or via the chat interface.

GPT-4o is right for you if you’re planning ahead and want to design for a future where multimodal interaction is the norm. It’s not something you can use today, but it should inform how you architect your UI and prompt handling.

OpenAI DALL·E 3

DALL·E 3 is OpenAI’s current image generation API, available via both the platform and ChatGPT. It translates text prompts into images and is known for interpreting prompts accurately and producing clean, useful visuals.

Its strengths are clarity, commercial readiness, and reliability. It’s easy to use and integrates well into frontend flows that involve text-to-image generation.

However, it lacks features like inpainting, style tuning, or detailed layout control. You also don’t get deep iteration features—each image is a new generation.

DALL·E 3 is a good fit if you want high-quality results from text prompts with minimal complexity. It’s especially useful for marketing visuals, content automation, and simple design tools.

Google Gemini (Imagen)

Gemini, powered by Google’s Imagen models, is available via fal.ai, Makersuite, Vertex AI. It supports not only text prompts, but also sketches and inpainting, making it one of the more flexible APIs for creative work.

Its big advantage is control. You can use sketches to guide composition and make visual edits to generated outputs. That makes it ideal for iterative design processes.

The downside is that it can be tricky to navigate Google’s ecosystem. Access and feature sets can change quickly, and the integration overhead is higher than OpenAI.

Gemini is right for you if your product needs image refinement, visual grounding, or sketch-to-image workflows. It fits e-commerce editors, mockup tools, and design collaboration features.

Adobe Firefly

Firefly is Adobe’s generative image model, integrated tightly into Creative Cloud. It stands out for its licensing model—images are trained on Adobe Stock, meaning they’re cleared for commercial use.

The biggest strength here is trust and integration. Designers already using Photoshop or Illustrator can use Firefly to generate content directly in their layers and work non-destructively.

The drawback is API access. There is no public endpoint for Firefly yet, and its features are embedded in Adobe’s own ecosystem.

Firefly is a strong option if you’re building for agencies, brand teams, or other users with high expectations around copyright and integration with existing Adobe workflows.

Stability AI (SDXL)

Stability AI offers an open-source model suite, with SDXL as the flagship for high-resolution image generation. It supports both text and image inputs and can be run locally or hosted via services like Replicate.

Its biggest advantage is flexibility. You can fine-tune models, build custom workflows, or even run inference offline. It’s ideal for teams that want full control.

The challenge is quality consistency. Compared to closed models like DALL·E, SDXL may require more tuning, and prompt engineering matters more. Hosting and scaling also require more effort.

SDXL is right for you if you need an open, customizable system that fits into a broader pipeline. It’s a solid choice for research tools, OSS projects, and privacy-conscious applications.

Midjourney

Midjourney is a proprietary model with a focus on aesthetic, stylized image generation. It runs exclusively via Discord and is popular for its distinctive look and community-driven prompts.

Its upside is the quality of its visuals, especially for stylized scenes or concept art. Designers often use it as an ideation tool.

The limitation is integration. There’s no API, no SDK, and limited ways to embed it in your own product beyond scraping or bots.

Midjourney is best used as an inspiration engine. If your workflow includes moodboarding or creative brainstorming, it can supplement—but not power—your product.

Hugging Face

Hugging Face is a hub for open models, offering hosted APIs for SDXL variants, Playground v2, and other creative generation tools.

The main benefit is diversity. You can try multiple models, experiment with variations, and deploy quickly using their hosted inference endpoints.

That said, it’s not always ready for production. Some models lack documentation or support, and you may need to piece together features.

Hugging Face is a great choice for experimental projects, prototyping, or if you want to stay vendor-neutral and build your own stack.

Runway Gen-2 and Leonardo.Ai

Runway and Leonardo are rising players at the edge of AI and media. Runway’s Gen-2 supports text-to-video and animated image generation, while Leonardo focuses on style-consistent 2D asset generation.

These platforms bring specialization. Runway is tailored to video and cinematic scenes, while Leonardo offers structured design features for asset creators.

They’re less open from a dev perspective. APIs are limited, and integration support is still maturing.

Use these tools if your use case leans into video, motion, or asset generation for games and content libraries. They’re best when you’re not looking to build your own editor, but to enhance creative capacity.

Quick Comparison

Model/API	Input	Output	Control	API Access	Best For
GPT-4o (OpenAI)	text, image (chat)	image (likely)	medium-high	not yet	assistants, multimodal UIs
DALL·E 3	text	image	medium	yes	content tools, illustrations
Gemini (Google)	text, sketch	image	high	yes	e-commerce, product mockups
Firefly (Adobe)	text	image, layers	very high	no	professional design tools
SDXL	text, image	image	high	yes	custom tools, OSS projects
Midjourney	text	image	very high	no	stylized inspiration
Hugging Face	text, image	image	medium-high	yes	experimentation, open models
Runway Gen-2	text	video/image	medium	yes	motion design, AI video
Leonardo.Ai	text	image	high	limited	game assets, style templates

Conclusion

If you’re building for creative users, especially those used to real-time feedback and control, then how you wrap these APIs into your workflow matters more than which model you use. It’s not just about generating images. It’s about how you let users prompt, refine, iterate, and remix inside your canvas.

That’s the opportunity here. Not just plugging in a model, but designing a loop where generation feels native to creation. The APIs are improving fast. The real challenge, and the real product value, is in how you build around them.

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

How To: Video Generation With Javascript

Antonello — Wed, 22 Jan 2025 09:53:00 GMT

Follow this tutorial to learn how to programmatically create videos in JavaScript directly in the browser.

In 2025 video is firmly established as powerhouse for engagement, gaining even more traction with the rise of short-form videos. After all, as humans, we naturally resonate with video content more than any other type of media.

The problem is that video generation is challenging—and doing it within the browser through JavaScript makes things even more complex. Fortunately, production-grade solutions like CreativeEditor SDK (CE.SDK) make the process not only possible but also easy to implement.

In this guide, you will learn how to programmatically generate videos in the browser via JavaScript using CreativeEditor SDK. We will cover various applications, including merging videos, adding audio tracks, and even integrating AI features.

Let’s dive in!

Why Programmatic Video Generation Matters

Videos are dominating social media, marketing, and online platforms such as e-commerce platforms (e.g., to showcase products). Numerous studies have proven that video is the most effective form of media for marketing, as it resonates deeply with us.

As the demand for personalized and dynamic content increases, traditional video production methods are becoming inefficient. The proliferation of channels with different demands on size and video quality as well as the need to create personalized videos at scale means that automated video creation is becoming indispensable. Most existing solutions run batch processing of videos on the server after gathering input data from various source. However, while server-side processing quickly becomes costly and introduces significant communication overhead client devices have become more powerful than ever.

CreativeEditor SDK bridges that gap by simplifying programmatic video generation in JavaScript within modern browsers. With CE.SDK, you can automate repetitive tasks and define scalable, personalized, and engaging video creation workflows, ensuring each generated video resonates with its target audience.

Get Started with CE.SDK for JavaScript

Now that we established why programmatic video generation is important, you are ready to get started with the tool designed for it: CreativeEditor SDK.

Learn how to integrate CE.SDK for video editing into a vanilla JavaScript application!

You can access the code for this tutorial on Github.

Requirements

The only prerequisites to follow this tutorial are:

A modern browser: CreativeEditor SDK runs directly in your browser’s JavaScript engine, so no additional setup is required. Any browser supporting WASM is enough.
A CreativeEditor SDK license: If you do not have one yet, sign up for a free trial.

If your web application uses Node.js, ensure you have the latest stable versions of both Node.js and NPM installed.

Import the library

In a vanilla JavaScript application, create a JavaScript module file (e.g., video-editor.js) and import the CreativeEditor SDK engine library inside it:

import CreativeEngine from '<https://cdn.img.ly/packages/imgly/cesdk-engine/1.42.0/index.js>';

Note: In this example, the SDK is served from our CDN for convenience. In a production environment, it is recommended to host all assets and libraries on your own servers for improved control and performance.

To use the video-editor.js module in an HTML page, import it with this line:

<script type="module" src="video-editor.js"></script>

Note the type=“module” attribute tells the browser to treat the script as an ES6 module, so that you can use import and export statements.

Alternatively, if you are using a module bundler like Webpack, Rollup, Parcel, or Vite, add the @cesdk/engine npm package to your project’s dependencies:

npm install @cesdk/cesdk-js

In this case, you can then import the headless SDK into your JavaScript code as follows:

import CreativeEngine from '@cesdk/engine';

Well done!

Setting Up Your Environment

In your JavaScript module, right below the library import, initialize the CreativeEditor SDK headless engine using the following code:

// src/video-editor.js

import CreativeEngine from '<https://cdn.img.ly/packages/imgly/cesdk-engine/1.42.0/index.js>';

// your CE.SDK license and user configs
const config = {
  license: '<YOUR_LICENSE>',
};

CreativeEngine.init(config).then(async (engine) => {
  // attach the engine canvas to the DOM
  document.getElementById('cesdk_container').append(engine.element);

  // do something with your instance of CreativeEditor SDK...

  // detach the engine and clean up its resource
  engine.element.remove();
  engine.dispose();
});

Replace <YOUR_LICENSE> with the license key provided in your CreativeEditor SDK subscription.

Ensure that the HTML page importing video-editor.js contains the following <div> element with the cesdk_container ID:

<div id="cesdk_container" style="width: 100%; height: 100vh;"></div>

The CreativeEngine.init() method will mount the editor engine component within this div.

Congratulations! You have successfully integrated the video editing engine.

Load a Video

Use the logic below inside the body of the CreativeEndine.init() callback function to load a video in CE.SDK:

// initialize a new video scene
const scene = engine.scene.createVideo();

// create a page block and attach it to the scene
const page = engine.block.create('page');
engine.block.appendChild(scene, page);

// set the dimensions of the video page
engine.block.setWidth(page, 720);
engine.block.setHeight(page, 1280);

// create a graphic block to hold the video content
// and set its shape to a rectangle
const video = engine.block.create('graphic');
engine.block.setShape(video, engine.block.createShape('rect'));

// create a fill type for the video and set the video file URI
const videoFill = engine.block.createFill('video');
engine.block.setString(
  videoFill,
  'fill/video/fileURI',
  '<https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-drone-footage-of-a-surfer-barrelling-a-wave-12715991.mp4>'
);

// apply the video fill to the video graphic
engine.block.setFill(video, videoFill);

// create a track block to manage the video timeline
// and attache the track to the page and the video graphic
// to the track
const track = engine.block.create('track');
engine.block.appendChild(page, track);
engine.block.appendChild(track, video);

// make the track cover the parent block
engine.block.fillParent(track);

This initializes a video scene using createVideo() and creates a "graphic" block to display the video in a rectangle, adding it to the page. Next, it loads a video from a remote resource and appends it as a track to the parent block.

For a complete explanation of how the above code works, refer to the documentation.

If you comment out the cleanup part, this is what you should see in your browser application:

Notice how the video from the remote URI has been loaded into the

element where CreativeEditor SDK is attached to the page’s DOM.Time to explore more programmatic video generation features!

Programmatic Video Generation Features

Follow the use cases below to see how CE.SDK makes automated video generation in JavaScript easier.Generate Video in Different Formats CE.SDK supports video export in MP4 format (with MP3 audio) with different resolutions and aspect ratios.To export a video with a specific aspect ratio, first adjust the width and height of the parent block to which the video block is attached:

// specify an aspect ratio of 9:16 (ideal for vertical videos)
engine.block.setWidth(page, 720);
engine.block.setHeight(page, 1280);

In this case, the video will be automatically adjusted to fit the specified dimensions. Horizontal videos will be cropped to match the defined vertical outline of the block, while vertical videos will be rendered as they are within the specified block.

Next, you can export the video with the configured 9:16 aspect ratio and a given resolution with:

// define the MIME type for the video export (MP4 format)
const mimeType = 'video/mp4';

// define a callback function to track the progress of video rendering and encoding
const progressCallback = (renderedFrames, encodedFrames, totalFrames) => {
  console.log(
    'Rendered', // log the number of rendered frames
    renderedFrames,
    'frames and encoded', // log the number of encoded frames
    encodedFrames,
    'frames out of', // log the total frames to be processed
    totalFrames
  );
};

// define the video export options
const videoOptions = {
  duration: 5, // video duration in seconds
  framerate: 30, // video framerate (frames per second)
  targetWidth: 480, // target video width in pixels (for the resolution)
  targetHeight: 852, // target video height in pixels (for the resolution)
};

// export the page as an MP4 video
const blob = await engine.block.exportVideo(
  page, // the page containing the design video block
  mimeType, // the MIME type for the video (MP4)
  progressCallback, // the callback to track video rendering progress
  videoOptions // the options for the video export
);

// create an anchor element to trigger the video download
const anchor = document.createElement('a');
anchor.href = URL.createObjectURL(blob);
anchor.download = 'video.mp4'; // output video name
anchor.click();

The exportVideo() function produces a blob video file of the specified MIME type. Note that the export occurs over multiple iterations of the update loop, with a frame being encoded in each iteration.

The targetWidth and targetHeight options are optional. If used, the video will be resized to fit the target dimensions while maintaining its aspect ratio. This is useful for setting a specific resolution for the output video while preserving the parent block’s width and height on the output video. If omitted, the produced video will match the resolution of the width and height specified with setWidth() and setHeight(), respectively.

The progressCallback() function will track and display the video generation progress in the browser’s console:

The final lines of the above code snippet trigger the download operation, mimicking the action of clicking a link to download a resource. This is a common JavaScript technique to programmatically trigger a file download.

Once the export process completes, the output video file (video.mp4) will be automatically downloaded to the browser’s download directory. Open its properties and this is what you will see:

Note the video file’s resolution and the fact that its duration is set to 5 seconds, as defined by the duration attribute.

These options enable you to easily create clips with different lengths, aspect ratios, and resolutions, optimized for various audiences and social media platforms like LinkedIn, Instagram, Facebook, X, TikTok, YouTube, and more.

Add Audio Tracks

Adding an audio track for narration or background music to your clip is straightforward. First, create an “audio” block and add your audio resource to it:

const audio = engine.block.create('audio');
engine.block.appendChild(page, audio);
engine.block.setString(
  audio,
  'audio/fileURI',
  'https://cdn.img.ly/assets/demo/v1/ly.img.audio/audios/far_from_home.m4a'
);

Next, you can easily adjust the volume, and apply fade-in and fade-out effects as follows:

// set the volume level to 80%
engine.block.setVolume(audio, 0.8);

// start the audio after 1 second of playback
engine.block.setTimeOffset(audio, 1);

// set the audio block's duration to 5 seconds
engine.block.setDuration(audio, 5);

Here is breakdown of the functions used above:

setVolume(): Adjusts the audio volume of the block, with a range from 0 (0%) to 1 (100%).
setTimeOffset(): Sets the time offset of the block relative to its parent. This determines when the block starts playing in the timeline.
setDuration(): Sets the playback duration of the audio block in seconds, defining how long the block will play during the scene.

In this example, the audio starts at the 1-second mark and plays for 5 seconds (from 1s to 6s), with the volume set to 80% of the original.

Merge Videos

To stitch multiple video clips together in CE.SDK, you first need to import and create separate video blocks:

// create the first video block
const video1 = engine.block.create('graphic');
engine.block.setShape(video1, engine.block.createShape('rect'));
const videoFill1 = engine.block.createFill('video');
engine.block.setString(
  videoFill1,
  'fill/video/fileURI',
  'https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-drone-footage-of-a-surfer-barrelling-a-wave-12715991.mp4'
);
engine.block.setFill(video1, videoFill1);

// create the second video block
const video2 = engine.block.create('graphic');
engine.block.setShape(video2, engine.block.createShape('rect'));
const videoFill2 = engine.block.createFill('video');
engine.block.setString(
  videoFill2,
  'fill/video/fileURI',
  'https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-kampus-production-8154913.mp4'
);
engine.block.setFill(video2, videoFill2);

At this point, you have two different video clips loaded from separate video files.

Now, assume you want to produce a 10-second video that includes both clips. Start by setting the duration of the page block to 10 seconds with the setDuration() method:

engine.block.setDuration(page, 10);

Then, suppose you want the first clip (video1) to play for 7 seconds, and the second clip (video2) to fade in after that and play for 3 seconds (from second 1 to second 4 of the original second clip). This is how you can set that up:

// set the duration of the first and second video clips
engine.block.setDuration(video1, 7); // video1 plays for 7 seconds
engine.block.setDuration(video2, 3); // video2 plays for 3 seconds

// wait for the second video to be fully loaded before applying the trim options
await engine.block.forceLoadAVResource(videoFill2);
// trim the second video to start at 1 second and play up to second 4
engine.block.setTrimOffset(videoFill2, 1);
engine.block.setTrimLength(videoFill2, 4);

// add a fade-in animation to the second video
const fadeInAnimation = engine.block.createAnimation("fade");
engine.block.setInAnimation(video2, fadeInAnimation);

These are the methods used above:

setDuration(): Defines the duration (in seconds) that a video block will be active during playback.
**forceLoadAVResource()**: Ensures that the audio or video resource (in this case, video2) is fully loaded before applying trim operations.
setTrimOffset(): Sets the start point (in seconds) within the video where playback begins.
setTrimLength(): Defines the length of the video to be played from the trim offset.
createAnimation(): Creates a fade-in animation for the second video (video2) to smoothly transition into the clip.

Finally, add both video blocks to the track block for playback:

const track = engine.block.create("track");
engine.block.appendChild(page, track);
engine.block.appendChild(track, video1); // add the first video to the track
engine.block.appendChild(track, video2); // add the second video to the track
engine.block.fillParent(track);

The result will be:

Note how the second clip fades in after 7 seconds of the first clip, lasting for 3 seconds as specified.

Incorporate Text Variables

Now, suppose you want to display cool words or messages in your videos. You can easily achieve this by adding a "text" block to your page, as shown below:

const text = engine.block.create("text");
// set the vluae of the text variable
engine.block.replaceText(text, "Surfing\nis\nCOOL!");
// set the text color to white and semi-transparent
engine.block.setTextColor(text, { r: 255.0, g: 255.0, b: 255.0, a: 0.8 });
// set the font size for the text
engine.block.setTextFontSize(text, 30);

// positioning the text on an absolute position on the video
engine.block.setWidthMode(text, "Auto");
engine.block.setHeightMode(text, "Auto");
engine.block.setPositionX(text, 130);
engine.block.setPositionY(text, 800);

// append the text block to the page
engine.block.appendChild(page, text);

The video canvas will now display a white “Surfing is COOL!” message as follows:

In this example, the content of the text block is static. However, it can easily be retrieved from a database or any other source for programmatic video generation with different messages, including personalized ones for tailored outreach (e.g., for custom greetings or offers).

Advanced Use Case: Prompt-Based Video Generation Using AI

As highlighted in our recent LinkedIn post, IMG.LY has just experimented with an MVP that turns keywords into fully edited short videos, complete with a script, voiceover, and visuals—all automated.

That is made possible by integrating CE.SDK with an LLM for coding and scriptwriting, ElevenLabs for voice synthesis, and Flux (via fal.ai) for visuals. Specifically, the LLM uses CE.SDK for video composition, animation, and rendering.

This is just the beginning, showing how AI can be combined with CE.SDK to programmatically generate video scripts—or even templates. A possible scenario would be to use the Creative Editor SDK plugin API to extend CE.SDK with custom plugins that adds AI directly to the design editor engine.

For example, you could develop a plugin with an LLM integration that allows users to input a prompt like: “Create a 15-second video with the text ‘Welcome to our App!’ and upbeat background music.” The LLM will process the request, using Creative Editor SDK to automate video creation by leveraging its powerful features.

Alternatively, the AI could produce a video that you can then automatically edit and enhance using CE.SDK capabilities. One thing is certain: the possibilities are endless with CE.SDK + AI!

Conclusion: The Future of Programmatic Video Generation

CE.SDK for JavaScript offers exceptional versatility in automating video workflows, allowing you to easily generate, edit, and render videos with just a few lines of code. You can stitch multiple video files, add background audio tracks, and incorporate dynamic text variables to finally produce videos in the browser—all thanks to the CE.SDK headless engine.

Looking ahead, AI integration with CE.SDK is unlocking even more powerful possibilities, such as prompt-based video content creation and automatic editing directly in the browser.

With the headless engine provided by CreativeEditor SDK, programmatic video generation in JavaScript can be implemented in just a few minutes. Explore the video capabilities of CE.SDK and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

A Modern Video Editor SDK for Your React Native App

Jan — Mon, 06 Jan 2025 10:40:35 GMT

Learn how to integrate IMG.LY’s video editor for React Native into your app and how to leverage its capabilities best.

Why Add a Video Editor to Your React Native App?

Video content is solidifying its status as the most engaging form of digital media. Platforms like TikTok, Instagram Reels, and YouTube Shorts have made video consumption and creation an engrained habit with billions of users. You can harness this trend by adding a video editor to your app significantly enhancing user engagement and retention.

React Native, with its ability to create cross-platform applications from a single codebase, is a perfect match for IMG.LY’s CreativeEditor SDK Video Editor. It ensures seamless performance on iOS and Android, powered by the same unified graphics engine across platforms.

Whether your app focuses on social media, marketing, or eCommerce, integrating a video editor empowers users with a creative tool set while elevating the overall experience.

Getting Started: Integrating the Video Editor in React Native

Requirements

Before diving into the integration, ensure your environment meets these requirements:

React Native: 0.73+
iOS: 16+
Swift: 5.10 (Xcode 15.4)
Android: 7+

To get started, add the @imgly/editor-react-native package to your project:

npm install @imgly/editor-react-native

Setting Up the Editor

Once the package is installed, initialize the editor by importing the necessary modules and creating an instance of EditorSettingsModel. You’ll need a license key, which you can obtain from the IMG.LY dashboard.

Here’s how to set up and launch the video editor:

import IMGLYEditor, { EditorSettingsModel } from '@imgly/editor-react-native';

export const openVideoEditor = async (): Promise<void> => {
  const settings = new EditorSettingsModel({
    license: 'YOUR_LICENSE_KEY',
  });

  const result = await IMGLYEditor.openEditor(settings);
};

This launches the editor with the Video Editor preset, enabling users to trim, cut, and enhance their videos with filters, text overlays, stickers, and music.

Use Cases: Building a TikTok-Like Experience

Now that the video editor is integrated into your React Native app, let’s explore some key use cases and how to configure the editor to support them.

Short-Form Video Creation

For apps targeting TikTok-style short-form video content, prioritize features that simplify the editing process:

Timeline Control: Allow users to trim clips and sync overlays like stickers or music.
Filters & Effects: Offer filters to let users add certain moods to their videos with themes like retro, high contrast, or pastel tones.
Text & Stickers: Add captions and playful stickers for videos that are often consumed on mute.
Music & Audio: Let users add soundtracks or sound effects, with a library of trending music for inspirational background music.
Video Templates: Provide ready-made templates that users can customize for specific occasions or themes.

This setup allows influencers and brands to create professional-looking videos that they can share across social media platforms with ease.

Influencers and Marketing

If your app serves influencers or businesses, the ability to create polished, on-brand videos is key. Features to consider:

Branded Templates: Pre-designed templates aligned with specific brand aesthetics, simplifying content creation for marketing campaigns.
Watermarks: Add brand logos or watermarks to videos, ensuring creators and brands maintain visibility.

These features empower influencers and businesses to quickly generate content that’s ready for distribution across platforms.

E-commerce and User-Generated Content

Video editing is a powerful tool for e-commerce apps, enabling sellers and customers to create engaging product demos, unboxing videos, reviews and tutorials. Features like trimming, filters, and music tools make it easy to produce compelling content that enhances the shopping experience.

For example:

Vendors can create promotional videos for products.
Customers can share authentic reviews or tutorials, increasing trust and boosting sales.

Customization Options with CreativeEditor SDK

The React Native plugin offers a range of customization options to adapt the editor to your app’s needs:

UI Customization: Change themes, colors, and layouts to match your app’s branding.
Video Presets: Configure presets for specific use cases, such as limiting video length or optimizing for a particular resolution.
Custom Assets: Add unique filters, stickers, and fonts to create a personalized experience for your audience.
Templates: Use the CreativeEditor SDK Web UI to create custom templates, enabling users to start with professional-grade designs.

Conclusion

Integrating a video editor into your React Native appwill improve your UX, and help boost engagement, retention, and potential distribution of your product, whether you’re building a social media platform, an influencer tool, or an e-commerce app.
With CreativeEditor SDK, you can create a TikTok-like video editing experience or offer specialized tools for businesses and creators.

By following the steps in this guide, you can empower your users to create professional-quality video content directly within your app. Explore the full capabilities of the SDK by visiting the React Native documentation and getting started today.

Stay tuned for updates, and don’t hesitate to reach out with any questions.

Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

Javascript Video Editing: Ultimate Guide for Developers and PMs

Jan — Fri, 20 Dec 2024 14:06:55 GMT

When developing or integrating a JavaScript-based video editor for the web, you must consider a number of factors to ensure the solution is both efficient and robust. This post is the definitive guide for anyone embarking on such a project. It explores the key technologies involved, their strengths and weaknesses, and how different use cases influence the choice of tech stack. We’ll examine diverse use cases, from lightweight, browser-based editors for quick edits to more advanced tools requiring complex processing and rendering. We’ll then discuss how these scenarios drive the selection of features and technology.

Additionally, we will show you the best open-source solutions that can accelerate development. This technical analysis will help you make an informed “build vs. buy” decision, ensuring you select the right approach for your project. Throughout this post, we will use the example of the IMG.LY’s JavaScript video editor (see here for a demo), showing you how these considerations shaped its architecture and feature set. You will learn practical insights into the decision-making process behind a successful web-based video editor.

Modern web technologies like WebGL, WebCodecs and WebAssembly have enabled browsers to efficiently perform resource-intensive tasks such as video editing that had hitherto been the sole domain of desktop applications. As a result Javascript based video editing tools such as veed.io have increased in popularity and users expects ever more sophisticated video editing capabilities inside modern video based web apps.

Video Editing Use Cases

Let’s first outline use cases and requirements of video editing applications on the web. These will inform our discussion of technical features of respective solutions below and grant a conceptual framework for evaluating potential benefits and drawbacks of each solution.

Simple Video Editing

Let’s start with the base case, simple video edits including trimming, cropping and resizing and simple effects such as adjusting brightness or saturation. This is usually sufficient feature set to support video editing for the following use cases:

Sales Outreach Videos: Sales teams often need to quickly edit customer-specific videos to personalize their outreach. This may involve trimming irrelevant portions, adding company logos, or adjusting brightness to ensure visual clarity.

Messaging Applications: These often require basic editing tools to allow users to crop, trim, or apply simple filters to shared videos, ensuring they’re concise and visually appealing.

Screencasting: Screencasting tools benefit from trimming and resizing capabilities to focus on key parts of recorded screens. Adding effects like brightness adjustment can make tutorials clearer and more professional.

CMS Systems: Content management systems may offer built-in video editing to help users optimize media assets for specific platform requirements, such as resizing for web embeds or adding subtle branding.

Screen Recording Applications: Screen recording applications often include simple editing options for cleaning up recorded content by trimming extraneous sections or cropping to highlight the most relevant parts.

Video Annotation

Next, users might want to add an additional layer of information to videos and overlay other assets, such as stickers, shapes, overlays, or text. Crucially these assets need to be time-based—that is, users need control over when the asset is shown so that particular video sequences can be referenced. This introduces the need for a timeline to arrange different video components relative to each other in time, as well as features like voice-over and audio track support.

E-commerce Reviews: Sellers and reviewers can annotate product demo videos with callouts, price tags, and feature highlights to make the content more engaging and informative.

Claims Management / Insurance: Insurance companies can use annotation to highlight key details in submitted video claims, such as timestamps of damage or explanatory text overlaying critical sections of footage.

Real Estate: Realtors might annotate property walkthroughs by adding labels, dimensions, or descriptive text overlays to highlight key features of the home or property.

Educational Applications: Instructors can use annotation tools to emphasize key moments in lectures or tutorials, such as overlaying text with formulas or concepts, or adding visual shapes to guide attention.

Productivity Tools: Users can annotate meeting recordings with timestamps, text notes, or overlay diagrams to summarize key decisions or action points.

Healthcare and Telemedicine: Medical professionals might annotate diagnostic videos or procedure recordings to explain findings or highlight areas of interest for training purposes.

Customer Support & Onboarding Tools: Companies can add annotations to video tutorials or troubleshooting guides to direct users through specific steps or highlight important information.

Video Composition

As we’re moving away from single media editing to creating video compositions of several types of media such as audio tracks, text, images, animations, and effects to create appealing visual designs, a well-designed timeline becomes even more important. Users need to manage many different video components in time, requiring user-friendly ways to browse and integrate external assets. This editor variant is most relevant for the following use cases:

Marketing Tech (Promotional Videos): Marketers can create visually stunning promotional videos by combining custom animations, background music, and text overlays for branding.

Social Media (Stories and Reels): Social media creators can quickly craft short, engaging videos that combine multiple assets like stickers, animations, and dynamic transitions tailored to platform-specific formats.

Event Highlight Reels: Event organizers can compile videos, photos, and music into cohesive highlight reels that encapsulate the essence of the occasion.

Template-based Video Creation

For many applications, users need starting points and examples for their designs. Starting from a blank canvas is rarely necessary for most use cases. Take a simple product video that includes promotional text, a brand logo, and animations. There is no need to reinvent the wheel for these types of videos. Instead, design applications should offer template libraries to accelerate user workflows.

Once we introduce workflows involving several stakeholders—such as designers setting up a certain brand framework and marketers working with and adapting these preconfigured designs—templates need to be able to constrain what edit operations adaptors can perform. These requirements are particularly important for these use cases:

Digital Asset Management: Organizations can manage and deploy branded video templates across teams, ensuring consistent design and messaging.

Social Media Publishing: Social media managers can utilize templates for quick turnaround on platform-specific video formats, such as Instagram Stories or LinkedIn posts.

Marketing Tech: Marketing teams can rely on pre-designed templates to churn out campaign videos at scale while maintaining brand consistency.

Training Videos: HR or L&D departments can adapt existing templates to quickly produce training materials customized for different teams or scenarios.

Creative Automation for Video

Finally, you can leverage data and automation to generate variations of your templates at scale, significantly boosting the productivity of use cases where users need to either test a large number of designs, publish to different channels with different requirements, or adapt designs to many slightly different instances, for example, menu designs for a franchise.

Marketing Tech: Marketers can generate hundreds of ad variations for A/B testing or localization by dynamically adjusting text, visuals, or calls to action.

Social Media Publishing: Social media teams can automate the creation of videos tailored to platform specifications, such as aspect ratios or resolution, while retaining core branding.

E-commerce: Retailers can produce personalized product showcase videos for different customer segments, featuring tailored offers, pricing, or recommendations.

Hospitality Industry: Hotels and restaurants can dynamically generate location-specific promotional videos showcasing seasonal offers, menus, or events.

Essential Video Editing Features

The above use cases are enabled by a set of features from simple transforms to complex compositions. Some video editors are focused more on manipulating individual video clips while others are more oriented towards video creation providing fully-fledged multi-media composition tool. This concise overview serves as a reference for evaluating video editing solutions:
Transforms

Cut, Trim, and Split: Basic editing tools for segmenting video clips.
Resize and Scale: Adjusts clip dimensions, ensuring proper fit or aspect ratio.
Crop and Rotate: Controls framing and orientation.
Zooming Capabilities: A UX feature enabling detailed editing and close-up views for precision.

Adjustments

Basic Adjustments: Controls brightness, contrast, and other visual enhancements.
Filters and Effects: Sets the visual appearance and atmosphere of videos.
Audio Tracks and Mixing: Supports multi-track audio adjustments and volume balancing.
Text and Overlay Options: Allows visual enhancements with text, captions, and emojis, providing opacity and layering options.

Composition

Multi-media Composition: Overlaying images, stickers, text as well as audio tracks are essential for creating videos through composition.
Multi-Track Editing: Elements and layers need to be position relative to each other in time allowing the creation of complex composition.
Canvas-Based Editing: Provides a flexible workspace for positioning and layering media elements.
Timeline Management: Split, join and arrange clips on a timeline. Manages the positioning of clips and timing for seamless transitions.
Animations: Make static elements dynamic and control their behavior in time. Can be use to create videos from scratch and enhance the storytelling workflow of existing ones.

The Technology Landscape

Some time in the early 2000s the web started to be viewed as a platform to run applications not just a distributed database to deliver static html documents. While Javascript development was not yet at a point to serve as foundation for the kind of application development we see today, Adobe Flash filled the role of development platform. Video editing in the early days of the web was mainly provided by Flash-based tools involving simple transform operations such as trimming and basic transitions. Java applets were another way to deliver processing intensive programs via the browser while executing them outside of the browser inside the JVM. These early attempts were either too clunky, since they could not run natively in the browser or lacked the performance to be useful for high quality editing.

When HTML5 came onto the scene in the early 2000s it became possible to handle multimedia content natively in the browser. Coupled with significant performance advances of Javascript and the introduction of WebRTC this opened the door for video editing on the web.

Some early cloud-based editors, such as WeVideo and Magisto, while more convenient and user-friendly than its predecessors still suffered from performance issues due to latency and lacked the depth of features found in desktop software.

In the past few years advances such as WebAssembly and Javascript libraries built upon it e.g. ffmpeg.js have revolutionised browser-based video editing. User can now perform complex editing tasks like multi-track timelines, video effects and real-time in-browser rendering. Modern web-based solution have even come to rival desktop apps, because they can take advantage of GPU acceleration, modern APIs like WebGL and UI frameworks such as React.

Let’s have a look at the state of the art web technologies that allow us to create performant video editing experiences inside the browsers. We are going to explore their relative strength, their complexities and conclude with when to best use each of these technologies:

Canvas API

The HTML5 Canvas API should be most familiar to most modern web developers, since it’s a familiar and popular way to render 2D graphics in the browser. It’s use for video editing is limited and restricted to simple overlays, trimming, composition and animation:

Complexity: Canvas is the least complex of the technologies introduced below and overlaps with the WebGL feature set to some extend, it’s API is well-known to most web developers.
Advantages: Canvas is lightweight, compatible across all browsers, and easy to implement, making it a great choice for simple use cases that require basic effects and no timeline.
Downsides: While well suited for simpler tasks Canvas lacks the GPU-acceleration of WebGL, hence it is not recommended for resource intensive operation or performance sensitive use cases.
When to Use: Choose Canvas for simple editing tasks, like trimming or basic overlays, or if you need a fallback technology that works well on legacy browsers and devices.

WebGL

WebGL opens up the power of GPU-accelerated graphics to the browser, making it the default choice for any use case requiring high performance rendering. As such modern Javascript video editing solution are well served by webGL when looking to render visual effects in real time, add transitions or facilitate multi-layer compositions by leveraging the GPU.

Complexity: WebGL comes with a steep learning curve especially if you are unfamiliar with 3D graphics processing, shaders and GPU programming.
Advantages: WebGL delivers high performance for processing complex video effects and can handle significant volume of data without latency. That makes it ideal for advanced, highly responsive video editing features.
Downsides: It can be challenging to engineer modular and maintainable WebGL code, debugging shaders and ensuring consistent performance across devices is an art in itself. WebGL might also be overpowered for simple editing use cases that do not justify the added complexity.
When to Use: Use WebGL if you’re creating a video editor with high-performance demands, particularly if real-time effects, transitions, or multi-layer compositing are needed.

WebCodecs API

Next up in our arsenal of formidable web technologies is a recent addition to modern browsers, WebCodecs, provide direct access to hardware accelerated video encoding and decoding.

The WebCodecs API is a recent addition to the browser, allowing direct access to hardware-accelerated video decoding and encoding. This API makes it easier to handle high-resolution video without delays, thanks to efficient decoding directly in the browser.

Complexity: WebCodecs is relatively easy to use for those familiar with media formats. However, you may need other technologies for rendering and complex editing.
Advantages: This API offers quick access to hardware acceleration, ideal for managing large video files. With it, you can efficiently handle high-definition video playback and streaming.
Downsides: Browser support for WebCodecs is still expanding, so compatibility might be a concern. Additionally, WebCodecs alone doesn’t provide a full editing suite; it needs to be combined with rendering and compositing technologies.
When to Use: WebCodecs is perfect if you need efficient decoding and encoding, especially for handling high-definition playback or live streaming.

WebAssembly (Wasm)

The next performance frontier browser pushed towards was enabling near-native performance by compiling languages like C++ or Rust into a format that efficiently runs in Javascript. This format is WebAssembly, which makes it possible to handle computationally expensive tasks like encoding, decoding or frame manipulation inside the browser.

Complexity: Working with WebAssembly is significantly more complex that the technologies discussed above. Developers need knoweldge of C++ or Rust as well as experience compiling to Wasm.
Advantages: WebAssembly enables browsers to handle complex video processing tasks, such as encoding and decoding, with near-native performance. Libraries such as FFmpeg build upon WebAssembly to make familiar video tools available to web developers.
Downsides: Development with Wasm is complex, and debugging can be a challenge. Additionally, Wasm modules may increase load times.
When to Use: Use WebAssembly when building native libraries for complex operation and expensive computations, such as encoding and decoding.

Our article on building a video editor with React and Wasm gives you a real-world starting point for your own apps.

MediaStream API

The MediaStream API is essential for gaining easy access to live video sources from within the browser, such as webcams. If your video editing app includes a feature for real-time recording or streaming the MediaStream API is an essential tool.

Complexity: MediaStream exposes a fairly simple API and is relatively easy to learn for experienced web developers.
Advantages: This API is well-suited for video feeds with low latency such as recording or streaming and does require addition encoding or decoding.
Downsides: Any use case requiring post-production video has to rely on additional technologies or frameworks, as the MediaStream API is limited to live feeds. The quality of the stream depends on the input source and can vary.
When to Use: Use MediaStream for applications with live recording or streaming functionality.

Web Audio API

The final web technology essential for video editing is the Web Audio API. Whether you want to add effects or mix multiple audio tracks, this API offers sophisticated tools for audio manipulation.

Complexity: For developers unfamiliar with audio processing the learning curve can be stepp, however the API is well designed and there is a host of educational resources.
Advantages: Web Audio API allows precise control over audio enabling your app to offer audio adjustments such as effects or reverb and multi-track editing.
Downsides: It can be challenging to accurately synchronize audio with video, especially if your app includes a real-time component.
When to Use: Opt for Web Audio API when audio editing is a core feature of your video editor.

Using These Technologies Together

Together, these technologies can be use to build a comprehensive browser based editor that is virtually indistinguishable from its desktop based counterparts. Different parts of the video editing workflow can be handled by each technology, while some choices have to made depending on the complexity and requirements of your projects, e.g. whether you’ll need resource intensive processing or an integration will real-time sources.

Video Decoding - WebCodecs API (for efficient video loading and playback).
Rendering - WebGL (for high-performance effects) and Canvas API (for simpler editing features).
Complex Processing - WebAssembly (for encoding/decoding and using native libraries like FFmpeg).
Audio Processing - Web Audio API (for mixing and adding effects to soundtracks).
Live Recording - MediaStream API (to capture live video from devices).

Open Source Javascript Video Editing Libraries

Fortunately in many cases someone has already done the heavy lifting and abstracted those browser APIs into simple to use libraries and frameworks. We’ll explore the most prominent ones distinguishing between projects that are for processing raw video and those that already provide an API for you to use. It is important to note that the libraries introduced below do not provide a video editing UI, unfortunately there are no open source video editors offering an end user interface that meet the standards for inclusion in a production grade application. None of the projects we evaluated met the demands we place on third-party libraries with respect to robustness, stability, maintenance and future development. However many can provide good starting points and references on how to build your own UI such as Reactive Video for React UIs.

FFmpeg.js

FFmpeg.js is the JavaScript port of the popular FFmpeg library compiled into WebAssembly. It enables developers to manipulate and process raw video files entirely in the browser. You can trim, convert formats, extract audio, and apply filters to videos without needing external software. However, FFmpeg just gives you the raw toolchain for building video editing features, you are still tasked with building the entire UX and editing workflows. We have an extensive guide on FFmpeg that you can consult to explore its capabilities and get started learning FFmpeg syntax.

Advantages:
- Most comprehensive library for web based video and audio processing.
- Fully client-side, no need for server-side processing.
- Highly flexible, allowing fine-grained control over video workflows.
Downsides:
- Fairly complex for beginners with a steep learning curve, requires familiarity with FFmpeg’s command-line syntax.
- FFmpeg is licensed under LGPL which makes it ill-suited for most commercial projects.
**When to Use:**FFmpeg.js is best suited for projects with custom requirements and UI making it necessary to exert fine-grained control over video processing tasks, especially for use cases like format conversion, advanced editing pipelines, or serverless video workflows.

The following two libraries allow you to programatically create and edit videos, we focus on the most stable, popular libraries here, while there might be promising up and comers we want to ensure a degree of dependability of the third-party code to be used in production systems.

Remotion

Remotion is a React-based framework that enables developers to edit or create videos programmatically in a declarative fashion using React components. React’s declarative syntax makes it comparatively simple to manage video content and Remotion provides a convenient UI for previewing and editing videos. While its studio UI cannot be embedded directly, Remotion can serve as the graphics processing engine for custom UIs. Although it is a commercial project, it offers a generous free tier for individuals and small businesses.

Advantages:
- Leverages React’s declarative model for seamless video creation.
- Provides robust video preview capabilities for iterative development.
- Flexible integration with custom-built UIs.
- Generous free tier for non-commercial and small-scale projects.
Downsides:
- The built-in studio UI cannot be embedded, requiring developers to create their own UI for embedding.
- React-specific, limiting use in non-React environments.
**When to Use:**Use Remotion when building React-based video applications that require programmatic video generation or dynamic video content. It’s ideal for teams already familiar with React and for applications where embedding is not a core requirement.

Etro.js

Similarly to Remotion, Etro.js is a JavaScript framework for programmatic video editing, it is framework agnostic and provides an API for simple composition of layers, effects and exporting videos.

Advantages:
- Includes support for advanced effects using GLSL shaders.
- Designed for in-browser video workflows.
Downsides:
- Minimal community support compared to larger projects.
- Requires additional effort to create a full UI.
When to Use: Suitable for applications where programmatic control and customization are key.

Extensibility

An important consideration when deciding to buy a prebuilt video editing solution is the degree of customization and extensibility required by your use case versus how critical speed to market is. While most SDKs provide enough flexibility by exposing cor API to build a solution that best suits your needs, this level of technical control comes at the expense of higher development and maintenance efforts. White-label solutions on the other end of the spectrum offer limited customization often relying on iFrames for embedding while providing maximum speed to market. If you are still in the phase where you need to make the business case for video editing features these might be key to allowing to quickly iterate. To learn more about weighing the pros and cons of SDK vs. White-label explore our extensive guide on the topic.

IMG.LY’s CE.SDK achieves extensibility by providing an interface for plugins that allow developers to add custom features and UI elements.

Cross Platform Video Editing

While this post is dedicated specifically to Javascript video editing an important consideration for choosing a video editing tech stack is whether there exists an intermediary output format that preserves editing operations so that videos can be edited on other devices and platforms. This is important for collaborative video editing use cases as well as use cases where videos are captured and basic edits performed on mobile and the finishing touches performed on the desktop.
These consideration are particular pertinent for the Real Estate, Telemedicine and Productivity use cases described above, since they involve either a multi-device or collaborative component.

At most common libraries or SDKs can achieve that by offering support for some form of serialization, that is exporting the edit operations along with the unedited video to then reapply edits upon import.

IMG.LY’s CE.SDK, on the other hand, is the only video editing SDK that is truly cross-platform. That means it is built atop a single creative engine that is portable to any platform. Whether iOS, Android, Desktop, or the Web, every platform uses the same underlying tech and uses the same custom format for scenes which contain all assets and edits making error-prone serializations obsolete.

Furthermore:

iOS and Android make use of the same underlying API.
Cross platform feature parity is baked into the cake. Since core functionality is implemented at the engine level, features are guaranteed to be available on both platform, although the timeline might differ somewhat.
Designs are 100% interoperable and consistent between platforms. Exporting and importing design files across platforms works seamlessly and final renderings are guaranteed to be consistent.

Javascript Video Editing & Generative AI

Generative AI is impacting video editing and generation across the board. Video generation obvious application of generative AI, but AI is also changing the way we we compose, create an edit videos in more subtle ways.

Features such as automated transitions, color grading, voice enhancement, voice over generation, background removal or inpainting are

Currently, these capabilities are offered by multiple specialized providers, such as ElevenLabs and OpenAI’s Sora. Firstly, that means modern Javascript video editors need to be open enough to serve as integrators for various, programmatically combining voice tracks, video clips, text, audio and other media before presenting users with an editing UI. Secondly, traditional video editing needs to enable the human in the loop. AI is still imperfect and manually refining unchangeable artifacts through prompts is inefficient and tedious. Instead the output by AI must remain editable and any editor solution smoothly interface with those artifacts to allow users to add the necessary polish, swap out media and regenernate individual parts of the scene.

To explore what this might look like in practice have a look at our CEO Eray Basar’s latest project building an AI video generator on top of CE.SDK.

Conclusion

When building or integrating a JavaScript video editor, developers and PMs have to navigate a complex technology landscape and make myriad decisions along the way. This guide has covered everything from essential use cases and feature sets to the cutting-edge technologies that underly browser-based video editing. Whether your project requires basic trimming tools or advanced multimedia composition, understanding the benefits and downsides of the various solutions like Canvas, WebGL, WebCodecs, WebAssembly, and more is crucial for choosing the right stack.

We’ve also explored how open-source tools like FFmpeg.js, Remotion, and Etro.js can accelerate development, while SDKs like IMG.LY’s CE.SDK offer unmatched extensibility and cross-platform compatibility. For those navigating the impact of generative AI on video editing, the need for seamless integration and user-centric design remains important.

If you’re looking for a robust, modern video editing solution for your web application, explore our showcase and start a free trial.

CreativeEditor SDK Now Supports React Native on iOS & Android

Jan — Thu, 19 Dec 2024 11:57:22 GMT

We’re thrilled to announce the release of our official @imgly/editor-react-native package, bringing the power of CreativeEditor SDK to the React Native ecosystem for iOS and Android!

CreativeEditor SDK enables developers to seamlessly integrate design, video, and photo editing features into their applications. Over the past few months, we’ve been focused on enhancing mobile editor solutions—refining features and expanding to new use cases, including the Video Editor.

To make CreativeEditor SDK accessible to even more developers, we’re expanding our support to include mobile cross-platform frameworks. We started with Flutter and are now adding React Native as another highly popular frameworks for building truly cross-platform apps.

Getting Started

Getting started with CreativeEditor SDK in React Native is simple! First, make sure you meet the following minimum requirements:

React Native: 0.73+
iOS: 16+
Swift: 5.10 (Xcode 15.4)
Android: 7+

Then, add the @imgly/editor-react-native package to your project using your favorite package manager.

To launch the editor, you’ll need an instance of EditorSettings - you can use EditorSettingsModel for convenience. All you need to get started is a license key, which you can obtain from your IMG.LY dashboard.

Here’s an example of opening the Design Editor:

import IMGLYEditor, { EditorSettingsModel } from '@imgly/editor-react-native';

export const openEditor = async (): Promise<void> => {
  const settings = new EditorSettingsModel({ license: 'YOUR_LICENSE_KEY' });
  const result = await IMGLYEditor.openEditor(settings);
};

Editor UI Presets

The React Native plugin builds on our native editor solutions, allowing you to leverage the existing editor UI variants. At launch, you can choose from:

Simply choose the editor solution that best suits your use case. By default, the Design Editor opens, but switching to other solutions is straightforward.

Android and iOS Setup

To setup our module in your application, you’ll need to add the expo-build-properties dependency which handles the necessary adjustments to your native folders.

Android

For Android, you’ll need to make a few adjustments to your app.json:

Set the minSdkVersion to 24 (Android 7) or later.
Add the IMG.LY Maven repository.
Ensure your Kotlin version is compatible (e.g., 1.9.10).

Here’s an example configuration:

{
  "expo": {
    "android": {
      "minSdkVersion": 24,
      "extraMavenRepos": ["<https://artifactory.img.ly/artifactory/maven>"],
      "kotlinVersion": "1.9.10"
    }
  }
}

iOS

On iOS, ensure your deployment target is set to 16.0. You can configure this in your app.json file:

{
  "expo": {
    "ios": {
      "deploymentTarget": "16.0"
    }
  }
}

Check out the Getting Started Guides or the examples on GitHub for a complete reference.

The Only Truly Cross-Platform SDK

If you’re working with React Native, you’re likely focused on faster time to market, reduced development complexity, and feature consistency across platforms. The CreativeEditor SDK shares these goals, using a single, unified creative engine that ensures:

Consistent APIs: iOS and Android share the same underlying API.
Feature Parity: Core functionality is implemented at the engine level, guaranteeing feature availability across platforms.
Interoperable Designs: Export and import designs seamlessly across platforms with consistent rendering.

Fully Customizable UI

The React Native plugin offers flexibility for theming, styling, and integrating callbacks into your editor’s workflow. Soon, we’ll introduce more advanced customization options, such as enabling or disabling features by default.

Refer to our documentation for detailed instructions on customizing the editor to match your branding and use case.

What’s Next?

To showcase the capabilities of our React Native plugin, we’ve prepared a dedicated example application available on GitHub. You can explore it to see the editor in action.

Additionally, our documentation provides all the details needed to get started, and you can visit the @imgly/editor-react-native npm page to begin your integration.

Remember to grab your license from your dashboard after starting a free trial, and let the CreativeEditor SDK unlock your users’ creative potential.

Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

Plugins Part 2: Custom Panels

Jan — Wed, 23 Oct 2024 12:53:27 GMT

In Part 1 of our series, we introduced a plugin system for customizing UI elements and quick actions within the CreativeEditor SDK.

Now, in Part 2, we’re introducing Customizable Panels, a new feature that gives developers even more control over the editor. These panels can handle tasks like guiding users through complex workflows, interacting with external APIs or gathering detailed user input—allowing for anything from simple tweaks to fully custom workflows.

Let’s take a quick look at where panel plugins fit within the CE.SDK Web UI. To dive in instead, consult the documentation or explore one of the examples below.

Extension Points in the UI

Before customizing the UI, it’s important to understand the user experience that makes the editor intuitive for creating and editing designs. Below are the key areas of the editor UI where your plugins can be integrated:

The quick actions we introduced in part one are used to implement simple one-click actions to edit canvas elements, primarily inside the inspector bar and canvas menus. This release opens up the panel as additional UI element to facilitate more involved features requiring UI interactions beyond a simple click.

New Possibilities with Panels

Panels serve as customizable UI elements that provide plenty of room for your plugin’s unique requirements. Here’s how developers can utilize them:

Guide Users Through Workflows
Implement panels to assist users in navigating workflows or controlling specific plugin settings, tailoring the editor experience to meet individual needs.

Complex Interaction with External AI or Services
While it’s possible to connect to third-party service using quick actions, some use cases require more user interaction. To support these cases you can now create panel UIs that connect with third-party APIs to gather data or trigger processes, such as AI-driven design tools or content generation.

Collect Detailed User Input
Use panels to collect more complex input from users, such as advanced export options, template customization, theming or batching design adjustments.

These new capabilities round out our goal of empowering developers to shape CE.SDK themselves, tailoring the user experience to fit perfectly with their particular use case.

Example Plugins

To demonstrate the potential of panels, we’ve developed three example plugins that showcase different approaches to extending the CreativeEditor SDK’s functionality.

Plugins can be grouped into two categories:

Functionality-Enhancing Plugins: These integrate external services, assets, or custom code. For example, the QR embed plugin adds a feature by interacting with a third-party API.

Custom Editor Behavior Plugins: These plugins interact directly with the CE.SDK Engine API, enhancing the SDK’s existing functionality by introducing new modes of interaction or extending the user interface.

QR Code Embed Plugin

The first category of plugins is exemplified by this QR plugin, which add the ability to embed a QR code directly in the canvas. The plugin interacts with a third-party QR code generator API and displays a panel where users can enter a URL and customize the color of the QR code.

The next two examples illustrate plugins from the second category, which interface with the Engine API to enhance CE.SDK functionality:

Export Options Panel

This showcase provides a reference implementation for a panel that allows users to configure export options including export formats and file types, such as PNG, JPEG, or PDF, which pages to export and at what quality and resolution. A corresponding showcase is also available for video scenes, allowing users to control resolution and quality of the exported video file.

Template Adaptation UI

In this example, we add a panel that collects user input via a form, which is then used to dynamically adapt a design template. Users can input text, select colors, or upload images, and the template is modified based on these choices. This demonstrates how panels can streamline workflows, making it easier for users to customize templates to fit their specific needs.

Outlook

With the release of panels, we are advancing our mission to empower developers to fully customize and extend the CreativeEditor SDK. These new features for creating sophisticated, interactive plugins will unlock exciting opportunities for innovation, whether integrating external services, designing dynamic user interfaces, or automating complex workflows.

We encourage you to explore the power of panels in your own projects. Visit our plugin documentation to get started, and let us know what you’re building! As always, we’re here to support your journey in redefining creative editing.

Stay tuned for more updates, and feel free to contact us if you have any questions or need assistance.

Over 3,000 creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK Video Editor Now Available on Android

Jan — Tue, 22 Oct 2024 11:44:02 GMT

We are excited to announce the release of CE.SDK’s video editing and video content creation tools for Android!

After the successful iOS launch, CE.SDK’s professional video editing capabilities are now available for Android. Whether your app focuses on social media, e-commerce, or marketing technology, CE.SDK enables users to create high-quality videos directly within your app.

Getting Started

Getting up and running with an instance of the CE.SDK video editor for Android is easy.

After downloading your license (get your trial license here) from your dashboard, simply invoke VideoEditor as composable function, Activity or Fragment.

Our Video Editing Backstory

Since we launched VideoEditor SDK almost ten years ago, we have seen an astounding range of creative and unique use cases of our SDK in customer projects. From Cameo connecting celebrities and their fans in new ways, over Multibrain supercharging their users marketing campaigns with editable video content, to Zigazoo powering video content within their social network for kids while putting safety and parental control first.

Since then, millions of users have experienced IMG.LY’s VideoEditor SDK through our customers’ products, shaping how they expect to interact with and modify videos. Continuous feedback from both our customers and their users has also influenced the way we approach video interface design, helping us refine and innovate along the way.

Our goal was to combine the reliability and robustness of the VideoEditor SDK with true cross-platform compatibility and the flexibility to meet diverse needs. By applying the lessons we’ve learned, we aimed to deliver an exceptional video editing user experience straight out of the box. We described how those learnings shaped the design of the mobile video editing interface.

Key Capabilities

With CE.SDK’s sleek timeline interface, your Android users can now take their video editing from simple transforms and adjustments to complex composition and multi-track editing:

Timeline-Based Editing
Split, trim, arrange video and audio tracks. Position elements such as images and stickers on the timeline and control their duration.

Multi-Track Editing
Work with multiple video and audio tracks, allowing for complex composition.

Transform & Enhance
Crop, flip, rotate, and use presets for common video formats. Add filters, effects, blur and essential adjustments.

Customizable Templates
Create and import templates using our web video editor, to jumpstart your users’ video composition and designs.

Asset Integration
Integrate your own asset libraries for audio tracks, sound effects, images, and more.

Cross-platform Interoperability
CE.SDK’s video capabilities extends across iOS, Android and the Web, enabling cross-platform applications to offer seamless video editing capabilities. Users can edit and save work in progress and pick up on any device.

Check out the showcase app for an interactive demo of these features.

Customization

The ability to tailor the video editor to your app’s particular use case has been a front and center concern in its development.

UI Customization: While theming is currently limited to dark and light model, future releases will introduce more flexible theming, allowing you to modify the look and feel of the editor to match your app’s branding. Furthermore, we expose callbacks that make it possible to hook into editor events and overlays for custom dialogues.
Configure Video Presets: You can configure the editor to create video presets based on your users’ needs. For instance, you may want to fix the video’s aspect ratio or optimize for certain resolutions, depending on the target platform.
Video Templates: With the CE.SDK video editor web UI, you can create an unlimited number of video templates, such as collages, text designs, and animations, providing your users with professional-quality starting points for their video designs.
Assets: Provide custom filters, fonts, and stickers to enhance the user experience. For niche apps, this can be an excellent way to engage users by offering content that feels unique to your community.

CE.SDK’s new Android video editing capabilities are perfect for a variety of use cases, including:

Social Media Applications: The CE.SDK video editor for Android excels at short-form video, ideal for social media applications, whether your users are publishing to other networks or you run a network yourself.
Ecommerce: Allow your users to create engaging product showcases, ads, and promotional videos, testimonials or reviews.
Marketing Tech Platforms: Users can leverage our versatile templating to easily create polished social media and marketing graphics from within your app.

What’s Next?

This Android release is just the beginning! We are about to launch a Camera UI for Android and are working on more advanced template creation features. So stay tuned.
To see some of these features in action, check out our iOS demo, where the next phase of innovation is unfolding.

Get Started

Explore our Video UI Showcase and Documentation for Android to discover how CE.SDK can streamline your content creation projects and make Android video editing easier than ever.

Over 3,000 creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

A Modern Video Editor SDK for your Flutter App

Jan — Fri, 13 Sep 2024 11:26:15 GMT

Video has been the only type of content with steadily growing demand, and it will remain a staple and an expected medium for users to create and consume.

With platforms such as TikTok, Instagram Reels, and YouTube Shorts dominating and shaping user habits, the ability to create and edit videos directly within your app can significantly increase engagement.

In this post, we’ll show how to integrate a video editor into a Flutter app using CreativeEditor SDK and customize it for specific use cases, like creating TikTok-style short videos.

You can check out the code for the default and custom Flutter video editor on GitHub

Why add a Video Editor to Your Flutter App?

The last decade has made one thing clear: video content is king. Particularly, short-form dominates digital content. At the forefront of this trend are apps like TikTok, which made people more comfortable creating, editing, and sharing bite-sized clips.

These apps have capitalized on their ability to let users apply filters, and add audio overlays, text, and more, empowering users at all skill levels to make professional-looking videos with minimal effort.

A host of different use cases stand to profit from this trend by enhancing their user experience with video editing features, particularly marketing tech tooling and apps with a messaging or social media component.

In general, video creation lowers the threshold for user-generated content and can positively impact distribution and product engagement.

Flutter, Google’s UI toolkit for building apps across platforms from a single code base, is a perfect fit for IMG.LY’s Video Editor. What often motivates the choice to use Flutter is that It ensures performance, flexibility, and a seamless user experience on iOS and Android devices. Similarly, our SDK is designed to offer a native cross-platform experience, while relying on the same underlying graphics processing engine, ensuring consistency and interoperability.

Getting Started: Integrating the Video Editor in Flutter

Let’s dive into how to set up a video editor in your Flutter app using CreativeEditor SDK.

Requirements

Before we get into the details, here’s what you’ll need:

Flutter 3.16.0+
Dart 2.12.0+
iOS 16 or later (as the video editor is currently only available for iOS, Android coming soon)
Swift 5.10 and Xcode 15.4

First, ensure your pubspec.yml file includes the necessary dependencies:

dependencies:
  flutter:
    sdk: flutter
  imgly_editor: 1.34.0

Once your environment is set up, you’re ready to start coding.

Setting Up the Editor

After adding the imgly_editor dependency, import the package in your code:

import "package:imgly_editor/imgly_editor.dart";

The next step is to initialize the editor by providing an instance of EditorSettings. This setup requires a license key, which you can obtain from CreativeEditor SDK, and an optional userId that helps track monthly active users (MAUs) across different devices.

Here’s a basic example of how to open the video editor:

class VideoEditorSolution {
  /// Opens the editor.
  void openEditor() async {
    final settings = EditorSettings(
        license: "YOUR_LICENSE",
        userId: "YOUR_USER_ID"
    );

    const preset = EditorPreset.video;

		/// Open the editor and retrieve the editing result.
    final result = await IMGLYEditor.openEditor(
        preset: preset,
        settings: settings
    );
  }
}

This launches the editor with the video preset, enabling users to trim, cut, and add filters, text overlays, music, and more to their videos.

Use Cases: Building a TikTok-Like Experience

Now that we have integrated the basic editor configuration into your Flutter app, let’s revisit some prominent use cases and how to configure the editor to support them.

Short-Form Video Creation

For a short-form video, TikTok-like user experience, the most important factor is the ease of use of any video editor and content such as stickers or audio overlays.

The following SDK capabilities can help you achieve that goal:

Control Video Duration and Position in Time: Users should be able to trim video clips on a timeline and position them relative to other tracks, such as audio clips.
Filters & Effects: Filters are essential for setting the tone of the video. Whether users want a retro, high contrast, or soft pastel look, filters provide creative control over the visual style.
Text & Stickers: Allow users to add text captions—especially important for content often watched on mute. Stickers and emojis can add a playful element to videos.
Music & Audio: Like TikTok, you can allow users to add background music or sound effects. Providing a library of popular tracks or sound snippets can enhance the content creation experience.
Video Templates: Give your users a head start with customizable video templates for different themes, topics, and occasions. IMG.LY’s CE.SDK comes with a web video editor to help build and deliver these templates to your Flutter app. You can explore a range of different templates in our video editor demo.

By integrating these features, you’ll empower users to create engaging, shareable content, elevating your app’s video creation experience to the level of consumer-grade social media apps.

Influencers and Marketing

If your app is targeting influencers or businesses, the ability to quickly create branded video content is key. You can offer:

Branded Templates: Pre-made templates that align with a brand’s style, allowing users to easily drop in their video footage and text while maintaining brand consistency. Again, check out the web editor demo to see how to build video templates.
Watermarks: Adding a watermark or logo as a brand signifier to videos can help with brand visibility and make sure content always points back to the creator.

This setup allows influencers and brands to create professional-looking videos that they can share across social media platforms with ease.

eCommerce and User-Generated Content

Video is a powerful medium to showcase products, especially on marketplaces that can leverage user-generated content and enable vendors as well as customers to create product or review videos.

For example, users could create unboxing videos, product reviews, or tutorials using the same trimming, filter, and music tools offered in a TikTok-style editor. These videos can then be shared on social media or used within the app to enhance the shopping experience.

Likewise, showcasing products using authentic demo videos tap into familiar patterns of video consumption and can boost sales for vendors.

Customization Options with CreativeEditor SDK

We have designed the Flutter plugin to be adaptable to match your brand and use case-specific requirements. Soon the plugin will also support more advanced customization options such as the ability to activate or deactivate features by default.

UI Customization: Modify the look and feel of the editor to match your app’s branding. We provide options for theming, styling color palettes, and callbacks to hook into editor events.
Video Presets: You can create custom video presets based on your users’ needs. For instance, you may want to limit video length or optimize for certain resolutions, depending on the target platform.
Video Templates: You can use the CE.SDK web UI to generate any number of video templates including collages, text designs, and animations to give your users professional-looking starting points for their video designs.
Assets: Provide custom filters, fonts, and stickers to enhance the user experience. For niche apps, this can be an excellent way to engage users by offering content that feels unique to your community.

Conclusion

Integrating a video editor into your Flutter app will improve your UX, and help boost engagement, retention, and potential distribution of your product, whether you’re building a social media platform, an influencer tool, or an e-commerce app. With CreativeEditor SDK, you can create a TikTok-like video editing experience or offer specialized tools for businesses and creators.

By following the steps outlined in this post, you can bring professional-level video editing features to your users. Explore CE.SDK’s video capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

Jan – IMG.LY Blog

How 600+ Teams Shipped Creative Editing in 14 Days

The IMG.LY Impact Report

What Customers Unlock

1. Ship in days, not quarters

14 days vs. 6 months: how long it takes to ship creative editing

2. Open new pricing and market tiers

3. Lift your own conversion and engagement

4. Free your design, marketing, and support teams

5. Compound instead of freeze

In-house builds freeze. CE.SDK customers compound.

6. Become deal-eligible and privacy-credible

The economics, in one table

When building in-house still makes sense

Methodology

Frequently asked questions

Only 31% of custom software projects succeed

Building a CapCut-Like Video Editor with CE.SDK and AI

Build this with AI in minutes

Introduction

Architecture Overview

Step 1: Project Setup

Why so many packages?

Step 2: The React Component

Step 3: Creating the Video Scene

Browser support

Step 4: The CapCut Dark Theme

Setting the base theme

Custom CSS overrides

Responsive scale

Step 5: AI Plugin Integration

The unified AiApps approach

Provider categories explained

The proxy server requirement

Background removal

Step 6: UI Customization with the Component Order API

Adding the AI button to the dock

AI options in the canvas context menu

Custom export button in the navigation bar

Step 7: Wiring Generated Audio into the Asset Library

Step 8: Custom Labels with i18n

The Complete File Structure

What We Get Out of the Box

Going to Production

1. License key

2. Proxy server

3. Disable dry run

Key Takeaways

Resources

From Prompt to Editor: Running CE.SDK Inside ChatGPT with the Apps SDK

CE.SDK as a ChatGPT App

Working with MCP in Practice

Why This Pattern Matters

Beyond OpenAI: The MCP UI Standard

Introducing CE.SDK Renderer: The Missing Piece of Creative Automation

Why the CE.SDK Renderer Matters

Rendering Has Been the Last Barrier to Full Automation

What the CE.SDK Renderer Delivers

GPU-Accelerated Export Performance

Fully Licensed Video on the Backend (H.264/H.265)

Built for Scale

Perfect Visual Fidelity

How It Works

Async Exports

Batch Rendering

Example Use Case: Hyper-Personalized Video Ads at Massive Scale

Licensing Options

Open-Source Renderer

AV-Licensed Renderer (Fluendo)

Conclusion

How to Embed an Editable InDesign Template in Your Website

Why Editing InDesign Files in the Browser Matters Now

The Current Landscape: What’s Possible (and What Isn’t) with InDesign on the Web

Beyond Adobe: Existing Alternatives

Where IMG.LY’s CE.SDK Fits In

Introducing the CE.SDK InDesign Importer

A Stand-Alone Module, Built for Integration

How It Fits into the CE.SDK Ecosystem

What the Importer Delivers

Real-World Use Cases & Workflows