CE.SDK – IMG.LY Blog

Starter Kits: A New Way to Use and Configure Our Editor

João — Wed, 22 Apr 2026 17:14:54 GMT

When we built the first version of CE.SDK’s mobile editors, we optimized for speed and ease of integration. We designed a set of ready-made solution editors: Photo, Design, Video, Postcard, and Apparel Editor. Each is built around the use cases we believed would meet the needs of most of our customers. With one component and a license key, you had a full creative editor running in your app. That approach got products to market fast, and for many teams it was exactly the right starting point.

But products evolve, and so do the needs around it.

As integrations matured, teams needed deeper configuration, more granular control, and the ability to shape the editor into something uniquely theirs. What starts as “give us a working photo editor” becomes “give us our photo editor.”

That’s why, starting with CE.SDK v1.73 on iOS and Android, CE.SDK evolves from a set of preconfigured solution editors into a single, flexible Editor foundation paired with Starter Kits: complete, working editor projects that you own, customize, and control entirely.

Here’s what that means and why it’s worth it.

What changed

Previously, you integrated CE.SDK by dropping in a solution-specific component: PhotoEditor, DesignEditor, VideoEditor, ApparelEditor and PostacardEditor. Each came with built-in UI, default behaviors, and preconfigured toolbars. You could adjust some of it through configuration APIs, but the foundation was managed internally by our SDK.

With v1.73, those components are replaced by a single Editor entry point that starts minimal without any configuration and a set of Starter Kits that provide the full experience you’re used to.

Each Starter Kit matches a previous solution: Photo, Design, Video, Postcard, and Apparel. You download it, drop it into your project, and you have the same editor as before.

With Starter Kits, the setup becomes explicit and lives in your own codebase. Instead of relying on built-in defaults inside a prebuilt solution, you start from a ready-made configuration structure that you can inspect, own, and change directly.

And if none of the starter kits match your needs, you can build your own editor from scratch using our configuration directly.

Before, you were always starting from one of the predefined solutions and trying to bend it to fit your use case, which was limited and often frustrating. Now, the building blocks are there for you to assemble however you want.

What are the benefits?

The full picture in one place

Configuration used to be spread across multiple APIs and documentation pages. It wasn’t always clear what you could customize, where to find the right setting, or whether a particular behavior was even exposed for modification.

With a starter kit, the configuration is the documentation. Open the configuration file, and you see every callback, every toolbar item, every setting, the complete scope of what the editor does.

This also makes production behavior safer. Critical parts of the editor setup are now explicit in your code rather than implied by SDK defaults. Your team can see exactly what is configured, why, and review changes through your normal development process.

More control and capability over your editor

Starter kits unlock customizations that were simply not possible before: restructure how the editor initializes; inject custom logic between loading phases; defines layouts from the ground up; build workflows that are unique to your product; and more.

The old configuration APIs drew a line between “customizable” and “not customizable.” Starter kits erase that line. If you can see it in the code, you can change it.

In simple terms, you can now adapt the editor to your product, instead of adapting your product to the editor, and this was one of our main drivers.

Opt in, never opt out

Until now, an SDK update could introduce visible changes to your editor, like a new toolbar button, a different menu option, a changed default, without you asking for it. Your release process had to account for UI changes that came from a dependency, not from your team.

With CE.SDK v1.73, that dynamic is reversed. SDK updates bring new features, engine improvements, performance gains, and bug fixes. But your editor UI stays exactly as you left it. New features and components become building blocks you adopt on your own schedule, documented, with examples, and ready when your implicit decide to add them. Nothing shows up in your product unless you put it there.

Two paths, your choice

Start from a Starter Kit
Download the one that matches your use case, run it, and customize from there. Each kit is a proven, production-ready structure that handles the complexity of editor setup so you don’t have to figure it out from scratch. Most teams will take this path.

Build from scratch
If you have very specific requirements or prefer to architect everything yourself, the Editor view accepts raw configuration directly. The starter kits serve as comprehensive, living reference implementations. Even if you never use them directly, they show you exactly how every piece fits together.

The engine you rely on hasn’t changed

It’s worth saying clearly: the core of CE.SDK, the rendering engine, export pipeline, asset handling, performance characteristics, and everything that powers the creative experience is unchanged. We improved how you configure and integrate the editor, not what powers it.

The Tradeoff

The old one-component integration was genuinely convenient for getting started. With starter kits, you have a small module in your project, a handful of well-organized files that you own and maintain.

We believe this is the right tradeoff for any team building a real product. The moment you need to customize anything you’ll be able to clearly structured and to modify without limits.

At its core, this change is about moving from implicit behavior to explicit control, and we believe that is the better long-term contract with our customers. Your editor should not change in meaningful ways unless you decide it should. Your upgrade path should be predictable. And your team should be able to understand the integration by reading its own code.

Getting Started

We’ve written detailed migration guides that walk through every change with before-and-after examples. Each guide also links to the starter kit repositories for your platform:

	iOS	Android
Migration Guide	Upgrading to v1.73	Upgrading to v1.73
Photo Editor	GitHub	GitHub
Design Editor	GitHub	GitHub
Video Editor	GitHub	GitHub
Postcard Editor	GitHub	GitHub
Apparel Editor	GitHub	GitHub

We’re here to help with the transition. If you have questions, reach out to our support team or open an issue on the starter kit repositories.

CE.SDK v1.69 Release Notes

Neslihan — Fri, 27 Feb 2026 18:29:53 GMT

v1.69 is the biggest developer experience release we’ve shipped. Agent Skills collapse the time from idea to working editor from days to under a minute. Production-ready Starter Kits eliminate days of boilerplate setup. PPTX and Canva Importer lets users bring their existing designs directly into your editor. And a comprehensive video overhaul across Web, iOS, and Android closes the gap between CE.SDK and native professional editing apps.

Let’s dive in!

Introducing IMG.LY Agent Skills for CE.SDK

The biggest change to CE.SDK’s developer experience since launch.

AI agents have changed how developers build. With Agent Skills, CE.SDK works natively with your AI agent.

Install the plugin for Claude Code or Cursor and your AI coding agent becomes a CE.SDK expert with bundled access to guides, API references, and starter kits across 10 web frameworks. No external services, no MCP servers, no context-switching.

What your developers can now do:

/cesdk:build — Describe a use case in plain language. The agent detects the framework, pulls the right starter kit, and scaffolds a working project autonomously.
/cesdk:explain — Ask any CE.SDK implementation question and get answers adapted to the specific framework and experience level.
/cesdk:docs-[framework] — Retrieve guides and API references directly from the IDE, without leaving the editor.

For your product, this means: developers prototype in minutes, not days. Non-technical team members can now build and extend functional editors independently. And onboarding new engineers to CE.SDK shrinks from a multi-day ramp to a single session.

→ Quick Start & Full Announcement
→ Explore Agent Skills Documentation

Launch in Minutes with Production-Ready Starter Kits for Web

Previously, getting CE.SDK running in a real product meant adapting complex demo projects: stripping features, adjusting architecture, and undoing assumptions baked into example code.

Starter Kits replace all of that with pre-configured, production-ready editor UIs.

Specialized kits for Photo, Video, and Design editors are scoped, clean, and immediately deployable. Each kit starts with a focused feature set. You explicitly opt in to what you need, so your UI stays performant and your users don’t see capabilities they can’t use.

For your product, this means: a functional, shippable editor in minutes rather than days of setup. Clean architecture from day one, without accumulating technical debt.

→ Explore Starter Kit Documentation

Let Users Bring Their PPTX & Canva Designs to CE.SDK

Some customers don’t start from scratch. They have existing creative assets locked inside PowerPoint and Canva — and until now, bringing those into CE.SDK required manual recreation.

The new PPTX Importer removes that barrier.

Users can import native PowerPoint files or export Canva designs as PPTX and bring them directly into your CE.SDK-powered editor, with layouts, elements, and structure intact.

→ Try the PPTX Import Demo
→ View PPTX npm

Professional-Grade Video Editing Across Platforms

This release expands CE.SDK’s video features across web & mobile platforms.

Control Video Speed (iOS & Android)

Users can now adjust the playback speed of video and audio clips via a dedicated Speed UI. This has been one of the most-requested mobile features and is now available on both platforms.
→ View Video Speed Documentation iOS
→ View Video Speed Documentation Android

Create Video Groups in the Timeline (Web)

Multiple clips can now be combined into a single manageable unit with synchronized timing, trimming, and a unified timeline representation.

This enables complex, multi-layer content production and speeds up the design process.

Video Timeline Now Grows Automatically (Web)

The timeline now automatically adjusts its height based on content. Manual track configuration is gone, and complex projects are easier to pick up for your new users.

Use a Clutter-Free iOS Timeline (iOS)

Non-video tracks now show a single thumbnail per clip, reducing clutter in multi-layer timelines on your iOS app.

Full Changelog

See all technical details, breaking changes, and performance improvements in the v1.69.0 Changelog.

Thank you for building with IMG.LY.

Building a CapCut-Like Video Editor with CE.SDK and AI

Jan — Thu, 26 Feb 2026 22:50:34 GMT

Build this with AI in minutes

This tutorial is a great way to understand how CE.SDK works under the hood. But if you just want to get to the result, you can build this entire editor using IMG.LY Agent Skills, no manual setup required.

Install the skills, then run:

/cesdk:build Build a CapCut-like video editor with a dark theme, multi-track timeline, AI video/image/audio generation, background removal, and MP4 export.

Want to understand a specific concept in depth? Use /cesdk:explain — for example:

/cesdk:explain How does the video timeline and block hierarchy work?

Introduction

CapCut has set the standard for accessible video editing. Its dark UI, intuitive timeline, and AI-powered features make it feel like a professional tool that anyone can use. But what if you could build something similar embedded directly in your own web application in minutes? Yes, minutes!

In this tutorial, we’ll walk through how we built a CapCut-like video editor using IMG.LY’s CreativeEditor SDK (CE.SDK) for React. The finished editor includes:

A multi-track timeline for video, audio, text, and captions
Trim, split, and join operations on video clips
A CapCut-inspired dark theme built with CSS custom properties
AI video generation (text-to-video, image-to-video) via Minimax, Kling, and Pixverse
AI image generation (text-to-image, image editing) via RecraftV3, IdeogramV3, and GPT Image
AI audio generation (text-to-speech, sound effects) via ElevenLabs
AI text generation (copywriting, translation) via Anthropic Claude
Background removal — client-side, powered by WebAssembly/WebGPU
MP4 export directly from the browser

The entire editor is a single React component with roughly 180 lines of JavaScript and 100 lines of CSS.

Architecture Overview

CE.SDK has two main layers:

CreativeEngine: The headless core that manages scenes, blocks, assets, and rendering. It handles the video timeline, playback, and export entirely client-side.
CreativeEditor UI: A pre-built, customizable UI layer that wraps the engine with a dock, inspector, timeline, canvas, and navigation bar.

Plugins extend both layers. The AiApps plugin, for example, registers AI providers with the engine and injects UI components (dock buttons, canvas menu items, generation panels) into the editor.

Step 1: Project Setup

We scaffolded a React project with Vite and installed CE.SDK alongside the AI plugin packages:

npm create vite@latest capcut-like-editor -- --template react
cd capcut-like-editor
npm install

# Core SDK
npm install @cesdk/cesdk-js

# AI plugins
npm install @imgly/plugin-ai-apps-web
npm install @imgly/plugin-ai-video-generation-web
npm install @imgly/plugin-ai-image-generation-web
npm install @imgly/plugin-ai-audio-generation-web
npm install @imgly/plugin-ai-text-generation-web

# Background removal (runs locally via WASM/WebGPU)
npm install @imgly/plugin-background-removal-web onnxruntime-web@1.21.0

Why so many packages?

CE.SDK follows a modular plugin architecture. Each AI capability is a separate package with its own provider modules. This means you only bundle what you use if you don’t need audio generation, don’t install @imgly/plugin-ai-audio-generation-web. The @imgly/plugin-ai-apps-web package is the unifying layer that brings them all together into a single dock panel.

Step 2: The React Component

CE.SDK provides a first-class React wrapper via @cesdk/cesdk-js/react. The <CreativeEditor> component handles mounting, initialization, and cleanup:

import CreativeEditor from '@cesdk/cesdk-js/react';

const config = {
  // license: 'YOUR_CESDK_LICENSE_KEY',
};

const init = async (cesdk) => {
  // All setup happens here — theme, assets, plugins, UI customization
};

export default function VideoEditor() {
  return (
    <CreativeEditor config={config} init={init} width="100vw" height="100vh" />
  );
}

The config object is passed to the engine at creation time. The init callback fires once the cesdk instance is ready — this is where all our customization lives.

Pitfall: Silent init errors. The <CreativeEditor> component swallows errors thrown inside init. If something fails, the editor loads but appears broken with no console output. Always wrap init in a try/catch:

Step 3: Creating the Video Scene

A video editor needs three things at startup: asset sources, a video scene, and a timeline.

// Load built-in asset libraries (stickers, shapes, filters, typefaces, etc.)
await cesdk.addDefaultAssetSources();

// Load demo content (sample videos, images, audio) + enable upload slots
await cesdk.addDemoAssetSources({
  sceneMode: 'Video',
  withUploadAssetSources: true,
});

// Create a video scene with timeline
await cesdk.createVideoScene();

createVideoScene() sets up the scene hierarchy that powers the timeline:

Scene
 └── Page (represents the video canvas — 1920x1080 by default)
      ├── Track (video track — holds video clips in sequence)
      ├── Track (overlay track — text, stickers, images)
      ├── CaptionTrack (subtitles synced to playback)
      └── Audio (background music, voiceover, sound effects)

Each element on the timeline is a block with timing properties: timeOffset (when it appears) and duration (how long it plays). The engine handles rendering each frame, compositing layers, and synchronizing audio.

Browser support

Video editing relies on modern web codecs (WebCodecs API), which are available in Chromium-based browsers (Chrome, Edge, Brave). Safari and Firefox support is limited.

Step 4: The CapCut Dark Theme

CapCut’s visual identity is defined by its deep charcoal backgrounds, teal accent colors, and subtle elevation layers. CE.SDK’s theming system maps perfectly to this through CSS custom properties.

Setting the base theme

cesdk.ui.setTheme('dark');

This activates CE.SDK’s built-in dark theme. But we want to go further — we need CapCut’s specific color palette.

Custom CSS overrides

CE.SDK scopes all its UI under .ubq-public with data-ubq-theme and data-ubq-scale attributes. We override the CSS custom properties to inject our palette:

.ubq-public[data-ubq-theme='dark'][data-ubq-scale='normal'],
.ubq-public[data-ubq-theme='dark'][data-ubq-scale='modern'] {
  /* Deep charcoal backgrounds — darker than CE.SDK's default dark */
  --ubq-canvas: hsl(220, 15%, 8%) !important;
  --ubq-elevation-1: hsl(220, 13%, 12%) !important;
  --ubq-elevation-2: hsl(220, 12%, 15%) !important;
  --ubq-elevation-3: hsl(220, 10%, 18%) !important;

  /* High-contrast white text */
  --ubq-foreground-default: hsla(0, 0%, 100%, 0.92) !important;
  --ubq-foreground-light: hsla(0, 0%, 100%, 0.55) !important;

  /* CapCut's signature teal/cyan accent */
  --ubq-interactive-accent-default: hsl(190, 85%, 48%) !important;
  --ubq-interactive-accent-hover: hsl(190, 85%, 42%) !important;
  --ubq-interactive-accent-pressed: hsl(190, 85%, 36%) !important;

  /* Subtle borders — barely visible, like CapCut */
  --ubq-border-default: hsla(0, 0%, 100%, 0.08) !important;
}

The key design decisions:

Property	Value	Why
`--ubq-canvas`	`hsl(220, 15%, 8%)`	Near-black with a slight blue tint — matches CapCut’s canvas area
`--ubq-elevation-1/2/3`	12% → 15% → 18% lightness	Subtle elevation steps create depth without harsh contrast
`--ubq-interactive-accent-*`	`hsl(190, 85%, 48%)`	CapCut’s teal — used for buttons, selections, and progress bars
`--ubq-border-default`	8% opacity white	Nearly invisible borders that only appear on close inspection

Responsive scale

We also configure responsive scaling so the editor adapts to touch devices:

cesdk.ui.setScale(({ containerWidth, isTouch }) => {
  if ((containerWidth && containerWidth < 768) || isTouch) {
    return 'large'; // Bigger touch targets on small/touch screens
  }
  return 'normal'; // Standard desktop sizing
});

Step 5: AI Plugin Integration

This is where the editor transforms from a basic video tool into something that feels like CapCut’s AI-powered experience. CE.SDK’s plugin system lets us add all AI capabilities through a single unified AiApps plugin.

The unified AiApps approach

Instead of registering each AI plugin separately, @imgly/plugin-ai-apps-web provides a single entry point:

import AiApps from '@imgly/plugin-ai-apps-web';
import FalAiVideo from '@imgly/plugin-ai-video-generation-web/fal-ai';
import FalAiImage from '@imgly/plugin-ai-image-generation-web/fal-ai';
import OpenAiImage from '@imgly/plugin-ai-image-generation-web/open-ai';
import Elevenlabs from '@imgly/plugin-ai-audio-generation-web/elevenlabs';
import Anthropic from '@imgly/plugin-ai-text-generation-web/anthropic';

await cesdk.addPlugin(
  AiApps({
    dryRun: true, // Simulate for development — no API calls
    providers: {
      text2text: Anthropic.AnthropicProvider({ proxyUrl: PROXY_URL }),
      text2image: [FalAiImage.RecraftV3({ proxyUrl }) /* ... */],
      image2image: [FalAiImage.GeminiFlashEdit({ proxyUrl }) /* ... */],
      text2video: [FalAiVideo.MinimaxVideo01Live({ proxyUrl }) /* ... */],
      image2video: [
        FalAiVideo.MinimaxVideo01LiveImageToVideo({ proxyUrl }) /* ... */,
      ],
      text2speech: Elevenlabs.ElevenMultilingualV2({ proxyUrl }),
      text2sound: Elevenlabs.ElevenSoundEffects({ proxyUrl }),
    },
  })
);

Provider categories explained

Category	What it does	Models we configured
`text2text`	AI copywriting — improve text, translate, change tone	Anthropic Claude
`text2image`	Generate images from text prompts	RecraftV3, IdeogramV3, GPT Image
`image2image`	Transform existing images with AI	Gemini Flash Edit, GPT Image
`text2video`	Generate video clips from text descriptions	Minimax Video, Kling Video, Pixverse
`image2video`	Animate a static image into video	Minimax Video, Kling Video
`text2speech`	Convert text to spoken audio with voice selection	ElevenLabs Multilingual V2
`text2sound`	Generate sound effects from text	ElevenLabs Sound Effects

When multiple providers are configured in an array (like text2image), the UI automatically shows a provider/model selection dropdown so users can choose which AI model to use.

The proxy server requirement

Every provider takes a proxyUrl parameter. This is critical for production:

Browser  →  Your Proxy Server  →  AI Provider (fal.ai, ElevenLabs, etc.)
                ↑
          Injects API keys
          server-side

Your API keys should never be in client-side code. The proxy server receives requests from CE.SDK, attaches your API key, and forwards to the AI provider. During development, dryRun: true simulates generation without any API calls.

Background removal

Background removal is a separate plugin because it runs entirely client-side — no proxy needed:

import BackgroundRemovalPlugin from '@imgly/plugin-background-removal-web';

await cesdk.addPlugin(
  BackgroundRemovalPlugin({
    ui: { locations: ['canvasMenu'] },
  })
);

This uses ONNX Runtime (WebAssembly + WebGPU) to run an AI segmentation model directly in the browser. The first run downloads ~40MB of model weights, which are then cached. Select any image on the canvas and click “Remove Background” in the context menu.

Step 6: UI Customization with the Component Order API

CE.SDK’s UI is built from five customizable areas: Dock, Inspector Bar, Canvas Menu, Navigation Bar, and Canvas Bar. The Component Order API lets us insert, remove, and reorder components in each area.

Adding the AI button to the dock

We want the AI Apps button to be the first thing users see in the dock:

cesdk.ui.insertOrderComponent(
  { in: 'ly.img.dock', position: 'start' },
  'ly.img.ai.apps.dock'
);

insertOrderComponent takes a location specifier and the component(s) to insert. position: 'start' puts it at the top of the dock. Alternative positions include 'end', a numeric index, or relative placement with before/after matchers.

When a user selects a text or image block and right-clicks, we want AI options available:

cesdk.ui.insertOrderComponent({ in: 'ly.img.canvas.menu', position: 'start' }, [
  'ly.img.ai.text.canvasMenu',
  'ly.img.ai.image.canvasMenu',
]);

Passing an array inserts multiple components at once.

We add an Export button with a real click handler that triggers MP4 export:

cesdk.ui.insertOrderComponent(
  { in: 'ly.img.navigation.bar', position: 'end' },
  {
    id: 'ly.img.action.navigationBar',
    key: 'export',
    label: 'Export',
    icon: '@imgly/Download',
    onClick: async () => {
      const engine = cesdk.engine;
      const page = engine.block.findByType('page')[0];
      if (page) {
        const blob = await engine.block.exportVideo(page, {
          mimeType: 'video/mp4',
        });
        const url = URL.createObjectURL(blob);
        const a = document.createElement('a');
        a.href = url;
        a.download = 'video.mp4';
        a.click();
        URL.revokeObjectURL(url);
      }
    },
  }
);

The exportVideo method renders every frame of the timeline — compositing video tracks, overlays, text, captions, and audio — into an MP4 blob entirely in the browser.

Step 7: Wiring Generated Audio into the Asset Library

AI-generated audio (speech and sound effects) is stored in provider-specific history sources. To make this audio browsable alongside regular audio assets, we inject the history source into the audio asset library entry:

const audioEntry = cesdk.ui.getAssetLibraryEntry('ly.img.audio');
if (audioEntry != null) {
  const existingSourceIds = Array.isArray(audioEntry.sourceIds)
    ? audioEntry.sourceIds
    : audioEntry.sourceIds({});
  cesdk.ui.updateAssetLibraryEntry('ly.img.audio', {
    sourceIds: [...existingSourceIds, 'ly.img.ai.audio-generation.history'],
  });
}

Now when users open the Audio panel in the dock, they’ll see their AI-generated audio alongside the sample library.

Step 8: Custom Labels with i18n

CE.SDK’s i18n system lets us customize any UI string. We used it to make the AI prompt fields more inviting:

cesdk.i18n.setTranslations({
  en: {
    'ly.img.plugin-ai-video-generation-web.fal-ai/minimax/video-01-live.property.prompt':
      'Describe your video...',
    'ly.img.plugin-ai-image-generation-web.fal-ai/recraft-v3.property.prompt':
      'Describe your image...',
  },
});

Translation keys follow the pattern {plugin-id}.{provider-id}.property.{field}. You can also add multi-language support by including keys for de, fr, es, etc.

The Complete File Structure

capcut-like-editor/
├── src/
│   ├── App.jsx              # Root — imports VideoEditor + theme CSS
│   ├── VideoEditor.jsx      # The entire editor (single component, ~180 lines)
│   ├── capcut-theme.css     # CapCut dark theme overrides (~100 lines)
│   ├── index.css            # Global reset (margin/padding/overflow)
│   └── main.jsx             # React entry point
├── index.html
├── package.json
└── vite.config.js

That’s it. The entire CapCut-like editor is three files: one React component, one CSS file, and a thin App wrapper.

What We Get Out of the Box

Because CE.SDK’s video UI is pre-built, we didn’t write any code for these features — they come from createVideoScene() and the default UI:

Multi-track timeline with drag-to-reorder, drag-to-resize
Trim and split — drag clip edges or use the split tool
Join and arrange — drag clips between tracks
Transform — crop, flip, rotate via the inspector
Playback controls — play, pause, seek, scrub
Text overlays — add styled text with the text tool
Stickers and graphics — from the asset library
Filters and effects — LUT-based color grading
Undo/redo — full history stack
Zoom and pan — standard canvas navigation

The AI plugins add their own UI components (dock buttons, generation panels, context menu items) through the plugin system. We just positioned them where we wanted.

Going to Production

To take this from a prototype to production, you need three things:

1. License key

Get a free trial key at https://img.ly/forms/contact-sales and set it in the config (yes, if you want to ship to production you’ll have to talk to our lovely colleagues in sales):

const config = {
  license: 'YOUR_CESDK_LICENSE_KEY',
};

2. Proxy server

Set up a server that forwards AI requests with your API keys:

const PROXY_URL = 'https://your-server.com/api/ai-proxy';

See the CE.SDK Proxy Server guide for Express.js and other server examples.

3. Disable dry run

Remove dryRun: true from the AiApps configuration to enable real AI generation.

Key Takeaways

CE.SDK does the heavy lifting. The timeline, playback, rendering, and export are all handled by the engine. We wrote zero video processing code.
The plugin system is powerful. Six AI capabilities were added with a single addPlugin(AiApps({ ... })) call. Background removal was one more call.
CSS custom properties make theming painless. We matched CapCut’s aesthetic by overriding ~25 CSS variables. No forking, no patching.
The Component Order API is the customization backbone. insertOrderComponent with position-based placement is the cleanest pattern for adding UI elements.
Wrap init in try/catch. CE.SDK’s <CreativeEditor> swallows errors silently. This is the single most important debugging tip.
Pin your package versions. All @imgly/* plugins must match the @cesdk/cesdk-js version exactly. A version mismatch (like 1.68 vs 1.69) will cause peer dependency conflicts.

Resources

Introducing IMG.LY Agent Skills

Neslihan — Mon, 16 Feb 2026 18:54:06 GMT

TL;DR
We are moving beyond static documentation. IMG.LY Agent Skills is a specialized intelligence layer that transforms AI coding assistants (Claude Code, Cursor, Windsurf) into CE.SDK implementation experts. Instead of mapping APIs manually, you now inject our full SDK expertise directly into your IDE and coding agent.

→ Explore Agent Skill Documentation
→ Access Agent Skills GitHub Repository

The “Quick Start” Paradox

In our recent analysis of client integrations, a consistent pattern emerged: even with a high-performance SDK, the “Time to First Edit” is often delayed by research. Developers spend a significant portion of their initial integration time context-switching between their IDE and documentation, manually mapping complex API hierarchies to their specific framework.

We figured: You don’t just need better documentation; you need an expert partner who is already inside your codebase.

The Solution: IMG.LY Agent Skills for Web

We are moving beyond static documentation. Today, we are launching IMG.LY Agent Skills, a specialized intelligence layer that transforms AI coding assistants like Claude Code, Cursor, and Gemini into CE.SDK implementation experts.

Instead of searching for answers, you now give your agent our complete SDK knowledge with a single command.

How It Works: Your Autonomous Implementation Partner

By injecting versioned, live documentation and framework-specific starter kits directly into your agent’s context window, we’ve created three adaptive paths to launch:

1. The Explain Path

Architecture is complex. The Explain Skill acts as a digital implementation partner that adapts to you. It can provide a technical deep-dive on the export pipeline for a Senior Dev or a high-level summary for a Product Manager—in any language and at any level of detail.

2. The Build Path

Command your agent to “Add a photo editor to my React app.” The agent autonomously detects your environment, pulls the correct starter kit, and handles the boilerplate. It doesn’t just show you code; it builds the foundation for you.

3. The Docs Path

Stop the tab-switching. Your agent retrieves versioned API references offline, keeping your focus entirely within the terminal or IDE.

The Shift to Autonomous Engineering

This release marks a strategic shift for IMG.LY. We believe the next era of software development isn’t about “better documentation,” but about Portable Expertise.

By providing that AI with a “skill” rather than a manual, we are empowering teams to move from a multi-month build cycle to a 30-second scaffold.

Supported Frameworks

The initial Web release supports 10 frameworks out of the box:
React · Vue.js · Svelte · Angular · Next.js · Nuxt.js · SvelteKit · Electron · Node.js · Vanilla JS

Get Started

Run the command:

npx skills add imgly/agent-skills

For example, Claude Code users, can run:

claude plugin marketplace add imgly/agent-skills

claude plugin install cesdk@imgly

→ Explore the full Agent Skill Documentation

The “editor of your dreams” is now a conversation away.

Join 3,000+ creative professionals who get early access to new features and updates—subscribe.

From Prompt to Editor: Running CE.SDK Inside ChatGPT with the Apps SDK

Jan — Fri, 19 Dec 2025 15:43:16 GMT

With the new ChatGPT Apps SDK and Model Context Protocol (MCP), chat interfaces are starting to look less like Q&A tools and more like places where work actually happens. To explore what that means for creative workflows, we built a small technical demo: CE.SDK running directly inside ChatGPT.

From a user’s perspective, the flow is almost trivial. You ask ChatGPT for something like an ecommerce template. ChatGPT searches our template catalog, selects a matching design, and opens it instantly in a fully interactive CE.SDK editor — right inside the chat interface. What looks like a preview is, in fact, a real editor loaded with a real template scene.

This isn’t meant as a product announcement. It’s a technical proof of concept showing how creative SDKs can plug directly into AI-native interfaces.

CE.SDK as a ChatGPT App

https://www.youtube.com/embed/AoDqUNVLLJ4?feature=oembed

The integration is built around a custom MCP server that exposes CE.SDK to ChatGPT as a tool. The server speaks OpenAI’s JSON-RPC–style MCP and implements the standard lifecycle methods (initialize, tools.list, tools.call, resources.read). It knows about our premium template catalog and emits structured payloads that the frontend understands.

On the client side, a Next.js app listens to tool output events streamed from ChatGPT, renders CE.SDK widgets, and hydrates them with the payloads returned by the tool — such as a scene URL, placeholder values, or export permissions. Templates are loaded via CE.SDK’s Template API, either from a URL or from a serialized scene string.

Under the hood, the stack is fairly conventional:

Next.js 15 (App Router)
CE.SDK Web / CreativeEngine
A custom MCP handler to normalize JSON-RPC
Vercel for hosting

What’s new is not the technology itself, but the context in which it runs.

Working with MCP in Practice

The hardest part of the demo wasn’t CE.SDK — it was MCP.

OpenAI’s MCP implementation is extremely strict. Even the smallest schema mismatch can trigger the infamous “TaskGroup 424” error, usually without any hint as to what went wrong. In many cases, the HTTP response is technically successful, but the JSON structure doesn’t match the expected schema closely enough.

The key lesson here is to treat MCP responses as hard contracts:

Validate every response against a schema (for example with zod).
Mirror OpenAI’s field names exactly, even for empty or optional capabilities.
Assume that a 424 almost always means “your JSON shape is wrong”.

Another important insight was how critical visual context is in chat-based tools. If your MCP responses don’t include thumbnails or preview images, ChatGPT will often fall back to rendering links. For creative tools, that immediately breaks the experience. In a chat UI, visuals aren’t an enhancement — they are the interface.

State handling also requires a shift in mindset. ChatGPT can replay tool calls, and each prompt effectively creates a new widget instance. You can’t rely on mutating an existing editor. The frontend needs to be idempotent: load scenes from serialized state first, then apply changes. Every tool call should be treated as a fresh render.

Why This Pattern Matters

This demo points to a broader change in how creative software may be accessed. Chat becomes a coordination layer, not just a conversational one. Instead of explaining how something could be designed, the AI opens the actual editor and lets the user continue from there.

For CE.SDK, this fits naturally. Editors become embeddable capabilities rather than standalone applications, and AI systems become the entry point into creative workflows. Prompting turns into doing.

Beyond OpenAI: The MCP UI Standard

Although this demo uses OpenAI’s MCP, the architecture maps cleanly to the new MCP UI standard recently introduced by Anthropic. That standard aims to make tool definitions and UI rendering more consistent across models and platforms.

Because this integration already separates tool logic from UI rendering and relies on structured, explicit payloads, transferring it to Anthropic’s MCP UI model is conceptually straightforward. CE.SDK can act as a reusable creative surface across ChatGPT, Claude, and future AI app ecosystems.

You can read more about Anthropic’s MCP UI direction here:

https://blog.modelcontextprotocol.io/posts/2025-11-21-mcp-apps/

This demo is intentionally small and technical, but it highlights a meaningful shift: AI systems that don’t just describe creative outcomes, but open the tools to actually create them.

What is Visual Prompting?

Jan — Tue, 29 Jul 2025 13:06:11 GMT

A New Paradigm for Creative AI, Built by IMG.LY

To say it’s trite to refer to the impact of AI in this or that domain as disruptive or groundbreaking would be an understatement. Yet, few areas have been as profoundly affected as the creative process. With just a text prompt, anyone can produce stunning images, remix visual styles, and explore design possibilities at a scale and speed never seen before. AI has inserted itself so quickly into this process that its gone from curious novelty to an essential part of the creator toolchain.

The more serious adoption we see, however, the more key limitations of today’s AI tooling come into focus: the prompt itself.

Text alone, for all its expressive power, struggles to capture the essence of visual intent. Most creative work doesn’t begin with a sentence it begins with a sketch, a layout, a mood board, or an arrangement of elements. Visual ideas are shared by pointing, placing, showing.

At IMG.LY, we have begun to think about better ways to direct AI for visual generation, the term we use is Visual Prompting.

Visual Prompting: the practice of composing a visual scene or layout as input for a generative model.

Instead of describing what you want with paragraphs of text, you show it directly using a canvas of images, text, spatial cues, and annotations. This visual composition then becomes the prompt for the AI to generate new content in return. It’s a more natural, intuitive, and powerful way to collaborate with AI, especially when integrated directly into the creative process.

Problem: the Chat Disconnect

The current generation of AI tools has largely been shaped by language-first interfaces. Whether it’s ChatGPT for writing or Midjourney for image generation, the assumption is the same: the user will type a descriptive prompt, and the AI will generate a result based on it.

But when it comes to design, this workflow quickly runs into friction. Visual ideas are inherently spatial and non-linear. Trying to express layout, balance, mood, or specific spatial relationships through text can feel like trying to describe a painting over the phone. It’s possible but unnecessarily cumbersome.

A designer might want to:

Indicate that a certain area in the image should be blue.
Replace a background with a texture sample.
Position a character precisely in a composition.
Annotate which parts of a scene to preserve or modify.

All of these are difficult to express fluently in text. But they’re effortless in a visual interface. The truth is: an image is worth more than a thousand words when prompting an image.

What Is Visual Prompting?

Visual Prompting is a multimodal approach to generative AI, where the input to the model is not just text, but a full visual composition: images, text, annotations, and layout.

Rather than prompting AI in isolation, the user builds their intent on a canvas. This might include:

Reference images that communicate mood or style.
Text blocks indicating desired copy or instructions.
Annotations pointing to specific areas with notes like “make this glow” or “replace this object.”
Spatial composition: where elements are arranged meaningfully to convey intent.

The visual prompt is then interpreted by a multimodal model such as OpenAI’s gpt-image-1 to generate new visual content that reflects not only the textual description, but also the visual context.

How Visual Prompting Works in CE.SDK

About time for an example. As part of our recent AI released we demoed how to use OpenAIs gpt-image-1 model to build visual prompting into CreativeEditor SDK (CE.SDK).

Here’s what the process looks like inside CE.SDK:

Compose Visually: The user creates a layout with reference content, uploaded images, icons, color schemes, design elements, placeholder text, and annotations. This composition represents the “prompt” in visual form.
Add AI Layers: With a single click, the user can trigger image generation using CE.SDK’s built-in AI plugin. The plugin sends the visual context (alongside any optional text input) to a multimodal model capable of interpreting both.
Refine and Iterate: Users can adjust the layout, reposition elements, change annotations, or layer in new references, then prompt again. Because the canvas is interactive and editable, the feedback loop is tight.
Build Up Complexity: Over time, users can layer generated images with manually designed components or other generated outputs, creating rich compositions that blend AI creativity with human direction.

This workflow turns the traditional prompt/response cycle into a conversation between the designer and the model, with the canvas acting as the shared language.

Who Is Visual Prompting For?

The use cases for Visual Prompting extend across industries:

Creative teams can go from reference to generation in seconds, iterating visually instead of wrangling prompts.
Marketing teams can generate regionalized or personalized creative variants from a shared layout.
Product designers can prototype in context, turning layouts into realistic screens without leaving the editor.
Storytellers and content creators can use annotated sketches to generate detailed illustrations or scene variations.
E-commerce platforms can give sellers the power to visually customize their brand materials with AI assistance.

In every case, Visual Prompting replaces friction with flow and text-based prompting with something more expressive, more reliable, and more fun.

Built for This: Multimodal Models and CE.SDK’s Plugin System

Visual Prompting is only possible because of two parallel advancements:

Multimodal AI models, such as OpenAI’s gpt-image-1, that can interpret both images and text, understand spatial relationships, and respond to annotated cues.
A flexible, composable editor SDK like CE.SDK, which enables the construction of visual prompts on a live canvas, and makes it easy to integrate AI models directly into the design flow.

Our SDK was built from the ground up to support AI-first creative workflows. Its plugin architecture allows you to add any model or API, image generation, video generation, captioning, text rewriting and use it natively inside the editor without the need to switch tools or copy/paste.

Generative AI’s full potential is only unlocked when it is embedded directly into the tools creatives use not siloed in chatbots or separate interfaces. Visual Prompting allows that embedding to go even deeper, aligning the mode of input (visual) with the desired output (visual).

Explore It Yourself

🎨 Try out Visual Prompting in our AI Editor demo
📘 Learn How to Integrate AI into CE.SDK
💬 Contact Us to Bring Visual Prompting to Your Product

Build vs. Buy: Is Fabric.js Right for You

Jan — Thu, 15 May 2025 09:31:43 GMT

When you’re building a design editor into your product whether for social content, marketing assets, video, or design workflows choosing the right foundation matters. Many teams start with Fabric.js, an open-source canvas library, attracted by its flexibility and permissive license. But as the project scope grows, so do the limitations.

This article explores the advantages, but also the pitfalls of building a creative tool with Fabric.js and why many product teams ultimately decide to switch or start with a commercial SDK like IMG.LY’s CreativeEditor SDK (CE.SDK).

Consult our comparison page of Fabric.js and IMG.LY for a feature by feature breakdown of the differences between our SDKs and Fabric.js.

Why Teams Switch from Fabric.js

We’ve spoken with dozens of companies, from startups to enterprise clients, who evaluated Fabric.js and in many cases opted for IMG.LY instead. Here’s what we consistently hear:

1. Time-to-Market vs. Reinventing the Wheel

Open-source is attractive at first. But once the excitement of building wears off, teams often face a sobering reality: stitching together a decent editing experience from Fabric.js is a months-long effort.

“We used Fabric.js before, but if IMG.LY’s SDK is cost-effective, we’d rather not reinvent the wheel.”

— B2B SaaS Prospect

They want a modern editor UI, fast iteration cycles, and a stable base not to spend their next quarter building the basics like zoom, text editing, object grouping, layer management, or responsive templates.

2. Lack of Advanced Features

Fabric.js excels at low-level canvas manipulation, offering a robust API for working directly with objects on the HTML5 canvas - shapes, images, text, and transformations. It’s a great starting point for developers who want fine-grained control over rendering and interactivity. However, it falls short when it comes to modern editing paradigms. Fabric.js provides no native concept of reusable templates, lacks support for layout and design constraints like alignment guides or snapping behavior, and does not include higher-level abstractions for content-aware or AI-powered editing. In short, Fabric.js gives you a raw toolkit, not a polished, plug-and-play editing solution. If you’re building a sophisticated design application, you’ll need to layer these capabilities yourself or integrate additional frameworks to bridge the gap.

“We needed a template system and advanced photo editing. Open-source libraries couldn’t do it out of the box.”

— Marketplace Platform Prospect

Teams chasing parity with Canva or Adobe tools quickly hit walls. What starts as a proof-of-concept becomes a rebuild of the tables stakes of design editing.

Real-World Use Cases & Vertical-Specific Challenges

We’ve seen teams from various industries reach the same conclusion: Fabric.js doesn’t scale to meet their creative, technical, or business needs. A few common themes:

E-commerce & Web-to-Print: Retailers building product customization flows need template constraints, consistent output quality, and export formats that go beyond canvas. They can’t afford rendering inconsistencies or unpredictable export fidelity.
MarTech & Social Media Tools: Marketers need batch generation, AI-assisted creative workflow, brand constraints and consistent creative output across devices. Fabric.js lacks built-in support for these workflows.
Mobile-first Apps: Teams building hybrid apps or web apps that need to be functional on mobile struggle with Fabric.js’ lacklustre mobile support. The inability to deliver a consistent UX across iOS, Android, and Web leads to dropped features or split tech stacks.
Video and Multimedia Platforms: Fabric.js has no native video support, track-based editing, or multi-frame logic. Prospects in this space frequently abandon their Fabric.js POCs once real constraints emerge.

A major pain point across all verticals is rendering consistency. CE.SDK renders identically across browser, server, and mobile thanks to a shared rendering core (CreativeEngine). This is critical for:

Workflows that begin on web and finish on mobile
Server-side generation (e.g., previews, PDFs, batch exports)
Feature parity and UX reliability across platforms

Check out our demo page to explore these use cases.

Technical Debt: The Hidden Cost of Fabric.js

On paper, Fabric.js seems like a fast way to get started. But its real cost emerges over time in performance bottlenecks, missing architecture, and maintenance drag.

Aging Codebase and Patchy Maintenance

As of 2025, Fabric.js has over 400 open issues on GitHub, some dating back years. Many feature requests such as better performance for large object sets, async rendering, or text-on-path improvements are either unresolved or uncertainly prioritized.

The core maintainers do their best, but progress is slow. You’re relying on volunteer effort for critical infrastructure.

Slow Issue Resolution

Many of the GitHub issues (e.g., #5885, #6582) highlight long-standing rendering bugs and performance problems. These are hard to fix, and updates can break your own hacks or workarounds.

Unlike a commercial SDK, Fabric.js doesn’t guarantee backward compatibility. No SLAs. Little roadmap visibility.

Customer Feedback: Why They Walked Away

Here are some soundbites from our customers and prospects on why they decided against building with Fabric.js:

“The quality and UX didn’t meet our standards.”
Product Customizer

“Developer experience was poor. We spent more time debugging than building features.”
B2B SaaS Ad Design

“We needed enterprise support, scalability, and documentation. Open source didn’t cut it.”
Web to print Customer

“Cross-platform support and minimum SDK version constraints were a major blocker.”
Claim Management Application

Voice of the Dev

Beyond enterprise evaluations, developers working with Fabric.js often share their struggles publicly on GitHub, Stack Overflow, and developer forums. Their commentary offers real-world insight into day-to-day frustrations:

“FabricJS doesn’t seem to work well on Mobile.”

Github issue #6980

“Fabric.js object controls don’t work until after a common selection is made. Had to hack around it with extra event listeners.”

StackOverflow

“Trying to integrate Fabric.js with Next.js + Rollup is a mess. Unexpected tokens, config rewrites — it doesn’t play well with modern bundlers.”

GitHub Issue #8444

“We’re blocked by the use of ‘unsafe-eval’ due to our content security policy. Fabric.js needs a rewrite to be CSP-compliant.”

GitHub Issue #9666

Concrete takeaways from developer feedback:

Basic UI control quirks: Common issues with selection handling and control behavior require manual workarounds.
Compatibility friction: Fabric.js does not play well with modern build tools like Next.js and Rollup without additional configuration or patching.
Security blockers: The reliance on unsafe-eval creates CSP conflicts, making it unsuitable for projects with strict security requirements.
Uncertain roadmap: Critical features like async rendering may sit open for years, and niche features important to your use case may never land if they don’t align with community priorities.

While every open source project faces these difficulties, they do represent systemic issues with performance, extensibility, and production-readiness. For teams building long-lived, customer-facing tools, this developer feedback is often a leading indicator of future roadblocks.

Back-of-the-Envelope Cost of Building with Fabric.js

Many teams underestimate the cost of building a design editor from scratch. Here’s a rough breakdown of the time investment we’ve heard from teams who tried:

Task	Estimated Engineering Time
Core canvas-based editor UI (zoom, drag, resize, selection, tool switching)	6–8 weeks
SVG support, snapping, grouping, and layer management	4–6 weeks
Template constraints	3–5 weeks
Cross-platform adaptation (ironing out issues on mobile browsers, defining fallbacks)	4–6 weeks
Bug triage, ongoing maintenance, and refactors (first 12–18 months)	6–9 weeks

Total: ~6–8 months of senior dev time just to get to feature parity with what CE.SDK offers out of the box.

This doesn’t include the cost of QA, product management, or future extensibility. Nor the opportunity cost of what your team could be building instead.

IMG.LY’s CE.SDK: A Ready-Made Solution Built for Growth

By contrast, IMG.LY offers a production-grade SDK built for cross-platform creative tools, backed by a dedicated engineering team and used by major apps across industries.

Category	IMG.LY (CE.SDK)	Fabric.js	Notes
Out-of-the-box UI	✅ Prebuilt modern UI	❌ Build from scratch	CE.SDK ships with full UI/UX patterns
Plugin System	✅ Native plugin architecture	⚠️ Manual code extension	IMG.LY supports formal plugin APIs
Free Drawing Tools	⚠️ Requires integration	✅ Built-in pencil + shapes	Fabric.js excels at shape-level control
Advanced Editing	✅ Vector + raster support, layering, filters	⚠️ Basic vector only	CE.SDK includes pro-grade editing capabilities
Templating System	✅ Dynamic placeholders + constraints	❌ Manual implementation	IMG.LY supports automation workflows
Video Editing	✅ Multi-track timeline editor	❌ Not supported	No video support in Fabric.js
Cross-Platform	✅ Web, iOS, Android, Node, Flutter	⚠️ Web only	IMG.LY has native SDKs and server tools
Asset Integrations	✅ Unsplash, Getty, Brandfolder	❌ Manual setup	CE.SDK integrates assets out-of-the-box
Enterprise Support	✅ SLAs, onboarding, deployment help	❌ Community support only	IMG.LY offers dedicated engineering help
Design File Import Support	✅ PSD, AI, INDD	⚠️ None	CE.SDK supports importing common design file formats
AI Editing	✅ Background removal, integrate any AI model for image/video/text/audio gen	❌ Not supported	AI-native editing pipeline available
Rendering Consistency	✅ Unified engine across platforms	⚠️ Browser dependent	CE.SDK ensures pixel-parity on all targets

When Does Fabric.js Still Make Sense?

This article might read like a put-down, but it’s not supposed to be that. Until now we were simply evaluating Fabric.js with respect to building fully-featured design editing tools. There are, however, a number of well-scoped use cases where Fabric.js excels. These include:

Building an internal tool, proof of concept, or educational app
Creating a custom, single-platform sketching or annotation layer
Developing a low-fidelity prototype with full code-level control
Building canvas collaboration tools such as drag and drop editors or interactive canvases, such as Miro.

An example of where Fabric.js worked well is a tool such as Mockola, a drag-and-drop diagram editor, where shape manipulation, lightweight rendering, and canvas-level control are more important than cross-platform design fidelity. Mockola’s is a good use case, because its emphasis on real-time collaboration and planning rather than high-fidelity design. This means that features like free drawing, drag-and-drop, and JSON-based serialization are a great fit, all of which Fabric.js supports out of the box.

For teams willing to invest in building and maintaining their own editor infrastructure and whose use case does not require cross-platform parity, advanced media editing, or enterprise scaling Fabric.js continues to be a compelling foundation.

However, for products that need to ship quickly, scale reliably, and support modern creative workflows across platforms, IMG.LY’s CE.SDK offers the infrastructure and polish that customers now expect.

The Bottom Line

As MIT CIO Symposium speaker Mark Holst-Knudsen put it:

“You shouldn’t build anything that’s available off the shelf because it’s not a source of competitive advantage if everybody else can avail themselves of it.”

For most companies, building an editor from scratch is not a competitive edge, it’s a distraction from shipping and scaling. And Fabric.js, while capable in the hands of specialists, isn’t a shortcut to the finish line.

IMG.LY’s CE.SDK gives you a head start with advanced editing features, AI automation, template support, plugin extensibility, rendering consistency, and battle-tested scalability across platforms. We put together a data report based on 600+ customers, 28 structured customer interviews, and a self-reported customer survey that speaks for itself when it comes to real-life outcomes.

So before you dive into Fabric.js and start reinventing core functionality, ask yourself: What competitive advantage are you going to build on top of a design editor and which functionality are you better served buying from a trusted vendor?

CE.SDK v1.49 Release Notes

Neslihan — Thu, 17 Apr 2025 12:55:42 GMT

Get ready to add more motion and emotion to your designs! From fully integrated GIF Support to the cinematic Ken Burns animation and polished text background on iOS, this update packs features that elevate storytelling and enhance visual impact. Let’s dive in!

Use GIFs in Your Designs on Web and Mobile

GIFs are now fully integrated into the CE.SDK UI. Add and manage animated images like any other media asset. Drop a GIF into your timeline to play it once, or create a loop by extending its duration. You’re in control.

This update delivers a consistent experience across Web and Mobile, empowering dynamic content creation for social media, marketing, and beyond.

Getting Started:
GIF support itself is built-in—no setup required.
However, to add looping, enable the looping option in your assets. Find platform-specific guidance in our docs:

Add A Cinematic Ken Burns Animation

Give your images a cinematic touch with the new crop-zoom animation in CE.SDK. Inspired by the Ken Burns effect, this feature lets you smoothly pan and zoom across images, creating dynamic storytelling without complex editing.

Apply the Ken Burns animation directly from the CE.SDK animation library on the web. Learn how to use animations and their properties with our documentation.

Add Text Backgrounds on iOS

Make your text pop! Apply solid color backgrounds to text on iOS. Additionally, adjust the padding and rounded corners of your text background.

This feature ensures readability and adds a polished, professional touch to your designs. By elevating text design, your app helps users create visually compelling content that stands out. Available for Web and iOS, soon on Android!

Get started with our text background documentation for iOS.

Improvements

Speed Up Your HDR Video Editing

We’ve dramatically accelerated HDR-to-SDR video conversion in CE.SDK on iOS. While HDR videos previously required lengthy processing, especially for 4K content, they now convert to SDR with minimal delay, letting users start editing their footage faster than ever before.

Stay tuned: Smooth HDR-to-SDR conversion will also be available for Web with CE.SDK v1.50.

Upcoming

Boost Engagement and Accessibility with Captions

Captions are essential in today’s social media landscape, where videos often autoplay on mute and clear messaging is key. Our upcoming Captions for Web feature will allow you to:

Manually add subtitles to videos or upload an SRT file for seamless integration.
Adjust timing and formatting in an intuitive interface to match your brand’s style.
Embed captions directly into videos, ensuring they remain visible across all platforms.

Thanks for reading! View all bug fixes and changes in the Changelog.

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.48 Release Notes

Neslihan — Tue, 08 Apr 2025 12:13:35 GMT

This release makes it easier than ever to tailor the Android editing experience to your brand. Customize the navigation bar to guide your users seamlessly and create an interface that feels uniquely yours.

Let’s jump in:

You now have more control over customizing the navigation bar on Android, ensuring that it reflects your brand and optimizes user flows.

Key Features:

Add Custom Buttons: You can add custom buttons to the navigation bar, to give users quick access to frequently used actions.
Rearrange Buttons: Arrange buttons in your navigation bar.
Integrate Custom Components: Customize the navigation bar by integrating unique elements that fit your app’s design and functionality.

These updates allow you to fully tailor the mobile editing experience to your specific needs.

💡 Did you know? In our previous update, we introduced customizability for the dock and inspector bar on Android—laying the foundation for more UI personalization.

We’re continuing that momentum by giving you full control over the navigation bar, ensuring that every part of your mobile editing experience can align with your product’s workflow.

Start customizing your navigation bar with our documentation.

Upcoming

Enjoy Vibrant HDR Videos
CE.SDK will support loading HDR videos on iOS, converting them to SDR on the fly for accurate and vibrant color representation—without the need for time-consuming transcoding.

Previously, HDR videos needed to be transcoded with delays—especially with large 4K videos. Soon, HDR content is processed instantly. Videos appear with correct color fidelity right away, making the editing experience more fluid and visually accurate for users.

This enhancement will remove friction in the editing workflow and lets your users start working with their content immediately. By offering a smoother, higher-quality experience, your app can boost user engagement and retention.

View all bug fixes and changes in the Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.46 Release Notes

Neslihan — Tue, 11 Mar 2025 15:58:30 GMT

Welcome to a new release! CE.SDK v1.46 introduces a more flexible and intuitive user interface for iOS, allowing you to optimize your workflow with a customizable dock and inspector. Read on for upcoming features.

Shape Your iOS Interface Around Your Workflow

We unlock full dock and inspector bar customization, giving you the flexibility to optimize your workflow. With an open API, you can now tailor these UI elements to better suit your editing needs—whether by reordering actions, adding custom functionality, or simplifying the interface.

Take control of your interface by modifying both the dock and inspector bar:

Rearrange & Customize Actions – Prioritize your most-used tools by reordering or removing actions to create a cleaner, more efficient workspace.
Add Custom Buttons & Views – Extend functionality by integrating custom buttons or views that trigger specific actions, such as opening an asset library or displaying additional information.

These enhancements provide a seamless, intuitive editing experience that adapts to your unique workflow needs.

Start customizing your dock and inspector bar today with our documentation.

Upcoming

Boost Engagement and Accessibility with Video Captions
Captions are essential in today’s social media landscape, where videos often autoplay on mute and clear messaging is key. Our upcoming Video Captions for Web will allow you to:

Manually add subtitles to videos or upload an SRT file.
Adjust timing and formatting in an intuitive interface to match your brand’s style.
Embed captions directly into videos, ensuring they remain visible across all platforms.

Stay tuned—Captions are coming soon!

View all bug fixes and changes in the Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

How to Build a Short Video Generator Using CE.SDK

Eray — Tue, 04 Mar 2025 12:57:47 GMT

In the following, I’m presenting a simple cookbook for building an AI-based video generator app, as described in my previous blog post. We’re using a combination of different APIs to generate text, audio, and images and will compose & render the final video using the headless CreativeEditor SDK. We also call it the Creative Engine.

This cookbook showcases the powerful capabilities of our client-side Creative Engine. The engine enables real-time video generation directly in the browser, eliminating the need for server-side processing. What sets this approach apart is its ability to produce editable source files that can then be opened with CreativeEditor SDK.

This approach is giving users complete control over every aspect of your video–from text and images to animations and overall composition. This means your users can refine and perfect your content even after the initial generation.

Get the complete code on GitHub.

Scope

This tutorial focuses on building an app with a simple UX:

Input your keywords/topics
Choose between landscape or portrait format
Generate and preview your video
Edit the video in the CE.SDK video editor

The app flow we will create:

The post-editing we will get with CE.SDK:

Technical Overview

The app follows three major steps to generate the video.

A script is generated based on User input, the output is a structured XML file.
The XML script is parsed to extract text and image information. The extracted data will then be used to generate audio & image files through third-party APIs
All assets are loaded into the creative engine. This is where the composition, including animation and effects, takes place. The Creative Engine then exports a video and scene file, which can be edited with the Creative Editor.

Setup

We’ll use a boilerplate with Next.js, React, Typescript & Tailwind. Make sure you retrieve all necessary keys:

Anthropic (LLM)
ElevenLabs (text to speech)
fal.ai (text to image)
IMG.LY CE.SDK – Retrieve a free trial key

// Required environment variables
NEXT_PUBLIC_ANTHROPIC_API_KEY = your_claude_api_key;
NEXT_PUBLIC_FAL_API_KEY = your_fal_ai_key;
NEXT_PUBLIC_ELEVEN_LABS_KEY = your_eleven_labs_key;
NEXT_PUBLIC_IMG_LY_KEY = your_img_ly_key;

Implementation

1. Generate The Script

In this step, we’ll focus on generating the initial prompt and then passing it to the Anthropic API.

As with many things with LLM, there are many different strategies for structuring the initial prompt. From experience, the best result comes from providing examples of the desired output. We’ve decided to use an XML document; this can be easily parsed later on and is less error-prone compared to a JSON.

We now define the structure of how information should be saved in the XML.

<video>
  <group part="intro">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.2">
        Did you know these fascinating facts about pyramids?
      </text>
      <image>Ancient Egyptian pyramid at sunset</image>
    </element>
  </group>
  <group part="content">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.2">
        The Great Pyramid was the tallest structure for over 3,800 years!
      </text>
      <image>Great Pyramid comparison to modern buildings</image>
    </element>
  </group>
  <group part="outro">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.4">
        The pyramids continue to reveal their secrets to this day...
      </text>
      <image>A giant 3D question mark hovering over the pyramids</image>
    </element>
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.4">
        Stay curious - there's always more to discover!
      </text>
      <image>Pyramids under starry night sky</image>
    </element>
  </group>
</video>

In this tutorial, we’ll focus on the format trivia only as shown in this example. For later iterations, however, I’m planning to implement different content formats (e.g., trivia, quiz, recipe, etc.). Each of these formats will have its example XML. Therefore, I’m nesting the XML in a simple format object to scale this up easily later.

interface Format {
  name: string;
  example: string;
}

const formats: Record<string, Format> = {
  trivia: {
    name: 'Trivia',
    example: `<video><group>...</group></video>` // Add example from above
  }
};

Using this format object with the example, we can now generate the prompt.

What do we need for this prompt?

Description of the task
Description of the desired output, incl. an example for the specified format
Topic as provided by the user

The topic provided by the user is passed to the function as a string.

export const createVideoScriptPrompt = (
  topic: string,
  formatName: string = 'trivia'
) => {
  const format = formats[formatName];
  if (!format) throw new Error(`Format ${formatName} not found`);

  return `
Format: ${format.name}
Topic: ${topic}

Please write a detailed script for this short video, considering the specified format and topic.
Include an introduction, main content sections, and an outro. Each section should have an image.
Structure the script as an XML Document with clear sections, descriptions for the images.
The image description should be written as a prompt. This prompt will be used to generate an image.
Put the description between the image tags. The video shouldn't be longer than 30 seconds.

Example format:
${format.example}`;
};

2. Generate All Assets

In the second step, we’ll parse through the LLM response, which should be the XML. We’ll create a simple parsing function to extract all text information that should be sent to text-to-speech and text-to-image AIs.

Please note that all these steps can be easily streamlined by using AI-assisted coding. Just provide the example XML as input and your desired output.

API Calls
When finding text & image tags in the XML, we’ll call API functions for text-to-speech and text-to-image. For this example, I’m using ElevenLabs & fal APIs. You will find all API calls in the api.ts.

Since the LLM generated a script that includes image prompts, make sure to pass them to the API.

export async function generateImage(prompt: string): Promise<string | null> {
  try {
    console.log('Generating image for prompt:', prompt);
    const result = await fal.subscribe('fal-ai/flux/dev', {
      input: {
        prompt: prompt,
        size: 'portrait_16_9',
      },
    });
    const typedResult = result as { images: { url: string }[] };
    console.log('Image generation successful. URL:', typedResult.images[0].url);
    return typedResult.images[0].url;
  } catch (error) {
    console.error('Error generating image:', error);
    return null;
  }
}

Timestamps
Last but not least, we need to come up with timestamps. What’s the duration of each segment? This is critical information for composing the video. Luckily, this is quite easy: Each scene is as long as the generated audio for each segment. This duration for the audio segments can be calculated: Most TTS like ElevenLabs provide timestamps along the audio file. These are typically character-based timestamps, so we first have to calculate the timestamps for each word and then the duration for the entire text section.

Ready For The Next Steps
All Asset URLs that are generated will be saved in a VideoBlock object for convenience. The duration of the VideoBlock is the duration of the audio, as calculated above.

interface VideoBlock {
  text: string;
  imageUrl: string | null;
  audioUrl: string | null;
  startTime: number;
  duration: number;
  wordTimestamps: Array<{ word: string; start: number; duration: number }>;
}

3. Generate The Video

We have everything together now: The completed XML with timestamps, duration, and all assets. It’s now time to generate the video using the creative engine.

Let’s first add an empty container in our HTML that will be referenced for initiating the creative engine.

{
  /* Add container for Creative Engine */
}
<div id="cesdk_container" className="invisible mt-8 rounded-lg bg-gray-100" />;

We can now initialize the engine. Use this code snippet from our documentation.

We’ll then set up a function that creates a simple composition using the provided VideoBlocks. The engine requires you to first create a scene, append a page to the scene, and then create tracks within the page. The tracks are basically the layers in the timeline. I recommend setting one track as a background track using the following snippet:

// Set video track as a background track by connecting the page duration to the video track
engine.block.setAlwaysOnBottom(videotrack, true);
engine.block.setPageDurationSource(page, videotrack);

The Creative Engine provides powerful API calls to style & manipulate blocks in many ways. Here is an example of how we can animate the images with a slow zoom effect:

const imageZoomAnimation = engine.block.createAnimation('crop_zoom');
engine.block.setInAnimation(image, imageZoomAnimation);
engine.block.setDuration(imageZoomAnimation, block.duration);
engine.block.setBool(imageZoomAnimation, 'animation/crop_zoom/fade', false);

Export The Video & Scene
Exporting the video is easy. Just pass the page to the export function. In our example, we’re also saving the scene file so we can edit the video later.

// Export video
const progressCallback = (
  renderedFrames: number,
  encodedFrames: number,
  totalFrames: number
) => {
  console.log(`Progress: ${Math.round((encodedFrames / totalFrames) * 100)}%`);
};

const blob = await engine.block.exportVideo(
  page,
  'video/mp4',
  progressCallback,
  {}
);

// Save scene to string
const sceneData = await engine.scene.saveToString();

// Create scene blob
const sceneBlob = new Blob([sceneData], {
  type: 'text/plain',
});

4. Add A Video Editor

The last step is to add the video editor for post editing and pass the scene file. With CE.SDK, this effort is reduced to adding a few lines of code. In the init function, we’re configuring the editor and adding callbacks for the export:

const initEditor = async () => {
        const config = {
          license: 'A-O53TWXK5bfyconUx7e53S5YU7DzjuGpMAH5vvKjLd0zBa6IhsoF7zChy1uCVbj',
          userId: 'guides-user',
          theme: 'dark',
          baseURL: '<https://cdn.img.ly/packages/imgly/cesdk-js/1.44.0/assets>',
          role: 'Creator',
          ui: {
            elements: {
              view: 'default',
              panels: {
              },
              navigation: {
                position: 'top',
                action: {
                  save: true,
                  load: true,
                  close: true,
                  download: true,
                  export: true
                }
              },
              dock: {
                iconSize: 'normal', // 'large' or 'normal'
                hideLabels: true // false or true
              }
            }
          },
          callbacks: {
            onUpload: 'local',
            onSave: (scene: string) => {
              const element = document.createElement('a')
              const base64Data = btoa(unescape(encodeURIComponent(scene)))
              element.setAttribute(
                'href',
                `data:application/octet-stream;base64,${base64Data}`
              )
              element.setAttribute(
                'download',
                `video-${new Date().toISOString()}.scene`
              )
              element.style.display = 'none'
              document.body.appendChild(element)
              element.click()
              document.body.removeChild(element)
            },
            onClose: () => {
              onClose();
            },
            onLoad: 'upload',
            onDownload: 'download',
            onExport: 'download'
          }
        }

Conclusion

By following this cookbook, you can streamline the process of AI-generated video creation, making it fast and efficient. This method is especially useful for content creators, educators, and marketers looking to automate video production while maintaining creative control.
Next, try experimenting with video styles, refining AI scripts, or exploring advanced editing.
Feel free to GitHub repo and share your creations with us on X. Happy creating!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.45 Release Notes

Neslihan — Tue, 25 Feb 2025 14:11:57 GMT

Welcome to CE.SDK v1.45! We’re expanding creative possibilities and refining the editing experience. Reactions are now available on Android, enabling users to record, edit, and personalize real-time responses.

We also introduce improvements like smoother development with TypeScript and better text rendering across languages.

These solutions help creators and developers work smarter, engage better, and deliver high-quality content.

Record Reactions to Videos on Android

Now available on Android, the popular Reactions feature lets users record and add their real-time responses to videos.

This is a must-have for engagement-driven content, mirroring interactions on platforms like TikTok and Instagram. Reactions have become a key element of personal branding and connecting with other creators on social media. With Reactions, users can:

Connect with their audience—duet, respond to trends, or join conversations.
Add a personal touch—react to fan art, viral moments, or inspiring videos.
Show thought leadership—comment on user questions, industry news, or niche topics.
Increase reach—boost visibility and interaction.
Monetize content—sell products and showcase collaborations.

Reactions make content creation fast, easy, and personal.

With our shipped UI, users can select a vertical or horizontal layout to record their reaction video, then customize it with text, filters, adjustments, and more.

It’s easy to crop, trim, and further adjust your video clip—providing a flexible starting point for seamless editing.

Try the live demo and enable Reactions for Android with our documentation.

Improvements

Ensure Text Rendering Across Languages

When a glyph is unavailable in the selected font, CE.SDK automatically falls back to a configurable default font. This prevents localization issues and ensures seamless text rendering, perfect for your global projects.

Simplify Development with Complete TypeScript Support

We’ve improved TypeScript support by exposing previously missing types used in our UI APIs. This enhancement, based on user feedback, reduces complexity and ensures smoother development workflows.

View the Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.44 Release Notes

Neslihan — Thu, 06 Feb 2025 07:31:00 GMT

Welcome to CE.SDK v1.44! This release brings a range of updates designed to enhance your editing workflow across platforms. Enjoy more vibrant and true-to-life visuals with expanded P3 color support on iOS and Android. On the web, streamline your editing process with the new ability to add multiple clips per track, making timeline management more efficient.

Let’s dive into the updates!

Simplify Your Video Editing Workflow

Add multiple clips to a single track on your video timeline! With CE.SDK video editing for the web, you can insert video clips, text, overlays, and more on one track—resulting in a more organized and efficient workspace.

This update empowers you to create richer, more dynamic sequences while keeping your timeline neat and easy to navigate.

Maximize Efficiency
Simplicity and speed are key. By reducing the need for multiple tracks, this feature helps users to work smarter and faster—allowing them to focus on creative storytelling and deliver engaging content.

Get started with our Video Editing for Web documentation.

Get Vibrant Colors with P3/Wide Gamut Support

Unlock a broader color range with P3 color space support on iOS and Android devices. This enhancement ensures your designs and content are more vibrant, accurate, and true to life—giving you richer color options for all your projects.

Set up color spaces with our documentation for iOS and Android.

💡 The Canvas menu appears when you select a design block, letting you edit its settings, like duplicating a selected image.

Building on mobile configuration introduced in our last release, we’ve extended customization to the editor Canvas Menu for Android. This update gives you full control over how the menu and its items appear and behave in the editor:

Customization: Configure visibility, transition effects, and decoration. Adjust the style of the divider between buttons.
Item List Control: Modify, add, reorder, or hide menu items.
Flexible Button and Divider Configuration: Adjust button appearance, functionality, and divider styles to match your design requirements.

More customizations will be available soon, including for iOS.
Explore the documentation.

Improvements

Switch the Editor Language without Restarting

Easily switch languages in our web editor without restarting it. This update lets you update the UI language instantly via API, streamlining localization. Check our documentation, and translate right away!

Upcoming

Record Reactions to Videos on Android
We’re excited to announce that Reactions will soon be available on Android. Already live in CE.SDK on iOS, this feature—popularized by platforms like TikTok and Instagram—allows users to capture and share their real-time responses to videos. Enhance engagement and provide content creators with immediate, interactive feedback, making your video content even more dynamic.

View the Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

How To: Video Generation With Javascript

Antonello — Wed, 22 Jan 2025 09:53:00 GMT

Follow this tutorial to learn how to programmatically create videos in JavaScript directly in the browser.

In 2025 video is firmly established as powerhouse for engagement, gaining even more traction with the rise of short-form videos. After all, as humans, we naturally resonate with video content more than any other type of media.

The problem is that video generation is challenging—and doing it within the browser through JavaScript makes things even more complex. Fortunately, production-grade solutions like CreativeEditor SDK (CE.SDK) make the process not only possible but also easy to implement.

In this guide, you will learn how to programmatically generate videos in the browser via JavaScript using CreativeEditor SDK. We will cover various applications, including merging videos, adding audio tracks, and even integrating AI features.

Let’s dive in!

Why Programmatic Video Generation Matters

Videos are dominating social media, marketing, and online platforms such as e-commerce platforms (e.g., to showcase products). Numerous studies have proven that video is the most effective form of media for marketing, as it resonates deeply with us.

As the demand for personalized and dynamic content increases, traditional video production methods are becoming inefficient. The proliferation of channels with different demands on size and video quality as well as the need to create personalized videos at scale means that automated video creation is becoming indispensable. Most existing solutions run batch processing of videos on the server after gathering input data from various source. However, while server-side processing quickly becomes costly and introduces significant communication overhead client devices have become more powerful than ever.

CreativeEditor SDK bridges that gap by simplifying programmatic video generation in JavaScript within modern browsers. With CE.SDK, you can automate repetitive tasks and define scalable, personalized, and engaging video creation workflows, ensuring each generated video resonates with its target audience.

Get Started with CE.SDK for JavaScript

Now that we established why programmatic video generation is important, you are ready to get started with the tool designed for it: CreativeEditor SDK.

Learn how to integrate CE.SDK for video editing into a vanilla JavaScript application!

You can access the code for this tutorial on Github.

Requirements

The only prerequisites to follow this tutorial are:

A modern browser: CreativeEditor SDK runs directly in your browser’s JavaScript engine, so no additional setup is required. Any browser supporting WASM is enough.
A CreativeEditor SDK license: If you do not have one yet, sign up for a free trial.

If your web application uses Node.js, ensure you have the latest stable versions of both Node.js and NPM installed.

Import the library

In a vanilla JavaScript application, create a JavaScript module file (e.g., video-editor.js) and import the CreativeEditor SDK engine library inside it:

import CreativeEngine from '<https://cdn.img.ly/packages/imgly/cesdk-engine/1.42.0/index.js>';

Note: In this example, the SDK is served from our CDN for convenience. In a production environment, it is recommended to host all assets and libraries on your own servers for improved control and performance.

To use the video-editor.js module in an HTML page, import it with this line:

<script type="module" src="video-editor.js"></script>

Note the type=“module” attribute tells the browser to treat the script as an ES6 module, so that you can use import and export statements.

Alternatively, if you are using a module bundler like Webpack, Rollup, Parcel, or Vite, add the @cesdk/engine npm package to your project’s dependencies:

npm install @cesdk/cesdk-js

In this case, you can then import the headless SDK into your JavaScript code as follows:

import CreativeEngine from '@cesdk/engine';

Well done!

Setting Up Your Environment

In your JavaScript module, right below the library import, initialize the CreativeEditor SDK headless engine using the following code:

// src/video-editor.js

import CreativeEngine from '<https://cdn.img.ly/packages/imgly/cesdk-engine/1.42.0/index.js>';

// your CE.SDK license and user configs
const config = {
  license: '<YOUR_LICENSE>',
};

CreativeEngine.init(config).then(async (engine) => {
  // attach the engine canvas to the DOM
  document.getElementById('cesdk_container').append(engine.element);

  // do something with your instance of CreativeEditor SDK...

  // detach the engine and clean up its resource
  engine.element.remove();
  engine.dispose();
});

Replace <YOUR_LICENSE> with the license key provided in your CreativeEditor SDK subscription.

Ensure that the HTML page importing video-editor.js contains the following <div> element with the cesdk_container ID:

<div id="cesdk_container" style="width: 100%; height: 100vh;"></div>

The CreativeEngine.init() method will mount the editor engine component within this div.

Congratulations! You have successfully integrated the video editing engine.

Load a Video

Use the logic below inside the body of the CreativeEndine.init() callback function to load a video in CE.SDK:

// initialize a new video scene
const scene = engine.scene.createVideo();

// create a page block and attach it to the scene
const page = engine.block.create('page');
engine.block.appendChild(scene, page);

// set the dimensions of the video page
engine.block.setWidth(page, 720);
engine.block.setHeight(page, 1280);

// create a graphic block to hold the video content
// and set its shape to a rectangle
const video = engine.block.create('graphic');
engine.block.setShape(video, engine.block.createShape('rect'));

// create a fill type for the video and set the video file URI
const videoFill = engine.block.createFill('video');
engine.block.setString(
  videoFill,
  'fill/video/fileURI',
  '<https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-drone-footage-of-a-surfer-barrelling-a-wave-12715991.mp4>'
);

// apply the video fill to the video graphic
engine.block.setFill(video, videoFill);

// create a track block to manage the video timeline
// and attache the track to the page and the video graphic
// to the track
const track = engine.block.create('track');
engine.block.appendChild(page, track);
engine.block.appendChild(track, video);

// make the track cover the parent block
engine.block.fillParent(track);

This initializes a video scene using createVideo() and creates a "graphic" block to display the video in a rectangle, adding it to the page. Next, it loads a video from a remote resource and appends it as a track to the parent block.

For a complete explanation of how the above code works, refer to the documentation.

If you comment out the cleanup part, this is what you should see in your browser application:

Notice how the video from the remote URI has been loaded into the

element where CreativeEditor SDK is attached to the page’s DOM.Time to explore more programmatic video generation features!

Programmatic Video Generation Features

Follow the use cases below to see how CE.SDK makes automated video generation in JavaScript easier.Generate Video in Different Formats CE.SDK supports video export in MP4 format (with MP3 audio) with different resolutions and aspect ratios.To export a video with a specific aspect ratio, first adjust the width and height of the parent block to which the video block is attached:

// specify an aspect ratio of 9:16 (ideal for vertical videos)
engine.block.setWidth(page, 720);
engine.block.setHeight(page, 1280);

In this case, the video will be automatically adjusted to fit the specified dimensions. Horizontal videos will be cropped to match the defined vertical outline of the block, while vertical videos will be rendered as they are within the specified block.

Next, you can export the video with the configured 9:16 aspect ratio and a given resolution with:

// define the MIME type for the video export (MP4 format)
const mimeType = 'video/mp4';

// define a callback function to track the progress of video rendering and encoding
const progressCallback = (renderedFrames, encodedFrames, totalFrames) => {
  console.log(
    'Rendered', // log the number of rendered frames
    renderedFrames,
    'frames and encoded', // log the number of encoded frames
    encodedFrames,
    'frames out of', // log the total frames to be processed
    totalFrames
  );
};

// define the video export options
const videoOptions = {
  duration: 5, // video duration in seconds
  framerate: 30, // video framerate (frames per second)
  targetWidth: 480, // target video width in pixels (for the resolution)
  targetHeight: 852, // target video height in pixels (for the resolution)
};

// export the page as an MP4 video
const blob = await engine.block.exportVideo(
  page, // the page containing the design video block
  mimeType, // the MIME type for the video (MP4)
  progressCallback, // the callback to track video rendering progress
  videoOptions // the options for the video export
);

// create an anchor element to trigger the video download
const anchor = document.createElement('a');
anchor.href = URL.createObjectURL(blob);
anchor.download = 'video.mp4'; // output video name
anchor.click();

The exportVideo() function produces a blob video file of the specified MIME type. Note that the export occurs over multiple iterations of the update loop, with a frame being encoded in each iteration.

The targetWidth and targetHeight options are optional. If used, the video will be resized to fit the target dimensions while maintaining its aspect ratio. This is useful for setting a specific resolution for the output video while preserving the parent block’s width and height on the output video. If omitted, the produced video will match the resolution of the width and height specified with setWidth() and setHeight(), respectively.

The progressCallback() function will track and display the video generation progress in the browser’s console:

The final lines of the above code snippet trigger the download operation, mimicking the action of clicking a link to download a resource. This is a common JavaScript technique to programmatically trigger a file download.

Once the export process completes, the output video file (video.mp4) will be automatically downloaded to the browser’s download directory. Open its properties and this is what you will see:

Note the video file’s resolution and the fact that its duration is set to 5 seconds, as defined by the duration attribute.

These options enable you to easily create clips with different lengths, aspect ratios, and resolutions, optimized for various audiences and social media platforms like LinkedIn, Instagram, Facebook, X, TikTok, YouTube, and more.

Add Audio Tracks

Adding an audio track for narration or background music to your clip is straightforward. First, create an “audio” block and add your audio resource to it:

const audio = engine.block.create('audio');
engine.block.appendChild(page, audio);
engine.block.setString(
  audio,
  'audio/fileURI',
  'https://cdn.img.ly/assets/demo/v1/ly.img.audio/audios/far_from_home.m4a'
);

Next, you can easily adjust the volume, and apply fade-in and fade-out effects as follows:

// set the volume level to 80%
engine.block.setVolume(audio, 0.8);

// start the audio after 1 second of playback
engine.block.setTimeOffset(audio, 1);

// set the audio block's duration to 5 seconds
engine.block.setDuration(audio, 5);

Here is breakdown of the functions used above:

setVolume(): Adjusts the audio volume of the block, with a range from 0 (0%) to 1 (100%).
setTimeOffset(): Sets the time offset of the block relative to its parent. This determines when the block starts playing in the timeline.
setDuration(): Sets the playback duration of the audio block in seconds, defining how long the block will play during the scene.

In this example, the audio starts at the 1-second mark and plays for 5 seconds (from 1s to 6s), with the volume set to 80% of the original.

Merge Videos

To stitch multiple video clips together in CE.SDK, you first need to import and create separate video blocks:

// create the first video block
const video1 = engine.block.create('graphic');
engine.block.setShape(video1, engine.block.createShape('rect'));
const videoFill1 = engine.block.createFill('video');
engine.block.setString(
  videoFill1,
  'fill/video/fileURI',
  'https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-drone-footage-of-a-surfer-barrelling-a-wave-12715991.mp4'
);
engine.block.setFill(video1, videoFill1);

// create the second video block
const video2 = engine.block.create('graphic');
engine.block.setShape(video2, engine.block.createShape('rect'));
const videoFill2 = engine.block.createFill('video');
engine.block.setString(
  videoFill2,
  'fill/video/fileURI',
  'https://cdn.img.ly/assets/demo/v2/ly.img.video/videos/pexels-kampus-production-8154913.mp4'
);
engine.block.setFill(video2, videoFill2);

At this point, you have two different video clips loaded from separate video files.

Now, assume you want to produce a 10-second video that includes both clips. Start by setting the duration of the page block to 10 seconds with the setDuration() method:

engine.block.setDuration(page, 10);

Then, suppose you want the first clip (video1) to play for 7 seconds, and the second clip (video2) to fade in after that and play for 3 seconds (from second 1 to second 4 of the original second clip). This is how you can set that up:

// set the duration of the first and second video clips
engine.block.setDuration(video1, 7); // video1 plays for 7 seconds
engine.block.setDuration(video2, 3); // video2 plays for 3 seconds

// wait for the second video to be fully loaded before applying the trim options
await engine.block.forceLoadAVResource(videoFill2);
// trim the second video to start at 1 second and play up to second 4
engine.block.setTrimOffset(videoFill2, 1);
engine.block.setTrimLength(videoFill2, 4);

// add a fade-in animation to the second video
const fadeInAnimation = engine.block.createAnimation("fade");
engine.block.setInAnimation(video2, fadeInAnimation);

These are the methods used above:

setDuration(): Defines the duration (in seconds) that a video block will be active during playback.
**forceLoadAVResource()**: Ensures that the audio or video resource (in this case, video2) is fully loaded before applying trim operations.
setTrimOffset(): Sets the start point (in seconds) within the video where playback begins.
setTrimLength(): Defines the length of the video to be played from the trim offset.
createAnimation(): Creates a fade-in animation for the second video (video2) to smoothly transition into the clip.

Finally, add both video blocks to the track block for playback:

const track = engine.block.create("track");
engine.block.appendChild(page, track);
engine.block.appendChild(track, video1); // add the first video to the track
engine.block.appendChild(track, video2); // add the second video to the track
engine.block.fillParent(track);

The result will be:

Note how the second clip fades in after 7 seconds of the first clip, lasting for 3 seconds as specified.

Incorporate Text Variables

Now, suppose you want to display cool words or messages in your videos. You can easily achieve this by adding a "text" block to your page, as shown below:

const text = engine.block.create("text");
// set the vluae of the text variable
engine.block.replaceText(text, "Surfing\nis\nCOOL!");
// set the text color to white and semi-transparent
engine.block.setTextColor(text, { r: 255.0, g: 255.0, b: 255.0, a: 0.8 });
// set the font size for the text
engine.block.setTextFontSize(text, 30);

// positioning the text on an absolute position on the video
engine.block.setWidthMode(text, "Auto");
engine.block.setHeightMode(text, "Auto");
engine.block.setPositionX(text, 130);
engine.block.setPositionY(text, 800);

// append the text block to the page
engine.block.appendChild(page, text);

The video canvas will now display a white “Surfing is COOL!” message as follows:

In this example, the content of the text block is static. However, it can easily be retrieved from a database or any other source for programmatic video generation with different messages, including personalized ones for tailored outreach (e.g., for custom greetings or offers).

Advanced Use Case: Prompt-Based Video Generation Using AI

As highlighted in our recent LinkedIn post, IMG.LY has just experimented with an MVP that turns keywords into fully edited short videos, complete with a script, voiceover, and visuals—all automated.

That is made possible by integrating CE.SDK with an LLM for coding and scriptwriting, ElevenLabs for voice synthesis, and Flux (via fal.ai) for visuals. Specifically, the LLM uses CE.SDK for video composition, animation, and rendering.

This is just the beginning, showing how AI can be combined with CE.SDK to programmatically generate video scripts—or even templates. A possible scenario would be to use the Creative Editor SDK plugin API to extend CE.SDK with custom plugins that adds AI directly to the design editor engine.

For example, you could develop a plugin with an LLM integration that allows users to input a prompt like: “Create a 15-second video with the text ‘Welcome to our App!’ and upbeat background music.” The LLM will process the request, using Creative Editor SDK to automate video creation by leveraging its powerful features.

Alternatively, the AI could produce a video that you can then automatically edit and enhance using CE.SDK capabilities. One thing is certain: the possibilities are endless with CE.SDK + AI!

Conclusion: The Future of Programmatic Video Generation

CE.SDK for JavaScript offers exceptional versatility in automating video workflows, allowing you to easily generate, edit, and render videos with just a few lines of code. You can stitch multiple video files, add background audio tracks, and incorporate dynamic text variables to finally produce videos in the browser—all thanks to the CE.SDK headless engine.

Looking ahead, AI integration with CE.SDK is unlocking even more powerful possibilities, such as prompt-based video content creation and automatic editing directly in the browser.

With the headless engine provided by CreativeEditor SDK, programmatic video generation in JavaScript can be implemented in just a few minutes. Explore the video capabilities of CE.SDK and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

A Modern JavaScript Video Editor: Integration Guide

Antonello — Thu, 16 Jan 2025 15:29:04 GMT

Discover how to integrate IMG.LY’s JavaScript video editor into your web application and unlock its full range of features.

Video content has always been a powerhouse for engagement, becoming even more impactful with the rise of short-form videos. Platforms like YouTube and TikTok have built their empires on video content, as that remains the preferred medium for both creating and consuming information.

Not only have those platforms changed how we consume information, but they have also transformed user behavior. We do not just want to watch videos—we want to create them, expecting the process to be both simple and professional.

Here is why embedding video creation and editing capabilities directly into your website, web or mobile application can significantly boost user engagement and satisfaction.

In this guide, you will learn how to integrate a JavaScript video editor using CreativeEditor SDK (CE.SDK). We will explore when it is most useful and what customizations it supports to suit your unique needs.

Want to jump into the code? For a quick start, visit our GitHub repository.

Let’s dive in!

Why Does Your JavaScript Web App Need a Video Editor?

The world of content is constantly evolving—from text to images, from static visuals to long-form videos, and now from long videos to bite-sized clips. What stays consistent is that video content resonates deeply with us as humans.

No wonder, TikTok tops global app download charts, while YouTube, Facebook, and Instagram rank among the most visited sites right after Google. These platforms have not just built their empires on video—they have revolutionized how we think about video production.

They set the gold standard by enabling anyone to create professional-looking videos with minimal effort, offering tools for adding filters, overlays, audio, and text in just a few clicks. This ease of use has made intuitive video editing a must-have for web applications featuring social media integration, marketing ambitions, or content management capabilities.

By embedding a user-friendly JavaScript video editor in your application, you can ride this trend, providing a feature billions of users are already familiar with. The payoff? Lower barriers to user-generated content, enhanced engagement, and wider content distribution.

CreativeEditor SDK from IMG.LY delivers a seamless, high-performance, and feature-rich JavaScript solution for in-browser design, video, and photo editing—ready to meet your users’ expectations.

Getting Started: Adding a Video Editor in JavaScript

Follow this step-by-step guide to learn how to integrate a JavaScript video editor into your site using CreativeEditor SDK.

Get started with integrating CE.SDK in a vanilla JavaScript application!

Requirements

CreativeEditor SDK runs directly in your browser’s JavaScript engine, so no specific prerequisites are needed.

If your web application relies on Node.js, ensure that you have the latest stable versions of Node.js and NPM installed.

Installation and Library Import

In a vanilla JavaScript application, create a JavaScript module file (e.g., video-editor.js) and import the CreativeEditorSDK library:

import CreativeEditorSDK from '<https://cdn.img.ly/packages/imgly/cesdk-js/1.42.0/index.js>';

Note: In this example, the SDK is served from our CDN for convenience. In a production environment, you should host assets on your own servers for improved control and performance.

Alternatively, if you are using a bundler like Webpack, Rollup, Parcel, or Vite, and you want to integrate the CreativeEditor module within your existing project, simply add the @cesdk/cesdk-js npm package as a dependency to your project:

npm install [@cesdk/cesdk-js](<https://www.npmjs.com/package/@cesdk/cesdk-js>)

You can then import it as below:

import CreativeEditorSDK from "@cesdk/cesdk-js";

Well done!

Initialize the Video Editor

In your JavaScript module, right below the library import, initialize CreativeEditor SDK with the following logic:

// src/video-editor.js

import CreativeEditorSDK from '<https://cdn.img.ly/packages/imgly/cesdk-js/1.42.0/index.js>';

// your CE.SDK license and user configs
const config = {
  license: '<YOUR_LICENSE>',
  userId: '<YOUR_USER_ID>',
};

CreativeEditorSDK.create('#cesdk_container', config).then(async (editor) => {
  // do something with your instance of CreativeEditor SDK...

  // when done, clean up and free up resources
  editor.dispose();
});

Replace <YOUR_LICENSE> with the license key you get from CreativeEditor SDK and <YOUR_USER_ID> with your user ID. The user ID is optional and helps track MAUs (Monthly Active Users) across different devices.

To integrate the video-editor.js file, import it into your HTML page like this:

<script type="module" src="video-editor.js"></script>

Note the type=“module” attribute. This tells the browser to treat the script as an ES6 module, so that you can use import and export statements within your JavaScript code.

Make sure the same HTML page includes the following div element where the video editor will be embedded:

<div id="cesdk_container" style="width: 100%; height: 100vh;"></div>

This will load the video editor with the default video preset, giving users the ability to trim, cut, apply filters, add text overlays, include music, and more to enhance their videos.

Congratulations, you have successfully integrated the video editor!

CreativeEditor SDK: JavaScript Video Editing API for Web, Desktop, and Mobile

You just saw how to integrate the CE.SDK into a browser application. Yet, do not forget that the SDK supports a wide variety of frameworks and libraries—including mobile and web applications.

Web Framework Support

CE.SDK is designed to work with several popular JavaScript frameworks and libraries. For more detailed integration instructions, take a look at the official page on the documentation for each framework:

Additionally, the SDK is available for headless mode and works well in a Node.js environment.

Cross-Platform Support

One of the standout features of CreativeEditor SDK is its cross-platform philosophy. This extends beyond JavaScript ecosystems for the web, and includes mobile (iOS and Android) and desktop, thanks to support for:

That is made possible through a powerful engine that powers your app via features like:

Unified API: The same underlying API is shared across web, desktop, iOS, and Android applications.
Cross-Platform Feature Parity: CE.SDK guarantees that features are available consistently across platforms. Since core functionality is implemented at the engine level, features you use on one platform will be available on others, with only slight differences in release timelines.
Interoperable Designs: Designs created in the editor component on one platform work on all platforms supported by CE.SDK. Whether a user is designing on iOS, Android, or the web, their design files can be imported seamlessly between platforms.

Native iOS and Android libraries, along with other frameworks, are also available. Learn more in our documentation!

JavaScript Video Editor Use Cases

It is time to explore some key use cases. Let’s see how the unique features of the CreativeEditor SDK can cover many scenarios!

CreativeEditor SDK features what users need to create engaging and easy-to-share videos optimized for popular social media platforms like Instagram, TikTok, YouTube, LinkedIn, and more. This is possible due to a large set of capabilities, which include:

Reusable Templates: Users can either design their own templates and share them with the community or download those created by others. This variety of ready-to-use templates for different scenarios helps users skip the daunting blank canvas to bring their ideas to life. Explore various templates in our video editor demo.
Music and Audio Library: In CE.SDK, adding audio to videos is simple and intuitive. Given the impact of audio, users can set the tone with music or narration in just a few clicks, creating a more immersive atmosphere for their content.
Animations: The SDK includes a wide selection of animations to enhance the design elements positioned within video scenes.

After creating a video within your web app, users can export it directly from CE.SDK in the appropriate formats and aspect ratios for their target social media platforms.

Product Showcase for E-Commerce Applications

Images have long been a simple yet powerful tool for showcasing products on e-commerce platforms, and sites like eBay, Amazon, AliExpress, and Walmart have relied on them for years (and still do!).

However, in recent years, customer expectations have shifted. Today’s buyers crave more interactive and immersive ways to visualize products—they want to see how items look and perform in real-world settings.

Short, dynamic product videos offer a superior presentation experience. According to an analysis by Firework, video content increases purchase likelihood by 51%, highlighting its undeniable impact on buying decisions. This is why many sellers have already embraced video to enhance product displays.

With a CE.SDK-powered JavaScript video editor, producing compelling videos that deliver a seamless, multi-device shopping experience becomes effortless. Plus, reusable templates help you generate unique product videos while maintaining brand consistency.

Engaging Video Messages for Sales Outreach

Marketers and sales teams know that cold emails often come across as impersonal and boring. The solution to this challenge? Personalized video messages!

Customized video messages are proving to be an effective outreach tool, with studies showing significantly higher conversion rates.

With CreativeEditor SDK, sales and marketing teams can easily create personalized videos tailored to individual clients or leads, making their outreach more engaging and impactful. Unlike traditional email campaigns, personalized videos create a sense of real, one-on-one interaction.

Additionally, the SDK’s translation features help sales teams break down language barriers, allowing them to reach a global audience. This ensures that the personalized messages resonate with clients from different regions.

Explore this feature in our translation and internationalization demo.

Digital Asset Management

DAM, short for Digital Asset Management, is the process of organizing, storing, and managing digital assets, including video content. As your business grows, an effective DAM system becomes key to enabling multiple teams to easily access and collaborate on assets across different projects.

CreativeEditor SDK backs DAM workflows by offering powerful tools to organize, edit, and repurpose video content through an embeddable, centralized interface. Its core JavaScript video editor component empowers users to:

Effortlessly adapt and customize video assets.
Ensure brand consistency while automating updates across various assets.
Edit content efficiently within a collaborative environment.

Although CE.SDK does not provide built-in role-based permissions, it integrates smoothly with backend systems to control user access. In particular, its flexible architecture supports the definition of custom hooks for permission checks to customize the editor’s behavior.

Automated Video Creation

CreativeEditor SDK streamlines the automation of asset creation by enabling users to generate on-brand video content variations for multiple channels. Thanks to adaptable templates, dynamic text variables, and lockable designs, the SDK ensures consistency and brand compliance over all produced videos.

Given a template, the creative engine automates the generation of countless personalized assets by populating them with resources from various data sources. That eliminates manual effort, reduces design bottlenecks, and accelerates production, making it perfect for scaling campaign-specific videos or building dedicated marketing tech.

Customizing your CreativeEditor SDK Instance

CreativeEditor SDK is not a one-size-fits-all JavaScript video editor. Instead, it is built for flexibility, offering a wide set of customization options to tailor the editor to your unique requirements and brand identity. These include:

Theming Options: Match the editor’s look to your app’s style with built-in themes, generate new ones using the theme generator, or create custom themes manually. Explore theming possibilities in our dedicated demo.
Custom UI: Build entirely bespoke JavaScript video editing user interfaces that align with your specific use case.
Media Library Integration: Set up sortable, flexible media libraries and integrate popular third-party libraries like Unsplash and others. Get easy access to external assets.
Localization: Adapt the editor to different languages and regions with full i18n support, letting you extend or overwrite any language setting.
Toolbar Customization: Rearrange toolbar elements, update icons, or rename tools to deliver a tailored user experience.

With these customization options, CreativeEditor SDK ensures your video editor is as unique as your JavaScript application.

Conclusion

Implementing a JavaScript video editor in your web app can greatly improve the user experience and provide a powerful feature to drive higher engagement, retention, and even product distribution. This is true whether you are developing an e-commerce platform, a social media app, a tool for influencers, or a SaaS web solution.

With the CreativeEditor SDK, adding a fully customizable video editing experience to your web application takes only a few minutes. Explore CE.SDK’s video capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

A Modern React Design Editor: Integration Guide

Antonello — Thu, 16 Jan 2025 14:38:04 GMT

Learn how to integrate IMG.LY’s design editor for React into your web app and explore various use cases and customization options.

It is no surprise that content asset generation is a key for businesses in marketing and promotion, as well as for users on social media. Whether it is images or videos, these are the types of posts we engage with and relate to the most as human beings—much more than plain text.

Most content that resonates with an audience is not created from scratch. Instead, it often comes from pre-designed designs that are reused, repurposed, and adapted to fit different content or goals. That is why adding a design editor to your React application is so valuable.

In this guide, you will learn how to build a React design editor using the CreativeEditor SDK (also known as CE.SDK). We will also cover when this is useful and the customization options the editor supports.

For a quick start, check out the GitHub repository with the full code integration.

Let’s dive in!

Why Add a Design Editor to Your React App?

Before answering the question, you must understand what a design is and how helpful it can be. In this context, a design can be defined as a set of predefined assets, text overlays, placeholders, and other elements arranged in a specific layout.

Users can use designs to create images or videos with a particular structure and for specific use cases at scale. Thus, a design editor would enable your users to create personalized assets—and in the case of a React application, directly within the browser.

Building such a platform is not a piece of cake. A robust design editor requires numerous features, such as intuitive drag-and-drop functionality, template management, image manipulation tools, and more. It also generally requires integrations with third-party services and dependencies to enhance its capabilities.

Fortunately, an all-in-one solution like CreativeEditor SDK from IMG.LY makes it easy to embed this powerful design editor into your application. It supports various programming languages and technologies, including React. With extensive features and customization options, CE.SDK has no limits on the types of designs users can generate.

CE.SDK features adaptable design tools for diverse industries and use cases, such as social media post creation, digital asset management, and AI-assisted content personalization. That is why it is used by hundreds of customers, including governments, Fortune 500 companies, and dozens of startups.

CreativeEditor SDK is trusted by companies like Swiss Post, Brother, Optimizely, HP, and many more.

How to Integrate a Design Editor in React

Follow this step-by-step tutorial to see how to quickly set up a React design editor in your web app using CreativeEditor SDK. For a complete guide, read our tutorial on building a Canva alternative in React using CE.SDK.

Integrate CE.SDK into your React app using the instructions below!

Requirements

Before getting started, ensure that you meet the following prerequisites:

The latest LTS version of Node.js: If you do not have it installed, download it from the official site and follow the installation instructions.
A React 18+ application: If you do not have one set up, follow the instructions below to create it.
A CreativeEditor SDK license: In case you do not have a license, sign up for a free trial or buy a license.

If you do not have a React project set up yet, create a new one using Vite:

npm create vite@latest my-react-video-editor-app -- --template react

Open your project in a JavaScript IDE, and you are all set to start coding!

Install CE.SDK

Install CE.SDK by adding the @cesdk/cesdk-js package to your project’s dependencies:

npm install @cesdk/cesdk-js

Your package.json file will now list it as a dependency:


"dependencies": {
  "@cesdk/cesdk-js": "^1.41.1",
  "react": "^18.3.1",
  "react-dom": "^18.3.1"
}

Create the Design Editor Component

In the src folder of your React project, create a DesignEditor.jsx file. This will house your CE.SDK-based React design editor component.

Inside it, import CreativeEditorSDK from @cesdk/cesdk-js:

import CreativeEditorSDK from '@cesdk/cesdk-js';

Next, initialize your video editor React component with the following logic:

// src/DesignEditor.jsx

import CreativeEditorSDK from '@cesdk/cesdk-js';
import { useEffect, useRef, useState } from 'react';

// your CE.SDK license and user configs
const config = {
  license: '<YOUR_LICENSE>',
  userId: '<YOUR_USER_ID>',
};

export default function DesignEditor() {
  const cesdk_container = useRef(null);
  const [cesdk, setCesdk] = useState(null);

  useEffect(() => {
    // if the CE.SDK container has already been initialized
    if (!cesdk_container.current) {
      return;
    }
    let cleanedUp = false;

    // initialize a CE.SDK instance
    let instance;
    CreativeEditorSDK.create(cesdk_container.current, config).then(
      async (_instance) => {
        instance = _instance;
        if (cleanedUp) {
          instance.dispose();
          return;
        }

        // customize the design editor behavior...

        setCesdk(instance);
      }
    );

    // clean up the CE.SDK instance
    const cleanup = () => {
      cleanedUp = true;
      instance?.dispose();
      setCesdk(null);
    };

    return cleanup;
  }, [cesdk_container]);

  return (
    <div
      ref={cesdk_container}
      style={{ width: '100vw', height: '100vh' }}
    ></div>
  );
}

Replace <YOUR_LICENSE> with the license key from CreativeEditor SDK, and <YOUR_USER_ID> with your user ID to optionally track monthly active users (MAUs) across different devices.

The component defined above embeds CreativeEditor SDK into an HTML <div> element. See how to customize and use it in the docs.

Use the Component

You can now import the DesignEditor component as below:

import DesignEditor from "./DesignEditor";

Then, add it to the DOM as follows:

<DesignEditor />

This will open the design editor with the preset, enabling users to place assets, set constraints, and perform other tasks to create their final designs.

React Design Editor Use Cases

Now that you are familiar with integrating a React design editor into your web app. It is time to explore key use cases and see how CreativeEditorSDK can cover many scenarios.

CreativeEditorSDK offers a wide suite of tools to help users easily create high-quality, personalized, on-brand social media assets in your React application.

With a vast library of professionally designed templates, it streamlines the design process and ensures consistency across multiple platforms. Users can efficiently produce engaging content tailored for

Key features include customizable dimensions, filters, placeholders, and text overlays. These give users what they need to craft content optimized for various social networks, such as Instagram, X, LinkedIn, and more. That is possible thanks to platform-specific export options to produce ready-to-upload assets.

The SDK’s capabilities extend beyond static images, supporting the creation of dynamic short videos, stories, and reels. Do not forget that CE.SDK can also function as a modern React video editor.

Digital Asset Management

DAM, short for Digital Asset Management, is the process of organizing, storing, and efficiently retrieving a wide range of digital files.

CreativeEditorSDK supports DAM by helping teams centralize asset creation and editing directly within an embeddable React design editor—eliminating the need for external tools. This is achieved through features like pre-defining and locking branding elements—including logos, fonts, and color schemes—to ensure brand consistency across all generated content.

Additionally, CE.SDK offers :

Design validation tools to prevent the creation of misaligned assets.
Automatic asset generation to streamline content variations.
Robust asset management features for handling both generated content and external assets retrieved via third-party APIs.

Marketing Automation

Businesses can use CreativeEditor SDK to scale marketing campaigns through dynamic, unique, customizable design templates. In detail, this design editor allows you to generate campaign materials—like personalized flyers, ads, or banners—quickly and at scale.

Key features for marketing automation include:

Automation: Integrate with CRM or marketing tools to auto-populate templates.
Dynamic Templates: Easily create content with placeholders or custom text variables for personalized outputs and A/B testing.
Brand Compliance: Lock assets like logos and fonts for consistent messaging and branding.

AI-Powered Applications

AI is rapidly becoming a key component of more and more applications, with many integrating it in some form. Still, a critical step is often overlooked: manual editing and supervision.

CreativeEditorSDK bridges that gap by seamlessly integrating with AI-powered workflows while guaranteeing users the ability to manually define constraints, placeholders, and rules that the AI-driven automation process must follow.

Example of how a CE.SDK-powered React design editor could work with AI includes:

Automated template suggestions based on industry or use case.
Prompt-based content generation where users describe their needs, and the app delivers a draft design.
Automated content optimization, where the AI evaluates design elements and suggests improvements for readability and visual appeal.

AI-generated content is impressive, but it is essential to find the right balance between automation and manual control in the content generation process for the best results.

Product Mockups

CreativeEditorSDK simplifies the creation of product mockups, packaging designs, and marketing materials, including printable brochures and business cards.

CE.SDK features powerful customization tools and flexible capabilities, including alignment guides and snap tools to ensure perfect positioning. Additionally, color pickers and vector shape editors enable you to explore and create professional design concepts.

E-Learning Materials

Thanks to CreativeEditorSDK, you can integrate a design editor into your React e-learning application.

The embeddable design editor supports the entire process of creating e-learning materials, including custom graphics like annotations, diagrams, and visual explanations, or videos recorded directly from the camera by the teachers/educators/coaches. The component also allows for the creation of unique and customized student certificates.

Those features empower educators and content creators in the education sector to craft visually engaging and impactful learning materials.

Customization Options With CreativeEditor SDK

CreativeEditor SDK does not provide a static React design design experience. Instead, it offers extensive customization options to align with your application’s branding and workflow. Key customization options are:

Theming: Adjust the editor’s theme to match your app’s design. Choose from built-in themes, generate new ones using the theme generator, or create a custom theme manually. Test themes in the demo page.
Custom UI Plugins: Add custom UI components to fit your design style through custom plugins.
Enabling/Disabling Features: Tailor the editor by enabling or disabling features like specific fonts, templates, or tools.
Toolbar Customization: Reposition toolbar elements, change icons, or rename tools to suit your needs, creating a unique editing experience.
Custom Templates, Assets, Themes: Add your own templates, assets, or themes for a more personalized user experience.
Media Libraries: Set up customizable, sortable media libraries. CE.SDK supports integration with third-party libraries, allowing easy access to external media assets.
Custom Integrations via Events: Hook into the editor’s events to trigger workflows, save progress, or integrate with external systems.
Internationalization: Adapt the design editor component for different languages and regions with full i18n support. Overwrite and extend text strings in any language.

Conclusion

CreativeEditor SDK’s compatibility with React’s component-based architecture and large set of features ensures simple integration into any project, making it ideal for a wide range of industries—from marketing and e-learning to e-commerce and social media.

A CE.SDK-based design editor offers UI customization and unique feature options to adapt to any use case. Additionally, its support for AI workflows ensures your application remains future-proof and ready to meet evolving user needs.

By following the steps outlined in this blog post, you can bring professional-level design editing features to your web users. Explore CE.SDK’s capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

How I Built a Short Video Generator with AI & CE.SDK in One Day

Eray — Thu, 09 Jan 2025 11:27:13 GMT

Here’s the crux of product development in the age of LLMs: how much can AI truly accelerate the development process?

We have seen videos of solo developers building small apps entirely with AI with just a few prompts. But how does it scale to more complex development projects? As LLMs rapidly evolve, their scope and impact will only increase.

That’s why I regularly challenge myself to build a small project with the help of AI. I’m a prime candidate to test the AI productivity boost: a jack-of-all-trades (and a master of none) with a background in both design and engineering, yet no hands-on experience in the past five years. My latest challenge? Build a web-based short video generator within one day.

In this post, I’ll share the most intriguing takeaways from tackling this project.

Why a Short Video Generator?

Why focus on this idea? It’s simple: to ride the wave of a new trend. A format called “faceless” short videos is gaining traction among creators on platforms like YouTube and TikTok.

https://www.youtube.com/embed/DfQ3fhqfKVc?feature=oembed

What’s fascinating about these videos is their automation: an LLM generates a script, which is then transformed into speech, images, and text assets using various AI services. These assets are automatically assembled into a cohesive video.

The general concept is compelling: It’s still generative content, but mixed with classic video composition techniques. This approach offers greater accuracy, consistency, and control over pure generative AI.

The potential to automate video production at this scale is exciting. Add its relatively low complexity and high production value, and it became the perfect topic for my challenge.

Enter CE.SDK

Another reason I chose this challenge was its compatibility with CE.SDK, our design and video editor library. CE.SDK offers a robust editing toolkit that integrates into any product with just a few lines of code. Its features, like headless mode, are ideal for automating workflows like video generation.

Most faceless video services use React-based video generation and achieve fair results. However, using CE.SDK instead of a react-based library could potentially boost the overall experience with three critical improvements:

Editable Outputs: This is huge. Full automation often needs human adjustments for fine-tuning. CE.SDK enables automated video generation while allowing manual refinement of the results.
Enhanced Visual Quality: CE.SDK has its own rendering pipeline, allowing for more nuanced visual effects and animations. When you’re competing against others in this space, it can make a huge difference if you’re able to produce higher fidelity in the visual output.
Visual Design Workflow: Create design components or even entire templates visually, and then use them via code. This authoring workflow can be extremely helpful in creating rich, interesting designs for the generated videos.

The Ground Rules

To keep the challenge focused, I set strict rules:

Time Limit: Spend no more than 12 hours on the challenge.
No Manual Coding: Avoid writing any code yourself—everything should be built through conversations with AI.
Trust the AI: Do not read or analyze code generated by the AI. Rely entirely on its decisions.
Skip External Research: Do not read or explore the APIs you intend to use. Instead, provide links to the AI and let it determine how to use them.
Compare AI Performance: Alternate Claude Sonnet 3.5 and ChatGPT o1 for code generation to evaluate which performs better.

The Tools & Workflow

Code Editor: Cursor
Built on VSCode’s foundation, Cursor stood out as the only editor offering both an integrated chat interface and the ability to switch between different LLMs. However, with GitHub’s recent significant updates to Copilot, I’ll switch to VSCode with Copilot for future challenges.

UI Prototyping: Claude Artifacts
Rather than building the entire project in my code editor, I chose to prototype the UI directly through Claude’s web interface. The benefits were immense:

Instant results: To create an artifact, Claude streamlines development by automatically writing and compiling code while leveraging essential UI libraries and components. This automation eliminates setup time and technical overhead, allowing me to focus purely on design iterations.
Instants Variations: Claude enables rapid prototyping through parallel conversations. When a design direction didn’t quite work, I could simply start a fresh conversation with modified requirements and evaluate a new prototype. This approach helped me develop three viable concepts quickly - a pace that would have been impossible in a traditional code editor.
Quality of execution: Claude transforms rough concepts into polished, intuitive interfaces. Its suggestions often surpassed my initial ideas, offering sophisticated solutions I hadn’t considered.
Keep it clean: By prototyping outside the code editor, I kept the main project’s codebase clean and focused. This separation prevented the accumulation of experimental code and maintained the clarity of our primary development environment.

Quickly prototype your interface with Claude Artifacts.

APIs
Key APIs used in the project included:

Script Generation: Claude Sonnet 3.5 vs various ChatGPT models.
Image/Video Assets: Fal.ai Flux models.
Speech Synthesis: ElevenLabs.

Building the App: Divide and Conquer

After having prototyped the UI, I started to chat with the LLM inside the code editor so that it can code the app. To work with the AI efficiently, I followed a divide-and-conquer approach. Rather than simply asking it to “build me a video app,” I broke down the problem into manageable steps:

Generate a video script
Create an AI prompt that includes user input and examples of the desired output format. Pass this prompt to the LLM API.
Parse the script to generate assets (speech, images, text)
Parse the LLM’s response to extract image prompts and speech paragraphs. Send these to their respective APIs.
Compose the final video
Load all the generated assets into a predefined template to generate the finished video through the CE.SDK library.

After completing these steps, I was finally able to generate my first fully automated videos! With a few more tweaks and additions, I had an MVP ready within twelve hours.

The final result: A Short Video Generator

There are still some missing features, partly because I spent a significant amount of time refining the prompt to generate the video script. I also had to bend the rules occasionally—sometimes the LLM would hit a wall, and I had to read or write small snippets of code.

Key Takeaways

Engineering Knowledge Is Essential
You should have some engineering background to achieve the AI productivity boost in development.

AI doesn’t solve everything for you. You are still the architect. You provide a lot of input and guidance. AI often needs to be pointed to the right strategy. Foundational knowledge of computer science is hugely advantageous for working with AI effectively.
As mentioned, I had to read and write a few lines of code myself. Without coding experience, I would have probably not been able to progress, as the LLM was not able to.
The getting started experience is nowhere close to novice-friendly. How do you get started with a new project in a code editor that actually requires you to do the setup manually? My workaround was to create an empty project, and then ask the LLM to instruct me to use a boilerplate for react. Again, this is engineering knowledge, any novice would have hit a wall already at this point.

Claude Outperformed ChatGPT
Claude was a clear winner in the side-by-side comparison, because of three reasons:

Claude Artifacts was a game changer for UI prototyping.
It was generally better at writing and understanding code. Difficult to quantify, but in some cases Claude fixed the mess ChatGPT left in the code
Claude can process URLs, which makes working with APIs much smoother.

Who would have thought new LLMs would catch up to OpenAI so quickly after they released the first version of ChatGPT?

Complexity Slows AI
The more code in my project, the slower the overall progress. LLMs struggled with the growing complexity. Their context windows filled more quickly, and their responses became increasingly unreliable. At some point, it becomes extremely difficult to make architectural changes, especially if this affects multiple parts of the app. When trying to fix errors, you’ll often find yourself in a whack-a-mole game. While the AI would resolve one issue, it would inadvertently introduce new problems elsewhere, creating an endless loop of fixes and regressions.

Ultimately, the time invested in this challenge was well worth it. While LLMs can’t build products end to end on their own, they can significantly streamline product development when paired with the right human collaboration. The real question is whether development teams are ready to adapt their habits and explore new workflows to boost productivity.

Next Steps

This challenge has inspired me to refine and expand on this project. Future iterations will focus on harnessing CE.SDK’s unique features to push the boundaries of automated video generation.

Stay tuned for part two of this series—there’s much more to explore!

UPDATE: Read part two - a cookbook how to build your own short video creator!

Over 3,000 creative professionals gain early access to our new features, insights and updates—don’t miss out, and subscribe to our newsletter.

Javascript Video Editing: Ultimate Guide for Developers and PMs

Jan — Fri, 20 Dec 2024 14:06:55 GMT

When developing or integrating a JavaScript-based video editor for the web, you must consider a number of factors to ensure the solution is both efficient and robust. This post is the definitive guide for anyone embarking on such a project. It explores the key technologies involved, their strengths and weaknesses, and how different use cases influence the choice of tech stack. We’ll examine diverse use cases, from lightweight, browser-based editors for quick edits to more advanced tools requiring complex processing and rendering. We’ll then discuss how these scenarios drive the selection of features and technology.

Additionally, we will show you the best open-source solutions that can accelerate development. This technical analysis will help you make an informed “build vs. buy” decision, ensuring you select the right approach for your project. Throughout this post, we will use the example of the IMG.LY’s JavaScript video editor (see here for a demo), showing you how these considerations shaped its architecture and feature set. You will learn practical insights into the decision-making process behind a successful web-based video editor.

Modern web technologies like WebGL, WebCodecs and WebAssembly have enabled browsers to efficiently perform resource-intensive tasks such as video editing that had hitherto been the sole domain of desktop applications. As a result Javascript based video editing tools such as veed.io have increased in popularity and users expects ever more sophisticated video editing capabilities inside modern video based web apps.

Video Editing Use Cases

Let’s first outline use cases and requirements of video editing applications on the web. These will inform our discussion of technical features of respective solutions below and grant a conceptual framework for evaluating potential benefits and drawbacks of each solution.

Simple Video Editing

Let’s start with the base case, simple video edits including trimming, cropping and resizing and simple effects such as adjusting brightness or saturation. This is usually sufficient feature set to support video editing for the following use cases:

Sales Outreach Videos: Sales teams often need to quickly edit customer-specific videos to personalize their outreach. This may involve trimming irrelevant portions, adding company logos, or adjusting brightness to ensure visual clarity.

Messaging Applications: These often require basic editing tools to allow users to crop, trim, or apply simple filters to shared videos, ensuring they’re concise and visually appealing.

Screencasting: Screencasting tools benefit from trimming and resizing capabilities to focus on key parts of recorded screens. Adding effects like brightness adjustment can make tutorials clearer and more professional.

CMS Systems: Content management systems may offer built-in video editing to help users optimize media assets for specific platform requirements, such as resizing for web embeds or adding subtle branding.

Screen Recording Applications: Screen recording applications often include simple editing options for cleaning up recorded content by trimming extraneous sections or cropping to highlight the most relevant parts.

Video Annotation

Next, users might want to add an additional layer of information to videos and overlay other assets, such as stickers, shapes, overlays, or text. Crucially these assets need to be time-based—that is, users need control over when the asset is shown so that particular video sequences can be referenced. This introduces the need for a timeline to arrange different video components relative to each other in time, as well as features like voice-over and audio track support.

E-commerce Reviews: Sellers and reviewers can annotate product demo videos with callouts, price tags, and feature highlights to make the content more engaging and informative.

Claims Management / Insurance: Insurance companies can use annotation to highlight key details in submitted video claims, such as timestamps of damage or explanatory text overlaying critical sections of footage.

Real Estate: Realtors might annotate property walkthroughs by adding labels, dimensions, or descriptive text overlays to highlight key features of the home or property.

Educational Applications: Instructors can use annotation tools to emphasize key moments in lectures or tutorials, such as overlaying text with formulas or concepts, or adding visual shapes to guide attention.

Productivity Tools: Users can annotate meeting recordings with timestamps, text notes, or overlay diagrams to summarize key decisions or action points.

Healthcare and Telemedicine: Medical professionals might annotate diagnostic videos or procedure recordings to explain findings or highlight areas of interest for training purposes.

Customer Support & Onboarding Tools: Companies can add annotations to video tutorials or troubleshooting guides to direct users through specific steps or highlight important information.

Video Composition

As we’re moving away from single media editing to creating video compositions of several types of media such as audio tracks, text, images, animations, and effects to create appealing visual designs, a well-designed timeline becomes even more important. Users need to manage many different video components in time, requiring user-friendly ways to browse and integrate external assets. This editor variant is most relevant for the following use cases:

Marketing Tech (Promotional Videos): Marketers can create visually stunning promotional videos by combining custom animations, background music, and text overlays for branding.

Social Media (Stories and Reels): Social media creators can quickly craft short, engaging videos that combine multiple assets like stickers, animations, and dynamic transitions tailored to platform-specific formats.

Event Highlight Reels: Event organizers can compile videos, photos, and music into cohesive highlight reels that encapsulate the essence of the occasion.

Template-based Video Creation

For many applications, users need starting points and examples for their designs. Starting from a blank canvas is rarely necessary for most use cases. Take a simple product video that includes promotional text, a brand logo, and animations. There is no need to reinvent the wheel for these types of videos. Instead, design applications should offer template libraries to accelerate user workflows.

Once we introduce workflows involving several stakeholders—such as designers setting up a certain brand framework and marketers working with and adapting these preconfigured designs—templates need to be able to constrain what edit operations adaptors can perform. These requirements are particularly important for these use cases:

Digital Asset Management: Organizations can manage and deploy branded video templates across teams, ensuring consistent design and messaging.

Social Media Publishing: Social media managers can utilize templates for quick turnaround on platform-specific video formats, such as Instagram Stories or LinkedIn posts.

Marketing Tech: Marketing teams can rely on pre-designed templates to churn out campaign videos at scale while maintaining brand consistency.

Training Videos: HR or L&D departments can adapt existing templates to quickly produce training materials customized for different teams or scenarios.

Creative Automation for Video

Finally, you can leverage data and automation to generate variations of your templates at scale, significantly boosting the productivity of use cases where users need to either test a large number of designs, publish to different channels with different requirements, or adapt designs to many slightly different instances, for example, menu designs for a franchise.

Marketing Tech: Marketers can generate hundreds of ad variations for A/B testing or localization by dynamically adjusting text, visuals, or calls to action.

Social Media Publishing: Social media teams can automate the creation of videos tailored to platform specifications, such as aspect ratios or resolution, while retaining core branding.

E-commerce: Retailers can produce personalized product showcase videos for different customer segments, featuring tailored offers, pricing, or recommendations.

Hospitality Industry: Hotels and restaurants can dynamically generate location-specific promotional videos showcasing seasonal offers, menus, or events.

Essential Video Editing Features

The above use cases are enabled by a set of features from simple transforms to complex compositions. Some video editors are focused more on manipulating individual video clips while others are more oriented towards video creation providing fully-fledged multi-media composition tool. This concise overview serves as a reference for evaluating video editing solutions:
Transforms

Cut, Trim, and Split: Basic editing tools for segmenting video clips.
Resize and Scale: Adjusts clip dimensions, ensuring proper fit or aspect ratio.
Crop and Rotate: Controls framing and orientation.
Zooming Capabilities: A UX feature enabling detailed editing and close-up views for precision.

Adjustments

Basic Adjustments: Controls brightness, contrast, and other visual enhancements.
Filters and Effects: Sets the visual appearance and atmosphere of videos.
Audio Tracks and Mixing: Supports multi-track audio adjustments and volume balancing.
Text and Overlay Options: Allows visual enhancements with text, captions, and emojis, providing opacity and layering options.

Composition

Multi-media Composition: Overlaying images, stickers, text as well as audio tracks are essential for creating videos through composition.
Multi-Track Editing: Elements and layers need to be position relative to each other in time allowing the creation of complex composition.
Canvas-Based Editing: Provides a flexible workspace for positioning and layering media elements.
Timeline Management: Split, join and arrange clips on a timeline. Manages the positioning of clips and timing for seamless transitions.
Animations: Make static elements dynamic and control their behavior in time. Can be use to create videos from scratch and enhance the storytelling workflow of existing ones.

The Technology Landscape

Some time in the early 2000s the web started to be viewed as a platform to run applications not just a distributed database to deliver static html documents. While Javascript development was not yet at a point to serve as foundation for the kind of application development we see today, Adobe Flash filled the role of development platform. Video editing in the early days of the web was mainly provided by Flash-based tools involving simple transform operations such as trimming and basic transitions. Java applets were another way to deliver processing intensive programs via the browser while executing them outside of the browser inside the JVM. These early attempts were either too clunky, since they could not run natively in the browser or lacked the performance to be useful for high quality editing.

When HTML5 came onto the scene in the early 2000s it became possible to handle multimedia content natively in the browser. Coupled with significant performance advances of Javascript and the introduction of WebRTC this opened the door for video editing on the web.

Some early cloud-based editors, such as WeVideo and Magisto, while more convenient and user-friendly than its predecessors still suffered from performance issues due to latency and lacked the depth of features found in desktop software.

In the past few years advances such as WebAssembly and Javascript libraries built upon it e.g. ffmpeg.js have revolutionised browser-based video editing. User can now perform complex editing tasks like multi-track timelines, video effects and real-time in-browser rendering. Modern web-based solution have even come to rival desktop apps, because they can take advantage of GPU acceleration, modern APIs like WebGL and UI frameworks such as React.

Let’s have a look at the state of the art web technologies that allow us to create performant video editing experiences inside the browsers. We are going to explore their relative strength, their complexities and conclude with when to best use each of these technologies:

Canvas API

The HTML5 Canvas API should be most familiar to most modern web developers, since it’s a familiar and popular way to render 2D graphics in the browser. It’s use for video editing is limited and restricted to simple overlays, trimming, composition and animation:

Complexity: Canvas is the least complex of the technologies introduced below and overlaps with the WebGL feature set to some extend, it’s API is well-known to most web developers.
Advantages: Canvas is lightweight, compatible across all browsers, and easy to implement, making it a great choice for simple use cases that require basic effects and no timeline.
Downsides: While well suited for simpler tasks Canvas lacks the GPU-acceleration of WebGL, hence it is not recommended for resource intensive operation or performance sensitive use cases.
When to Use: Choose Canvas for simple editing tasks, like trimming or basic overlays, or if you need a fallback technology that works well on legacy browsers and devices.

WebGL

WebGL opens up the power of GPU-accelerated graphics to the browser, making it the default choice for any use case requiring high performance rendering. As such modern Javascript video editing solution are well served by webGL when looking to render visual effects in real time, add transitions or facilitate multi-layer compositions by leveraging the GPU.

Complexity: WebGL comes with a steep learning curve especially if you are unfamiliar with 3D graphics processing, shaders and GPU programming.
Advantages: WebGL delivers high performance for processing complex video effects and can handle significant volume of data without latency. That makes it ideal for advanced, highly responsive video editing features.
Downsides: It can be challenging to engineer modular and maintainable WebGL code, debugging shaders and ensuring consistent performance across devices is an art in itself. WebGL might also be overpowered for simple editing use cases that do not justify the added complexity.
When to Use: Use WebGL if you’re creating a video editor with high-performance demands, particularly if real-time effects, transitions, or multi-layer compositing are needed.

WebCodecs API

Next up in our arsenal of formidable web technologies is a recent addition to modern browsers, WebCodecs, provide direct access to hardware accelerated video encoding and decoding.

The WebCodecs API is a recent addition to the browser, allowing direct access to hardware-accelerated video decoding and encoding. This API makes it easier to handle high-resolution video without delays, thanks to efficient decoding directly in the browser.

Complexity: WebCodecs is relatively easy to use for those familiar with media formats. However, you may need other technologies for rendering and complex editing.
Advantages: This API offers quick access to hardware acceleration, ideal for managing large video files. With it, you can efficiently handle high-definition video playback and streaming.
Downsides: Browser support for WebCodecs is still expanding, so compatibility might be a concern. Additionally, WebCodecs alone doesn’t provide a full editing suite; it needs to be combined with rendering and compositing technologies.
When to Use: WebCodecs is perfect if you need efficient decoding and encoding, especially for handling high-definition playback or live streaming.

WebAssembly (Wasm)

The next performance frontier browser pushed towards was enabling near-native performance by compiling languages like C++ or Rust into a format that efficiently runs in Javascript. This format is WebAssembly, which makes it possible to handle computationally expensive tasks like encoding, decoding or frame manipulation inside the browser.

Complexity: Working with WebAssembly is significantly more complex that the technologies discussed above. Developers need knoweldge of C++ or Rust as well as experience compiling to Wasm.
Advantages: WebAssembly enables browsers to handle complex video processing tasks, such as encoding and decoding, with near-native performance. Libraries such as FFmpeg build upon WebAssembly to make familiar video tools available to web developers.
Downsides: Development with Wasm is complex, and debugging can be a challenge. Additionally, Wasm modules may increase load times.
When to Use: Use WebAssembly when building native libraries for complex operation and expensive computations, such as encoding and decoding.

Our article on building a video editor with React and Wasm gives you a real-world starting point for your own apps.

MediaStream API

The MediaStream API is essential for gaining easy access to live video sources from within the browser, such as webcams. If your video editing app includes a feature for real-time recording or streaming the MediaStream API is an essential tool.

Complexity: MediaStream exposes a fairly simple API and is relatively easy to learn for experienced web developers.
Advantages: This API is well-suited for video feeds with low latency such as recording or streaming and does require addition encoding or decoding.
Downsides: Any use case requiring post-production video has to rely on additional technologies or frameworks, as the MediaStream API is limited to live feeds. The quality of the stream depends on the input source and can vary.
When to Use: Use MediaStream for applications with live recording or streaming functionality.

Web Audio API

The final web technology essential for video editing is the Web Audio API. Whether you want to add effects or mix multiple audio tracks, this API offers sophisticated tools for audio manipulation.

Complexity: For developers unfamiliar with audio processing the learning curve can be stepp, however the API is well designed and there is a host of educational resources.
Advantages: Web Audio API allows precise control over audio enabling your app to offer audio adjustments such as effects or reverb and multi-track editing.
Downsides: It can be challenging to accurately synchronize audio with video, especially if your app includes a real-time component.
When to Use: Opt for Web Audio API when audio editing is a core feature of your video editor.

Using These Technologies Together

Together, these technologies can be use to build a comprehensive browser based editor that is virtually indistinguishable from its desktop based counterparts. Different parts of the video editing workflow can be handled by each technology, while some choices have to made depending on the complexity and requirements of your projects, e.g. whether you’ll need resource intensive processing or an integration will real-time sources.

Video Decoding - WebCodecs API (for efficient video loading and playback).
Rendering - WebGL (for high-performance effects) and Canvas API (for simpler editing features).
Complex Processing - WebAssembly (for encoding/decoding and using native libraries like FFmpeg).
Audio Processing - Web Audio API (for mixing and adding effects to soundtracks).
Live Recording - MediaStream API (to capture live video from devices).

Open Source Javascript Video Editing Libraries

Fortunately in many cases someone has already done the heavy lifting and abstracted those browser APIs into simple to use libraries and frameworks. We’ll explore the most prominent ones distinguishing between projects that are for processing raw video and those that already provide an API for you to use. It is important to note that the libraries introduced below do not provide a video editing UI, unfortunately there are no open source video editors offering an end user interface that meet the standards for inclusion in a production grade application. None of the projects we evaluated met the demands we place on third-party libraries with respect to robustness, stability, maintenance and future development. However many can provide good starting points and references on how to build your own UI such as Reactive Video for React UIs.

FFmpeg.js

FFmpeg.js is the JavaScript port of the popular FFmpeg library compiled into WebAssembly. It enables developers to manipulate and process raw video files entirely in the browser. You can trim, convert formats, extract audio, and apply filters to videos without needing external software. However, FFmpeg just gives you the raw toolchain for building video editing features, you are still tasked with building the entire UX and editing workflows. We have an extensive guide on FFmpeg that you can consult to explore its capabilities and get started learning FFmpeg syntax.

Advantages:
- Most comprehensive library for web based video and audio processing.
- Fully client-side, no need for server-side processing.
- Highly flexible, allowing fine-grained control over video workflows.
Downsides:
- Fairly complex for beginners with a steep learning curve, requires familiarity with FFmpeg’s command-line syntax.
- FFmpeg is licensed under LGPL which makes it ill-suited for most commercial projects.
**When to Use:**FFmpeg.js is best suited for projects with custom requirements and UI making it necessary to exert fine-grained control over video processing tasks, especially for use cases like format conversion, advanced editing pipelines, or serverless video workflows.

The following two libraries allow you to programatically create and edit videos, we focus on the most stable, popular libraries here, while there might be promising up and comers we want to ensure a degree of dependability of the third-party code to be used in production systems.

Remotion

Remotion is a React-based framework that enables developers to edit or create videos programmatically in a declarative fashion using React components. React’s declarative syntax makes it comparatively simple to manage video content and Remotion provides a convenient UI for previewing and editing videos. While its studio UI cannot be embedded directly, Remotion can serve as the graphics processing engine for custom UIs. Although it is a commercial project, it offers a generous free tier for individuals and small businesses.

Advantages:
- Leverages React’s declarative model for seamless video creation.
- Provides robust video preview capabilities for iterative development.
- Flexible integration with custom-built UIs.
- Generous free tier for non-commercial and small-scale projects.
Downsides:
- The built-in studio UI cannot be embedded, requiring developers to create their own UI for embedding.
- React-specific, limiting use in non-React environments.
**When to Use:**Use Remotion when building React-based video applications that require programmatic video generation or dynamic video content. It’s ideal for teams already familiar with React and for applications where embedding is not a core requirement.

Etro.js

Similarly to Remotion, Etro.js is a JavaScript framework for programmatic video editing, it is framework agnostic and provides an API for simple composition of layers, effects and exporting videos.

Advantages:
- Includes support for advanced effects using GLSL shaders.
- Designed for in-browser video workflows.
Downsides:
- Minimal community support compared to larger projects.
- Requires additional effort to create a full UI.
When to Use: Suitable for applications where programmatic control and customization are key.

Extensibility

An important consideration when deciding to buy a prebuilt video editing solution is the degree of customization and extensibility required by your use case versus how critical speed to market is. While most SDKs provide enough flexibility by exposing cor API to build a solution that best suits your needs, this level of technical control comes at the expense of higher development and maintenance efforts. White-label solutions on the other end of the spectrum offer limited customization often relying on iFrames for embedding while providing maximum speed to market. If you are still in the phase where you need to make the business case for video editing features these might be key to allowing to quickly iterate. To learn more about weighing the pros and cons of SDK vs. White-label explore our extensive guide on the topic.

IMG.LY’s CE.SDK achieves extensibility by providing an interface for plugins that allow developers to add custom features and UI elements.

Cross Platform Video Editing

While this post is dedicated specifically to Javascript video editing an important consideration for choosing a video editing tech stack is whether there exists an intermediary output format that preserves editing operations so that videos can be edited on other devices and platforms. This is important for collaborative video editing use cases as well as use cases where videos are captured and basic edits performed on mobile and the finishing touches performed on the desktop.
These consideration are particular pertinent for the Real Estate, Telemedicine and Productivity use cases described above, since they involve either a multi-device or collaborative component.

At most common libraries or SDKs can achieve that by offering support for some form of serialization, that is exporting the edit operations along with the unedited video to then reapply edits upon import.

IMG.LY’s CE.SDK, on the other hand, is the only video editing SDK that is truly cross-platform. That means it is built atop a single creative engine that is portable to any platform. Whether iOS, Android, Desktop, or the Web, every platform uses the same underlying tech and uses the same custom format for scenes which contain all assets and edits making error-prone serializations obsolete.

Furthermore:

iOS and Android make use of the same underlying API.
Cross platform feature parity is baked into the cake. Since core functionality is implemented at the engine level, features are guaranteed to be available on both platform, although the timeline might differ somewhat.
Designs are 100% interoperable and consistent between platforms. Exporting and importing design files across platforms works seamlessly and final renderings are guaranteed to be consistent.

Javascript Video Editing & Generative AI

Generative AI is impacting video editing and generation across the board. Video generation obvious application of generative AI, but AI is also changing the way we we compose, create an edit videos in more subtle ways.

Features such as automated transitions, color grading, voice enhancement, voice over generation, background removal or inpainting are

Currently, these capabilities are offered by multiple specialized providers, such as ElevenLabs and OpenAI’s Sora. Firstly, that means modern Javascript video editors need to be open enough to serve as integrators for various, programmatically combining voice tracks, video clips, text, audio and other media before presenting users with an editing UI. Secondly, traditional video editing needs to enable the human in the loop. AI is still imperfect and manually refining unchangeable artifacts through prompts is inefficient and tedious. Instead the output by AI must remain editable and any editor solution smoothly interface with those artifacts to allow users to add the necessary polish, swap out media and regenernate individual parts of the scene.

To explore what this might look like in practice have a look at our CEO Eray Basar’s latest project building an AI video generator on top of CE.SDK.

Conclusion

When building or integrating a JavaScript video editor, developers and PMs have to navigate a complex technology landscape and make myriad decisions along the way. This guide has covered everything from essential use cases and feature sets to the cutting-edge technologies that underly browser-based video editing. Whether your project requires basic trimming tools or advanced multimedia composition, understanding the benefits and downsides of the various solutions like Canvas, WebGL, WebCodecs, WebAssembly, and more is crucial for choosing the right stack.

We’ve also explored how open-source tools like FFmpeg.js, Remotion, and Etro.js can accelerate development, while SDKs like IMG.LY’s CE.SDK offer unmatched extensibility and cross-platform compatibility. For those navigating the impact of generative AI on video editing, the need for seamless integration and user-centric design remains important.

If you’re looking for a robust, modern video editing solution for your web application, explore our showcase and start a free trial.

CreativeEditor SDK Now Supports React Native on iOS & Android

Jan — Thu, 19 Dec 2024 11:57:22 GMT

We’re thrilled to announce the release of our official @imgly/editor-react-native package, bringing the power of CreativeEditor SDK to the React Native ecosystem for iOS and Android!

CreativeEditor SDK enables developers to seamlessly integrate design, video, and photo editing features into their applications. Over the past few months, we’ve been focused on enhancing mobile editor solutions—refining features and expanding to new use cases, including the Video Editor.

To make CreativeEditor SDK accessible to even more developers, we’re expanding our support to include mobile cross-platform frameworks. We started with Flutter and are now adding React Native as another highly popular frameworks for building truly cross-platform apps.

Getting Started

Getting started with CreativeEditor SDK in React Native is simple! First, make sure you meet the following minimum requirements:

React Native: 0.73+
iOS: 16+
Swift: 5.10 (Xcode 15.4)
Android: 7+

Then, add the @imgly/editor-react-native package to your project using your favorite package manager.

To launch the editor, you’ll need an instance of EditorSettings - you can use EditorSettingsModel for convenience. All you need to get started is a license key, which you can obtain from your IMG.LY dashboard.

Here’s an example of opening the Design Editor:

import IMGLYEditor, { EditorSettingsModel } from '@imgly/editor-react-native';

export const openEditor = async (): Promise<void> => {
  const settings = new EditorSettingsModel({ license: 'YOUR_LICENSE_KEY' });
  const result = await IMGLYEditor.openEditor(settings);
};

Editor UI Presets

The React Native plugin builds on our native editor solutions, allowing you to leverage the existing editor UI variants. At launch, you can choose from:

Simply choose the editor solution that best suits your use case. By default, the Design Editor opens, but switching to other solutions is straightforward.

Android and iOS Setup

To setup our module in your application, you’ll need to add the expo-build-properties dependency which handles the necessary adjustments to your native folders.

Android

For Android, you’ll need to make a few adjustments to your app.json:

Set the minSdkVersion to 24 (Android 7) or later.
Add the IMG.LY Maven repository.
Ensure your Kotlin version is compatible (e.g., 1.9.10).

Here’s an example configuration:

{
  "expo": {
    "android": {
      "minSdkVersion": 24,
      "extraMavenRepos": ["<https://artifactory.img.ly/artifactory/maven>"],
      "kotlinVersion": "1.9.10"
    }
  }
}

iOS

On iOS, ensure your deployment target is set to 16.0. You can configure this in your app.json file:

{
  "expo": {
    "ios": {
      "deploymentTarget": "16.0"
    }
  }
}

Check out the Getting Started Guides or the examples on GitHub for a complete reference.

The Only Truly Cross-Platform SDK

If you’re working with React Native, you’re likely focused on faster time to market, reduced development complexity, and feature consistency across platforms. The CreativeEditor SDK shares these goals, using a single, unified creative engine that ensures:

Consistent APIs: iOS and Android share the same underlying API.
Feature Parity: Core functionality is implemented at the engine level, guaranteeing feature availability across platforms.
Interoperable Designs: Export and import designs seamlessly across platforms with consistent rendering.

Fully Customizable UI

The React Native plugin offers flexibility for theming, styling, and integrating callbacks into your editor’s workflow. Soon, we’ll introduce more advanced customization options, such as enabling or disabling features by default.

Refer to our documentation for detailed instructions on customizing the editor to match your branding and use case.

What’s Next?

To showcase the capabilities of our React Native plugin, we’ve prepared a dedicated example application available on GitHub. You can explore it to see the editor in action.

Additionally, our documentation provides all the details needed to get started, and you can visit the @imgly/editor-react-native npm page to begin your integration.

Remember to grab your license from your dashboard after starting a free trial, and let the CreativeEditor SDK unlock your users’ creative potential.

Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.40 Release Notes

Neslihan — Wed, 18 Dec 2024 11:41:58 GMT

This release is all about giving you more creative freedom and control. Whether you’re integrating CE.SDK into a React Native app, designing text with rich editing options, or customizing editor icons to match your brand, these updates empower you to craft more engaging and personalized user experiences.

Dive into the details below to explore how these features can elevate your projects!

Integrate CE.SDK in Your React Native App

Easily integrate CE.SDK into your React Native and Expo applications.

This plug-and-play solution simplifies integration, making it faster and easier to implement CE.SDK in your apps. Our React Native and Expo Support is built on top of our native editor solutions, enabling you to use our existing iOS and Android editor UI variants supporting a broad range of use cases.

At launch, you can choose between:

Additionally to our existing editor variants, you can create your own editor variant to suit your particular use case using our native API.

Get started with our React Native documentation.

Design with Multiple Text Styles in a Single Block

Empower your users to create visually dynamic text designs with Rich Text Editing: This feature allows multiple fonts, sizes, and styles within a single text block, unlocking endless customization possibilities.

From eye-catching headlines to seamless body text, your users can mix and match styles without needing multiple text blocks.

This enhancement simplifies design workflows, enabling users to achieve professional-quality results while streamlining their experience within your platform.

Customize Your Editor Icons

You can now easily customize your CE.SDK editor by adding your own icons. You can upload icons through our API to match your brand or design style. Whether you want to replace default icons or add custom ones to your features, this update gives you full creative control.

Get started with our documentation.

Upcoming

P3 Color Support on iOS and Android
Deliver stunning visuals with P3 Support on iOS. Wide-gamut P3 color space enables richer, more vibrant colors for professional-grade designs, ensuring high-quality results on supported devices.

Customize the Mobile Dock on iOS & Android
Streamline the user experience with a configurable Mobile Dock! This enhancement empowers clients to tailor key UI components. This improvement saves development time and opens the door for a more tailored and efficient creative experience, meeting the growing demand for customization in mobile apps.

View the v1.40 Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK – IMG.LY Blog

Starter Kits: A New Way to Use and Configure Our Editor

What changed

What are the benefits?

The full picture in one place

More control and capability over your editor

Opt in, never opt out

Two paths, your choice

The engine you rely on hasn’t changed

The Tradeoff

Getting Started

CE.SDK v1.69 Release Notes

Introducing IMG.LY Agent Skills for CE.SDK

Launch in Minutes with Production-Ready Starter Kits for Web

Let Users Bring Their PPTX & Canva Designs to CE.SDK

Professional-Grade Video Editing Across Platforms

Full Changelog

Building a CapCut-Like Video Editor with CE.SDK and AI

Build this with AI in minutes

Introduction

Architecture Overview

Step 1: Project Setup

Why so many packages?

Step 2: The React Component

Step 3: Creating the Video Scene

Browser support

Step 4: The CapCut Dark Theme

Setting the base theme

Custom CSS overrides

Responsive scale

Step 5: AI Plugin Integration

The unified AiApps approach

Provider categories explained

The proxy server requirement

Background removal

Step 6: UI Customization with the Component Order API

Adding the AI button to the dock

AI options in the canvas context menu

Custom export button in the navigation bar

Step 7: Wiring Generated Audio into the Asset Library

Step 8: Custom Labels with i18n

The Complete File Structure

What We Get Out of the Box

Going to Production

1. License key

2. Proxy server

3. Disable dry run

Key Takeaways

Resources

Introducing IMG.LY Agent Skills

The “Quick Start” Paradox

The Solution: IMG.LY Agent Skills for Web

How It Works: Your Autonomous Implementation Partner

1. The Explain Path

2. The Build Path

3. The Docs Path

The Shift to Autonomous Engineering

Supported Frameworks

Get Started

From Prompt to Editor: Running CE.SDK Inside ChatGPT with the Apps SDK

CE.SDK as a ChatGPT App

Working with MCP in Practice

Why This Pattern Matters

Beyond OpenAI: The MCP UI Standard

What is Visual Prompting?

A New Paradigm for Creative AI, Built by IMG.LY

Problem: the Chat Disconnect

What Is Visual Prompting?

How Visual Prompting Works in CE.SDK

Who Is Visual Prompting For?

Built for This: Multimodal Models and CE.SDK’s Plugin System

Explore It Yourself

Build vs. Buy: Is Fabric.js Right for You

Why Teams Switch from Fabric.js

1. Time-to-Market vs. Reinventing the Wheel

2. Lack of Advanced Features

Real-World Use Cases & Vertical-Specific Challenges

Technical Debt: The Hidden Cost of Fabric.js

Aging Codebase and Patchy Maintenance

Slow Issue Resolution