Learning – IMG.LY Blog

How to Build a Short Video Generator Using CE.SDK

Eray — Tue, 04 Mar 2025 12:57:47 GMT

In the following, I’m presenting a simple cookbook for building an AI-based video generator app, as described in my previous blog post. We’re using a combination of different APIs to generate text, audio, and images and will compose & render the final video using the headless CreativeEditor SDK. We also call it the Creative Engine.

This cookbook showcases the powerful capabilities of our client-side Creative Engine. The engine enables real-time video generation directly in the browser, eliminating the need for server-side processing. What sets this approach apart is its ability to produce editable source files that can then be opened with CreativeEditor SDK.

This approach is giving users complete control over every aspect of your video–from text and images to animations and overall composition. This means your users can refine and perfect your content even after the initial generation.

Get the complete code on GitHub.

Scope

This tutorial focuses on building an app with a simple UX:

Input your keywords/topics
Choose between landscape or portrait format
Generate and preview your video
Edit the video in the CE.SDK video editor

The app flow we will create:

The post-editing we will get with CE.SDK:

Technical Overview

The app follows three major steps to generate the video.

A script is generated based on User input, the output is a structured XML file.
The XML script is parsed to extract text and image information. The extracted data will then be used to generate audio & image files through third-party APIs
All assets are loaded into the creative engine. This is where the composition, including animation and effects, takes place. The Creative Engine then exports a video and scene file, which can be edited with the Creative Editor.

Setup

We’ll use a boilerplate with Next.js, React, Typescript & Tailwind. Make sure you retrieve all necessary keys:

Anthropic (LLM)
ElevenLabs (text to speech)
fal.ai (text to image)
IMG.LY CE.SDK – Retrieve a free trial key

// Required environment variables
NEXT_PUBLIC_ANTHROPIC_API_KEY = your_claude_api_key;
NEXT_PUBLIC_FAL_API_KEY = your_fal_ai_key;
NEXT_PUBLIC_ELEVEN_LABS_KEY = your_eleven_labs_key;
NEXT_PUBLIC_IMG_LY_KEY = your_img_ly_key;

Implementation

1. Generate The Script

In this step, we’ll focus on generating the initial prompt and then passing it to the Anthropic API.

As with many things with LLM, there are many different strategies for structuring the initial prompt. From experience, the best result comes from providing examples of the desired output. We’ve decided to use an XML document; this can be easily parsed later on and is less error-prone compared to a JSON.

We now define the structure of how information should be saved in the XML.

<video>
  <group part="intro">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.2">
        Did you know these fascinating facts about pyramids?
      </text>
      <image>Ancient Egyptian pyramid at sunset</image>
    </element>
  </group>
  <group part="content">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.2">
        The Great Pyramid was the tallest structure for over 3,800 years!
      </text>
      <image>Great Pyramid comparison to modern buildings</image>
    </element>
  </group>
  <group part="outro">
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.4">
        The pyramids continue to reveal their secrets to this day...
      </text>
      <image>A giant 3D question mark hovering over the pyramids</image>
    </element>
    <element>
      <text voiceId="50YSQEDPA2vlOxhCseP4" style="0.4">
        Stay curious - there's always more to discover!
      </text>
      <image>Pyramids under starry night sky</image>
    </element>
  </group>
</video>

In this tutorial, we’ll focus on the format trivia only as shown in this example. For later iterations, however, I’m planning to implement different content formats (e.g., trivia, quiz, recipe, etc.). Each of these formats will have its example XML. Therefore, I’m nesting the XML in a simple format object to scale this up easily later.

interface Format {
  name: string;
  example: string;
}

const formats: Record<string, Format> = {
  trivia: {
    name: 'Trivia',
    example: `<video><group>...</group></video>` // Add example from above
  }
};

Using this format object with the example, we can now generate the prompt.

What do we need for this prompt?

Description of the task
Description of the desired output, incl. an example for the specified format
Topic as provided by the user

The topic provided by the user is passed to the function as a string.

export const createVideoScriptPrompt = (
  topic: string,
  formatName: string = 'trivia'
) => {
  const format = formats[formatName];
  if (!format) throw new Error(`Format ${formatName} not found`);

  return `
Format: ${format.name}
Topic: ${topic}

Please write a detailed script for this short video, considering the specified format and topic.
Include an introduction, main content sections, and an outro. Each section should have an image.
Structure the script as an XML Document with clear sections, descriptions for the images.
The image description should be written as a prompt. This prompt will be used to generate an image.
Put the description between the image tags. The video shouldn't be longer than 30 seconds.

Example format:
${format.example}`;
};

2. Generate All Assets

In the second step, we’ll parse through the LLM response, which should be the XML. We’ll create a simple parsing function to extract all text information that should be sent to text-to-speech and text-to-image AIs.

Please note that all these steps can be easily streamlined by using AI-assisted coding. Just provide the example XML as input and your desired output.

API Calls
When finding text & image tags in the XML, we’ll call API functions for text-to-speech and text-to-image. For this example, I’m using ElevenLabs & fal APIs. You will find all API calls in the api.ts.

Since the LLM generated a script that includes image prompts, make sure to pass them to the API.

export async function generateImage(prompt: string): Promise<string | null> {
  try {
    console.log('Generating image for prompt:', prompt);
    const result = await fal.subscribe('fal-ai/flux/dev', {
      input: {
        prompt: prompt,
        size: 'portrait_16_9',
      },
    });
    const typedResult = result as { images: { url: string }[] };
    console.log('Image generation successful. URL:', typedResult.images[0].url);
    return typedResult.images[0].url;
  } catch (error) {
    console.error('Error generating image:', error);
    return null;
  }
}

Timestamps
Last but not least, we need to come up with timestamps. What’s the duration of each segment? This is critical information for composing the video. Luckily, this is quite easy: Each scene is as long as the generated audio for each segment. This duration for the audio segments can be calculated: Most TTS like ElevenLabs provide timestamps along the audio file. These are typically character-based timestamps, so we first have to calculate the timestamps for each word and then the duration for the entire text section.

Ready For The Next Steps
All Asset URLs that are generated will be saved in a VideoBlock object for convenience. The duration of the VideoBlock is the duration of the audio, as calculated above.

interface VideoBlock {
  text: string;
  imageUrl: string | null;
  audioUrl: string | null;
  startTime: number;
  duration: number;
  wordTimestamps: Array<{ word: string; start: number; duration: number }>;
}

3. Generate The Video

We have everything together now: The completed XML with timestamps, duration, and all assets. It’s now time to generate the video using the creative engine.

Let’s first add an empty container in our HTML that will be referenced for initiating the creative engine.

{
  /* Add container for Creative Engine */
}
<div id="cesdk_container" className="invisible mt-8 rounded-lg bg-gray-100" />;

We can now initialize the engine. Use this code snippet from our documentation.

We’ll then set up a function that creates a simple composition using the provided VideoBlocks. The engine requires you to first create a scene, append a page to the scene, and then create tracks within the page. The tracks are basically the layers in the timeline. I recommend setting one track as a background track using the following snippet:

// Set video track as a background track by connecting the page duration to the video track
engine.block.setAlwaysOnBottom(videotrack, true);
engine.block.setPageDurationSource(page, videotrack);

The Creative Engine provides powerful API calls to style & manipulate blocks in many ways. Here is an example of how we can animate the images with a slow zoom effect:

const imageZoomAnimation = engine.block.createAnimation('crop_zoom');
engine.block.setInAnimation(image, imageZoomAnimation);
engine.block.setDuration(imageZoomAnimation, block.duration);
engine.block.setBool(imageZoomAnimation, 'animation/crop_zoom/fade', false);

Export The Video & Scene
Exporting the video is easy. Just pass the page to the export function. In our example, we’re also saving the scene file so we can edit the video later.

// Export video
const progressCallback = (
  renderedFrames: number,
  encodedFrames: number,
  totalFrames: number
) => {
  console.log(`Progress: ${Math.round((encodedFrames / totalFrames) * 100)}%`);
};

const blob = await engine.block.exportVideo(
  page,
  'video/mp4',
  progressCallback,
  {}
);

// Save scene to string
const sceneData = await engine.scene.saveToString();

// Create scene blob
const sceneBlob = new Blob([sceneData], {
  type: 'text/plain',
});

4. Add A Video Editor

The last step is to add the video editor for post editing and pass the scene file. With CE.SDK, this effort is reduced to adding a few lines of code. In the init function, we’re configuring the editor and adding callbacks for the export:

const initEditor = async () => {
        const config = {
          license: 'A-O53TWXK5bfyconUx7e53S5YU7DzjuGpMAH5vvKjLd0zBa6IhsoF7zChy1uCVbj',
          userId: 'guides-user',
          theme: 'dark',
          baseURL: '<https://cdn.img.ly/packages/imgly/cesdk-js/1.44.0/assets>',
          role: 'Creator',
          ui: {
            elements: {
              view: 'default',
              panels: {
              },
              navigation: {
                position: 'top',
                action: {
                  save: true,
                  load: true,
                  close: true,
                  download: true,
                  export: true
                }
              },
              dock: {
                iconSize: 'normal', // 'large' or 'normal'
                hideLabels: true // false or true
              }
            }
          },
          callbacks: {
            onUpload: 'local',
            onSave: (scene: string) => {
              const element = document.createElement('a')
              const base64Data = btoa(unescape(encodeURIComponent(scene)))
              element.setAttribute(
                'href',
                `data:application/octet-stream;base64,${base64Data}`
              )
              element.setAttribute(
                'download',
                `video-${new Date().toISOString()}.scene`
              )
              element.style.display = 'none'
              document.body.appendChild(element)
              element.click()
              document.body.removeChild(element)
            },
            onClose: () => {
              onClose();
            },
            onLoad: 'upload',
            onDownload: 'download',
            onExport: 'download'
          }
        }

Conclusion

By following this cookbook, you can streamline the process of AI-generated video creation, making it fast and efficient. This method is especially useful for content creators, educators, and marketers looking to automate video production while maintaining creative control.
Next, try experimenting with video styles, refining AI scripts, or exploring advanced editing.
Feel free to GitHub repo and share your creations with us on X. Happy creating!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.44 Release Notes

Neslihan — Thu, 06 Feb 2025 07:31:00 GMT

Welcome to CE.SDK v1.44! This release brings a range of updates designed to enhance your editing workflow across platforms. Enjoy more vibrant and true-to-life visuals with expanded P3 color support on iOS and Android. On the web, streamline your editing process with the new ability to add multiple clips per track, making timeline management more efficient.

Let’s dive into the updates!

Simplify Your Video Editing Workflow

Add multiple clips to a single track on your video timeline! With CE.SDK video editing for the web, you can insert video clips, text, overlays, and more on one track—resulting in a more organized and efficient workspace.

This update empowers you to create richer, more dynamic sequences while keeping your timeline neat and easy to navigate.

Maximize Efficiency
Simplicity and speed are key. By reducing the need for multiple tracks, this feature helps users to work smarter and faster—allowing them to focus on creative storytelling and deliver engaging content.

Get started with our Video Editing for Web documentation.

Get Vibrant Colors with P3/Wide Gamut Support

Unlock a broader color range with P3 color space support on iOS and Android devices. This enhancement ensures your designs and content are more vibrant, accurate, and true to life—giving you richer color options for all your projects.

Set up color spaces with our documentation for iOS and Android.

💡 The Canvas menu appears when you select a design block, letting you edit its settings, like duplicating a selected image.

Building on mobile configuration introduced in our last release, we’ve extended customization to the editor Canvas Menu for Android. This update gives you full control over how the menu and its items appear and behave in the editor:

Customization: Configure visibility, transition effects, and decoration. Adjust the style of the divider between buttons.
Item List Control: Modify, add, reorder, or hide menu items.
Flexible Button and Divider Configuration: Adjust button appearance, functionality, and divider styles to match your design requirements.

More customizations will be available soon, including for iOS.
Explore the documentation.

Improvements

Switch the Editor Language without Restarting

Easily switch languages in our web editor without restarting it. This update lets you update the UI language instantly via API, streamlining localization. Check our documentation, and translate right away!

Upcoming

Record Reactions to Videos on Android
We’re excited to announce that Reactions will soon be available on Android. Already live in CE.SDK on iOS, this feature—popularized by platforms like TikTok and Instagram—allows users to capture and share their real-time responses to videos. Enhance engagement and provide content creators with immediate, interactive feedback, making your video content even more dynamic.

View the Changelog. Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

How I Built a Short Video Generator with AI & CE.SDK in One Day

Eray — Thu, 09 Jan 2025 11:27:13 GMT

Here’s the crux of product development in the age of LLMs: how much can AI truly accelerate the development process?

We have seen videos of solo developers building small apps entirely with AI with just a few prompts. But how does it scale to more complex development projects? As LLMs rapidly evolve, their scope and impact will only increase.

That’s why I regularly challenge myself to build a small project with the help of AI. I’m a prime candidate to test the AI productivity boost: a jack-of-all-trades (and a master of none) with a background in both design and engineering, yet no hands-on experience in the past five years. My latest challenge? Build a web-based short video generator within one day.

In this post, I’ll share the most intriguing takeaways from tackling this project.

Why a Short Video Generator?

Why focus on this idea? It’s simple: to ride the wave of a new trend. A format called “faceless” short videos is gaining traction among creators on platforms like YouTube and TikTok.

https://www.youtube.com/embed/DfQ3fhqfKVc?feature=oembed

What’s fascinating about these videos is their automation: an LLM generates a script, which is then transformed into speech, images, and text assets using various AI services. These assets are automatically assembled into a cohesive video.

The general concept is compelling: It’s still generative content, but mixed with classic video composition techniques. This approach offers greater accuracy, consistency, and control over pure generative AI.

The potential to automate video production at this scale is exciting. Add its relatively low complexity and high production value, and it became the perfect topic for my challenge.

Enter CE.SDK

Another reason I chose this challenge was its compatibility with CE.SDK, our design and video editor library. CE.SDK offers a robust editing toolkit that integrates into any product with just a few lines of code. Its features, like headless mode, are ideal for automating workflows like video generation.

Most faceless video services use React-based video generation and achieve fair results. However, using CE.SDK instead of a react-based library could potentially boost the overall experience with three critical improvements:

Editable Outputs: This is huge. Full automation often needs human adjustments for fine-tuning. CE.SDK enables automated video generation while allowing manual refinement of the results.
Enhanced Visual Quality: CE.SDK has its own rendering pipeline, allowing for more nuanced visual effects and animations. When you’re competing against others in this space, it can make a huge difference if you’re able to produce higher fidelity in the visual output.
Visual Design Workflow: Create design components or even entire templates visually, and then use them via code. This authoring workflow can be extremely helpful in creating rich, interesting designs for the generated videos.

The Ground Rules

To keep the challenge focused, I set strict rules:

Time Limit: Spend no more than 12 hours on the challenge.
No Manual Coding: Avoid writing any code yourself—everything should be built through conversations with AI.
Trust the AI: Do not read or analyze code generated by the AI. Rely entirely on its decisions.
Skip External Research: Do not read or explore the APIs you intend to use. Instead, provide links to the AI and let it determine how to use them.
Compare AI Performance: Alternate Claude Sonnet 3.5 and ChatGPT o1 for code generation to evaluate which performs better.

The Tools & Workflow

Code Editor: Cursor
Built on VSCode’s foundation, Cursor stood out as the only editor offering both an integrated chat interface and the ability to switch between different LLMs. However, with GitHub’s recent significant updates to Copilot, I’ll switch to VSCode with Copilot for future challenges.

UI Prototyping: Claude Artifacts
Rather than building the entire project in my code editor, I chose to prototype the UI directly through Claude’s web interface. The benefits were immense:

Instant results: To create an artifact, Claude streamlines development by automatically writing and compiling code while leveraging essential UI libraries and components. This automation eliminates setup time and technical overhead, allowing me to focus purely on design iterations.
Instants Variations: Claude enables rapid prototyping through parallel conversations. When a design direction didn’t quite work, I could simply start a fresh conversation with modified requirements and evaluate a new prototype. This approach helped me develop three viable concepts quickly - a pace that would have been impossible in a traditional code editor.
Quality of execution: Claude transforms rough concepts into polished, intuitive interfaces. Its suggestions often surpassed my initial ideas, offering sophisticated solutions I hadn’t considered.
Keep it clean: By prototyping outside the code editor, I kept the main project’s codebase clean and focused. This separation prevented the accumulation of experimental code and maintained the clarity of our primary development environment.

Quickly prototype your interface with Claude Artifacts.

APIs
Key APIs used in the project included:

Script Generation: Claude Sonnet 3.5 vs various ChatGPT models.
Image/Video Assets: Fal.ai Flux models.
Speech Synthesis: ElevenLabs.

Building the App: Divide and Conquer

After having prototyped the UI, I started to chat with the LLM inside the code editor so that it can code the app. To work with the AI efficiently, I followed a divide-and-conquer approach. Rather than simply asking it to “build me a video app,” I broke down the problem into manageable steps:

Generate a video script
Create an AI prompt that includes user input and examples of the desired output format. Pass this prompt to the LLM API.
Parse the script to generate assets (speech, images, text)
Parse the LLM’s response to extract image prompts and speech paragraphs. Send these to their respective APIs.
Compose the final video
Load all the generated assets into a predefined template to generate the finished video through the CE.SDK library.

After completing these steps, I was finally able to generate my first fully automated videos! With a few more tweaks and additions, I had an MVP ready within twelve hours.

The final result: A Short Video Generator

There are still some missing features, partly because I spent a significant amount of time refining the prompt to generate the video script. I also had to bend the rules occasionally—sometimes the LLM would hit a wall, and I had to read or write small snippets of code.

Key Takeaways

Engineering Knowledge Is Essential
You should have some engineering background to achieve the AI productivity boost in development.

AI doesn’t solve everything for you. You are still the architect. You provide a lot of input and guidance. AI often needs to be pointed to the right strategy. Foundational knowledge of computer science is hugely advantageous for working with AI effectively.
As mentioned, I had to read and write a few lines of code myself. Without coding experience, I would have probably not been able to progress, as the LLM was not able to.
The getting started experience is nowhere close to novice-friendly. How do you get started with a new project in a code editor that actually requires you to do the setup manually? My workaround was to create an empty project, and then ask the LLM to instruct me to use a boilerplate for react. Again, this is engineering knowledge, any novice would have hit a wall already at this point.

Claude Outperformed ChatGPT
Claude was a clear winner in the side-by-side comparison, because of three reasons:

Claude Artifacts was a game changer for UI prototyping.
It was generally better at writing and understanding code. Difficult to quantify, but in some cases Claude fixed the mess ChatGPT left in the code
Claude can process URLs, which makes working with APIs much smoother.

Who would have thought new LLMs would catch up to OpenAI so quickly after they released the first version of ChatGPT?

Complexity Slows AI
The more code in my project, the slower the overall progress. LLMs struggled with the growing complexity. Their context windows filled more quickly, and their responses became increasingly unreliable. At some point, it becomes extremely difficult to make architectural changes, especially if this affects multiple parts of the app. When trying to fix errors, you’ll often find yourself in a whack-a-mole game. While the AI would resolve one issue, it would inadvertently introduce new problems elsewhere, creating an endless loop of fixes and regressions.

Ultimately, the time invested in this challenge was well worth it. While LLMs can’t build products end to end on their own, they can significantly streamline product development when paired with the right human collaboration. The real question is whether development teams are ready to adapt their habits and explore new workflows to boost productivity.

Next Steps

This challenge has inspired me to refine and expand on this project. Future iterations will focus on harnessing CE.SDK’s unique features to push the boundaries of automated video generation.

Stay tuned for part two of this series—there’s much more to explore!

UPDATE: Read part two - a cookbook how to build your own short video creator!

Over 3,000 creative professionals gain early access to our new features, insights and updates—don’t miss out, and subscribe to our newsletter.

A Modern Vue.js Video Editor: Setup Guide

Antonello — Wed, 08 Jan 2025 11:24:43 GMT

Discover how to integrate IMG.LY’s video editor into your Vue.js application and unlock its full potential.

Platforms like Instagram, YouTube, and TikTok have revolutionized how we engage with content, setting new standards for consuming video media. With the growing popularity of short-form videos, this trend shows no signs of slowing down.

Users now not only love to watch videos but also expect intuitive video creation and editing features from modern web applications. Here is why adding a video editor to your Vue.js web application can significantly boost user engagement and satisfaction.

In this guide, you will discover how to integrate a video editor into a Vue.js project using CreativeEditor SDK (also referred to as CE.SDK). We will also discuss when this functionality is most beneficial and how to customize the editor for your specific needs.

For a quick setup, take a look at the GitHub Vue.js video editor integration repository to kick off your project.

Let’s dive in!

Why You Should Add a Video Editor to Your Vue.js App?

The digital landscape is constantly evolving, but one truth has remained steady: video content continues to dominate.

Statistics about videos are clear. According to Datareportal, over one-third of all Instagram reels are videos. Similarly, YouTube Shorts garners over 70 billion daily views, and TikTok boasts more than 1.5 billion monthly active users worldwide.

The reasons behind these staggering numbers are two:

Videos are incredibly engaging for audiences.
Social media platforms have made it remarkably easy to create professional-looking content.

Users now expect to be able to add filters, overlays, audio, music, and text with just a few clicks. As a result, not only do we anticipate video editing features in web applications, but we also demand that these features be simple to use.

By integrating an intuitive video editor, your Vue.js web app can capitalize on that trend. The goal is to provide a feature that billions of users are already familiar with—and likely expect from your application.

The payoff? Lowering the barriers to user-generated content, increasing engagement, and driving product distribution!

How to Integrate a Video Editor in Vue.js

In this section, you will learn how to set up a Vue.js video editor using CreativeEditor SDK.

Requirements

Before starting, ensure you have the latest LTS version of Node.js installed on your machine. Otherwise, download and install it from the official Node.js website.

If you do not already have a Vue.js project, create a new Vite-based Vue project by running the following command:

npm create vue@latest

This will install and run create-vue, the official Vue project scaffolding tool. During the process, you will be prompted to select optional features like TypeScript and testing support. Below is an example of how you can respond to the prompts:

√ Project name: ... my-vue-video-editor-app
√ Add TypeScript? ... No / Yes
√ Add JSX Support? ... No / Yes
√ Add Vue Router for Single Page Application development? ... No / Yes
√ Add Pinia for state management? ... No / Yes
√ Add Vitest for Unit Testing? ... No / Yes
√ Add an End-to-End Testing Solution? » No
√ Add ESLint for code quality? » Yes
√ Add Prettier for code formatting? ... No / Yes

If you are unsure about an option, simply select “No” by pressing Enter.

After the project is set up, navigate to the project folder and install CreativeEditor SDK via @cesdk/cesdk-js package with this command:

npm install @cesdk/cesdk-js

Your package.json file will now include the SDK as a dependency:

"dependencies": {
"@cesdk/cesdk-js": "^1.41.1",
"vue": "^3.5.13",
"vue-router": "^4.4.5"
}

Finally, open your project in a JavaScript IDE, and you are ready to start coding!

Create the Video Editor Component

In the src/components folder, create a new file named VideoEditor.vue. This file will serve as your Vue.js single-file component for the CE.SDK-based video editor.

In the like this:

import CreativeEditorSDK from '@cesdk/cesdk-js';

Next, create your video editor component using the following lines of code:


<!-- src/components/VideoEditor.vue -->

<template>
  <div id="cesdk_container" style="height: 100vh; width: 100vw"></div>
</template>

<script>
import CreativeEditorSDK from '@cesdk/cesdk-js'

export default {
  props: { config: Object },

  _cesdk: null,

  mounted: function mounted() {
    CreativeEditorSDK.create('#cesdk_container', this.config).then(async (instance) => {
      this._cesdk = instance
      // customize the editor behavior...
    })
  },

  methods: {},

  watch: {},

  beforeUnmount: function beforeDestroy() {
    // clean up the CE.SDK instance
    if (this._cesdk) {
      this._cesdk.dispose()
      this._cesdk = null
    }
  },
}
</script>

The component defined above embeds the CreativeEditorSDK JavaScript element into an HTML <div> container.

Use the Component

You can now import the VideoEditor component in the

import VideoEditor from './components/VideoEditor.vue';

Then, include it in the

A Modern React Video Editor: Integration Guide

Antonello — Wed, 08 Jan 2025 09:31:11 GMT

Learn how to integrate IMG.LY’s video editor for React into your web app and make the most of all its features.

Video content has seen steadily growing demand, especially with the rising popularity of short-form videos. It is no surprise that platforms like YouTube have heavily invested in short-form video content, as video remains a dominant medium for users to create and consume content.

Platforms like Instagram, TikTok, and YouTube have changed user habits forever. Here is why integrating video creation and editing directly within your web application can dramatically enhance user satisfaction.

In this guide, you will see how to integrate a video editor into a React app using CreativeEditor SDK (CE.SDK, for short). We will discuss when it is useful and explain how to customize the editor for your specific use cases.

For a quick start, check out the GitHub repository featuring the code for integrating a React video editor.

Let’s dive in!

Why Integrate a Video Editor in Your React Web App?

The digital landscape is ever-evolving, but one truth has remained constant for several years: video content reigns supreme.

The data speaks for itself. Platforms like TikTok boast over 1.04 billion monthly active users worldwide. Similarly, one of most recent Google’s earnings call revealed that YouTube Shorts generated approximately 70 billion daily views in just one quarter—a significant leap from 30 billion daily views just two years ago.

These platforms have set the gold standard by empowering users to create professional-looking videos effortlessly—adding filters, overlays, audio, and text with just a few clicks. This simplicity has made video editing an essential feature across various industries, from social media apps to marketing tools.

Then, the shift toward shorter, more engaging videos has transformed how users create content and how we consume it. Audiences now demand bite-sized, engaging clips that are easy to produce and share.

By integrating an easy-to-use video editor, your React web app can tap into this trend. The idea is to offer a feature that billions of users are familiar with—and may already expect from your application.

The result? Lowering the barrier for user-generated content, boosting engagement, and driving distribution!

React’s flexibility and component-based architecture make it the perfect match for IMG.LY’s CreativeEditor SDK, which guarantees a seamless, high-performance, and feature-rich experience for design, video, and photo editing—all directly in the browser.

Getting Started: Adding a Video Editor in React

Follow this step-by-step tutorial section and learn how to set up a React video editor in your web app using CreativeEditor SDK. For a different approach, see our guide on how to build a video editor in React with Wasm.

Integrate CE.SDK in React with the following instructions!

Requirements

Before getting started, make sure you have the latest LTS version of Node.js installed on your machine. Otherwise, download it from the official website and install it.

Next, if you do not already have a React project set up, create a new one using Vite:

npm create vite@latest my-react-video-editor-app – --template react

Install CE.SDK via the @cesdk/cesdk-js package with this command:

npm install @cesdk/cesdk-js

Your package.json file will now include it as a dependency:


"dependencies": {
  "@cesdk/cesdk-js": "^1.41.1",
  "react": "^18.3.1",
  "react-dom": "^18.3.1"
}

Load your project in a JavaScript IDE, as you are ready to start coding!

Create the Video Editor Component

In the src folder, create a VideoEditor.jsx file. This will contain your CE.SDK-based React video editor component.

Import CreativeEditorSDK from @cesdk/cesdk-js as below:

import CreativeEditorSDK from '@cesdk/cesdk-js';

Then, initialize your video editor React component with this logic:

// src/VideoEditor.jsx

import CreativeEditorSDK from '@cesdk/cesdk-js';
import { useEffect, useRef, useState } from 'react';

// your CE.SDK license and user configs
const config = {
  license: '<YOUR_LICENSE>',
  userId: '<YOUR_USER_ID>',
};

export default function VideoEditor() {
  const cesdk_container = useRef(null);
  const [cesdk, setCesdk] = useState(null);

  useEffect(() => {
    // if the CE.SDK container has already been initialized
    if (!cesdk_container.current) {
      return;
    }
    let cleanedUp = false;

    // initialize a CE.SDK instance
    let instance;
    CreativeEditorSDK.create(cesdk_container.current, config).then(
      async (_instance) => {
        instance = _instance;
        if (cleanedUp) {
          instance.dispose();
          return;
        }

        // customize the editor behavior...

        setCesdk(instance);
      }
    );

    // clean up the CE.SDK instance
    const cleanup = () => {
      cleanedUp = true;
      instance?.dispose();
      setCesdk(null);
    };

    return cleanup;
  }, [cesdk_container]);

  return (
    <div
      ref={cesdk_container}
      style={{ width: '100vw', height: '100vh' }}
    ></div>
  );
}

Replace <YOUR_LICENSE> with the license key you obtain from CreativeEditor SDK and <YOUR_USER_ID> with your user ID to optionally track monthly active users (MAUs) across different devices.

The component defined above loads the CreativeEditorSDK JavaScript element into an HTML

element.

Use the Component

You can now import the VideoEditor component as follows:

import VideoEditor from './VideoEditor';

Then, use it as below:

<VideoEditor />

This will open the video editor with the video preset, allowing users to trim, cut, apply filters, add text overlays, include music, and more to enhance their videos.

Have a look at our interactive demo to see what the resultant editor should look like.

React Video Editor Use Cases

Now that you know how to integrate a React video editor into your web app, you are ready to explore some key use cases. See how the specific features of CreativeEditor SDK can support different scenarios.

Social Media Publishing

CreativeEditor SDK equips users with the tools they need to create engaging video content. It makes it easier to produce shareable videos for popular social media platforms like Instagram, TikTok, and LinkedIn with features like:

Reusable Templates: Start with video templates downloaded from others or create your own templates and share them with the community. By selecting from a variety of ready-to-use templates for different use cases, users can avoid the blank canvas syndrome and bring their ideas to life quickly. You can explore a range of different templates in our video editor demo.
Music and Audio Library: Users can easily add audio files to their videos, setting the tone through music or narration and creating a captivating atmosphere for their content.
Animations: The SDK comes with a wide set of configurable animations to improve the appearance of design elements within video scenes, significantly boosting engagement.

Once a video has been created in your React application, CE.SDK lets users export it in the appropriate formats and aspect ratios for the target social media platforms.

Digital Asset Management

DAM, short for Digital Asset Management, refers to the process of organizing, storing, and managing any digital asset—including videos. Over time, DAM has become essential for businesses to maintain efficiency, streamline workflows, and ensure brand consistency across teams.

CreativeEditor SDK supports DAM by giving your team the ability to organize, edit, and repurpose video content within a centralized, embeddable system. Within the React video editor component, users can easily adapt and edit their assets, ensuring brand consistency while automatically propagating changes across multiple assets.

Video templates support placeholders and permissions to guide users’ designs, guaranteeing that all assets stay on-brand. While CE.SDK does not directly support role-based permissions, it can be integrated with any backend technology or framework to control user access.

Keep in mind that the SDK is also extendable. That way, developers can create custom hooks that check permissions and customize the editor’s behavior accordingly.

Marketing Automation

CreativeEditor SDK provides marketing tech to efficiently craft branded, campaign-specific videos at scale.

Features like pre-designed, editable, and customizable designs help save time in video production and management. Changes to these templates can be automatically propagated, also eliminating the need for manual updates.

The SDK also supports advanced scenarios like A/B testing, so that your marketing teams can campaign reach by comparing different solutions to see how they perform. Plus, the ability to generate customized videos with configurable text variables makes personalizing content easier than ever.

Personalized Videos for Sales Outreach

We all know how boring and annoying cold emails can be. Marketers and salespeople understand this too, which is why personalized video messages have become the new frontier for pitching.

Thanks to CreativeEditor SDK, sales teams can easily produce customized videos tailored to specific clients or leads. These videos are far more engaging than standard emails because they feel like a real conversation.

Also, the translation capabilities offered by the SDK allow sales teams to reach multiple audiences worldwide, breaking down language barriers and personalizing the message for a broader range of targets. Test this feature in our translation and internationalization demo.

E-Learning

CreativeEditor SDK provides educators with the tools they need to create informative videos. Teachers and coaches can quickly produce professional-quality content without requiring extensive technical skills.

The SDK enhances the learning experience on your React-based e-learning platform through features like:

Reusability: Build a library of templates that can be easily adapted and integrated into existing materials.
Easy video production: Record, edit, and arrange videos directly within the editor, accessible on desktop, mobile, or tablet.
Plugins for learner interactivity: Boost engagement by integrating plugins that add quizzes, polls, and other interactive components within the videos.

Customizing your CreativeEditor SDK Instance

CreativeEditor SDK does not offer a static, pre-configured React video editor experience. Quite the opposite, it presents several customization features to tailor the editor to your specific needs and brand requirements.

The key customization options are:

Build Custom UIs: Define fully custom user interfaces adapted to your specific use case.
Theming: Adjust the editor’s theme to match your app’s design. Choose from built-in themes, generate new ones using the theme generator, or manually create a custom theme. Test theming in our dedicated demo.
Media Libraries: Set up customizable, sortable media libraries. CE.SDK supports integration with third-party libraries accessible via API, such as Unsplash, allowing you to easily access and use external media assets.
Toolbar Customization: Reposition toolbar elements, change icons, or rename tools to suit your needs, providing a unique editing experience.
Internationalization: Adapt the video editor component to different languages and regions. CE.SDK offers full i18n support, enabling you to overwrite and extend all text strings in any language.

Conclusion

Integrating a React video editor into your web app can significantly enhance the user experience and serve as a valuable feature to boost retention, engagement, and potential product distribution. This holds true whether you are building a social media platform, a tool for influencers, a marketing SaaS, or an e-learning system.

With CreativeEditor SDK, you can quickly add a fully customizable video editing experience to your React application in just minutes.

By following the steps outlined in this blog post, you can bring professional-level video editing features to your web users. Explore CE.SDK’s video capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

A Modern Angular Video Editor: Setup Guide

Antonello — Wed, 08 Jan 2025 08:19:25 GMT

Learn how to integrate IMG.LY’s video editor for Angular into your web application and get the most out of it.

TikTok, Instagram, Facebook, WhatsApp Messenger, YouTube, and X are among the most popular social media platforms. They consistently rank in the top 10 most visited websites and downloaded mobile apps globally. Have you ever asked what is the common thread between them? Video content!

These platforms have revolutionized how we consume video content. Videos are now shorter, often include subtitles for silent viewing, and are designed to be engaging, professionally produced, and artfully crafted.The challenge lies in meeting user expectations, as modern web applications are now expected to offer intuitive video creation and editing tools. After all, Instagram and TikTok have set the bar quite high when it comes to seamless video production.

This guide will walk you through implementing a video editor in your Angular web app using CreativeEditor SDK (CE.SDK). We will also take a look at some scenarios where this video editing is useful and explain how to customize the editor to meet your unique requirements.To jumpstart your project, check out the GitHub repository for Angular video editor integration.

Let’s dive in!

Why Add a Video Editor to an Angular Application?

Over the past few months, AI has been revolutionizing the tech industry. It all started with AI-generated text and images, but what is next on the horizon? AI-generated videos!

The appeal is clear—numerous studies show that video is the most engaging medium, resonating deeply with us as humans. This is no surprise, given that YouTube and Instagram rank among the top five most visited websites, and TikTok is the most downloaded app worldwide.

The problem is that users now demand more than just video content. They expect engaging, professionally edited videos. On top of that, platforms like Instagram and TikTok have raised the bar for intuitive video production and editing.

As a result, we all expect to be able to add filters, overlays, audio, music, and text to videos with just a few clicks. Meeting these expectations is essential for any application offering video generation features.

Adding an easy-to-use video editor into your Angular project gives users a powerful tool that they are likely to desire and are already familiar with. That will surely boosts user engagement and help you accelerate the distribution of your online product or service!

How to Set Up a Video Editor in Angular

In this section, you will learn how to integrate an Angular video editor into your wev application through CreativeEditor SDK.

Requirements
Before you begin, make sure you have the latest LTS version of Node.js installed on your machine. If that is not your case, download and install it from the official Node.js website.

Next, open your terminal and run the following command to install the Angular CLI globally:

npm install -g @angular/cli

This will make it easy to create and manage Angular projects.

Once the installation is complete, if you don’t have an Angular project set up, initialize a new Angular project by running:

ng new my-angular-video-editor-app

You will be prompted with a few questions:

Would you like to share pseudonymous usage data about this project with the Angular Team at Google under Google's Privacy Policy at https://policies.google.com/privacy. For more details and how to change this setting, see https://angular.dev/cli/analytics.
No
Global setting: disabled
Local setting: No local workspace configuration file.
Effective status: disabled
✔ Which stylesheet format would you like to use? CSS [ https://developer.mozilla.org/docs/Web/CSS ]
✔ Do you want to enable Server-Side Rendering (SSR) and Static Site Generation (SSG/Prerendering)? No

If you are unsure, press Enter to accept the default answer.

After the project is created, navigate to the project folder and install the CreativeEditor SDK (@cesdk/cesdk-js) npm package by running:

npm install @cesdk/cesdk-js

Your package.json file will now include the SDK as a dependency:

"dependencies": {
  "@angular/animations": "^19.0.0",
  "@angular/common": "^19.0.0",
  "@angular/compiler": "^19.0.0",
  "@angular/core": "^19.0.0",
  "@angular/forms": "^19.0.0",
  "@angular/platform-browser": "^19.0.0",
  "@angular/platform-browser-dynamic": "^19.0.0",
  "@angular/router": "^19.0.0",
  "@cesdk/cesdk-js": "^1.41.1",
  "rxjs": "~7.8.0",
  "tslib": "^2.3.0",
  "zone.js": "~0.15.0"
}

Open your project in a JavaScript IDE, as you are ready to start coding!

Create a Video Editor Component

First, use the Angular CLI to generate the scaffold for your video editor component:

ng generate component video-editor

Open the video-editor.component.ts file inside the src/app/video-editor folder and initialize your CE.SDK-based Angular video editor component as below:


import { Component, ElementRef, ViewChild } from '@angular/core';
import CreativeEditorSDK, { Configuration } from '@cesdk/cesdk-js';

Component({
  selector: 'app-video-editor',
  imports: [],
  templateUrl: './video-editor.component.html',
  styleUrl: './video-editor.component.css'
})

export class VideoEditorComponent {
  ViewChild('cesdk_container') containerRef: ElementRef = {} as ElementRef;

  ngAfterViewInit(): void {
    const config: Configuration = {
      license: '<YOUR_LICENSE>',
      userId: '<YOUR_USER_ID>',
    };
    CreativeEditorSDK.create(this.containerRef.nativeElement, config).then(
      async (instance: any) => {
        // customize the editor behavior...
        await instance.createDesignScene();
      }
    );
  }
}

Replace <YOUR_LICENSE> with the license key provided by CreativeEditor SDK. Optionally, also set <YOUR_USER_ID> to your user ID to start collecting monthly active user (MAU) data.Then, define video-editor.component.html as below:

<div #cesdk_container [style.height.vh]="'100'" [style.width.vw]="'100'"></div>

This way, VideoEditorComponent will be mounted on the above

element.

Use the Component

You can now import the component and register it in your app.components.ts file with these lines of code:

import { Component } from '@angular/core';
import { RouterOutlet } from '@angular/router';
import { VideoEditorComponent } from './video-editor/video-editor.component'; // import the video editor component

@Component({
  selector: 'app-root',
  imports: [RouterOutlet, VideoEditorComponent], // register the video editor component
  templateUrl: './app.component.html',
  styleUrl: './app.component.css',
})
export class AppComponent {
  title = 'my-angular-video-editor-app';
}

Next, you can use it in app.component.html with:

<app-video-editor></app-video-editor>

This setup will open the video editor with a predefined video template, allowing users to trim, cut, apply filters, add text overlays, insert music, and make other video enhancements.

Have a look at our interactive demo to see what the resultant editor should look like.

Use Cases the Your Angular Video Editor

After learning how to create an Angular video editor component with a basic CreativeEditor SDK integration, it is time to explore some practical use cases in real-world scenarios.

Video Messages for Marketing Outreach

Cold emails often come across as generic and impersonal, often ending up in spam or being ignored. In contrast, personalized video messages are transforming marketing outreach.

CreativeEditor SDK enables marketers to create compelling videos that stand out from standard email approaches with capabilities like:

Personalized Video Messages for Leads: Create tailored videos for individual clients or prospects, adding a personal touch to your marketing outreach efforts and make a stronger impact compared to traditional cold emails.
Global Reach with Translation Features: Use the SDK’s built-in translation and internationalization tools to engage with a global audience. Customize messages to fit local cultures and ensure a personalized experience. Try out this feature in our translation demo.

Educational Videos for E-Learning Platforms

CreativeEditor SDK offers users in the e-learning industry an intuitive tool to create engaging and educational videos.

Educators, teachers, and coaches do not need technical skills, as video creation is simple and accessible. Specifically, users can record, edit, and organize videos across desktop, mobile, or tablet browsers with just a few clicks.

The SDK simplifies content creation through reusable templates. Instructors can integrate their materials into those template to maintain consistency while covering different topics. Finally, support for interactive plugins—such as quizzes, polls, and other engagement tools—adds a dynamic layer to video lessons.

CreativeEditor SDK simplifies the process of producing shareable videos for social media sites like Instagram, TikTok, YouTube, but also LinkedIn and others. That is possible thanks to features like:

Reusable Templates: Start with video templates from other creators or design your own and share them with the community. By choosing from a variety of pre-designed templates for different needs, users can avoid the frustration of starting from scratch and bring their ideas to life quickly.
Animations: Access a broad selection of customizable animations to enhance the visual appeal of video elements, driving higher engagement.
Audio Library: Effortlessly incorporate audio files into your videos, such as music, voiceovers, or sound effects, to set the mood and create an immersive experience for viewers.

Once a video is created in Angular application with CE.SDK, users can export it in the ideal formats and aspect ratios for different social media platforms.

Showcase Products on E-Commerce Applications

Images have long been a simple and effective way to showcase products on e-commerce platforms. Many sites, from eBay to Amazon, have used them for years. At the same time, customer expectations have evolved, and they now seek more engaging ways to visualize products.

Short, dynamic product videos significantly enhance product presentation. In particular, Amazon data shows that adding videos can boost sales by up to 9.7%. It is no surprise that many Amazon sellers have already adopted video to showcase their products.

With a CE.SDK-based Angular video editor, you can easily create videos that work seamlessly across multiple devices for a unified shopping experience. Thanks to reusable templates, you can customize product videos while maintaining scalability and brand consistency.By leveraging multiple templates and feeding them with diverse data, you can generate automated product video variations at scale.

Smart and Automated Workflows

CreativeEditor SDK supports the development of a tool for creating branded videos at scale.

As it is based on ready-made and editable templates, it simplifies video production and management. The reason is that users can define these templates once through an intuitive point-and-click interface, then effortlessly customize the produced videos via different input resources or configurable text variables.

That automated workflow process boosts productivity and scalability. The SDK also supports advanced features like A/B testing. That way, teams can create multiple versions of a campaign video, distribute them on social platforms, and compare their performance.

Customizing your CreativeEditor SDK Instance

CreativeEditor SDK features a versatile Angular video editor experience that can be easily configured and customized to fit your specific requirements and visual preferences.Key customization options supported by CE.SDK include:

Feature Activation: Enable or disable features as you need, so that you can control the functionality available to users.
Custom UIs: Create fully personalized user interfaces, adapting the editor to your unique use case. This includes repositioning toolbar elements, changing icons, or renaming tools for a tailored editing experience.
Internationalization: Adapt the video editor for different languages and regions. CE.SDK provides full support for internationalization, enabling you to customize all text strings in any language.
Export Options: Configure which file formats users can select for exporting, along with settings for quality, dimensions, and more.
Role Customization: CE.SDK supports roles, which are a set of global settings and scopes. When the role changes, the default settings for that role are applied. You can also use a callback to adjust configurations whenever the role changes and define custom roles to tailor settings for specific user needs or workflows.
Media Libraries: Create custom, sortable media libraries. CE.SDK supports integration with third-party libraries via API, offering easy access to external media assets.
Choose the Start Video: Set the editor to display a predefined video at the start, eliminating the blank canvas effect.
Theming: Adjust the editor’s theme to match your app’s design. You can select from built-in themes, generate new ones using the theme generator, or manually create a custom theme.

See all of these customizations in action in our customization demo.

Conclusion

Adding an video editor to your Angular site or web application greatly enhances user experience. Such a feature drives product success by improving user reviews and increasing retention. Whether you are creating a marketing tool, an e-learning or e-commerce platform, or any web app with social media functionality, a video editor adds immense value.

CreativeEditor SDK gives you what you need to implement a fully customizable video editing solution into your Angular application in minutes.

By following the steps in this guide, you learned how to equip your users with a professional and intuitive video editing experience. Explore CE.SDK’s video capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

A Modern Video Editor SDK for Your React Native App

Jan — Mon, 06 Jan 2025 10:40:35 GMT

Learn how to integrate IMG.LY’s video editor for React Native into your app and how to leverage its capabilities best.

Why Add a Video Editor to Your React Native App?

Video content is solidifying its status as the most engaging form of digital media. Platforms like TikTok, Instagram Reels, and YouTube Shorts have made video consumption and creation an engrained habit with billions of users. You can harness this trend by adding a video editor to your app significantly enhancing user engagement and retention.

React Native, with its ability to create cross-platform applications from a single codebase, is a perfect match for IMG.LY’s CreativeEditor SDK Video Editor. It ensures seamless performance on iOS and Android, powered by the same unified graphics engine across platforms.

Whether your app focuses on social media, marketing, or eCommerce, integrating a video editor empowers users with a creative tool set while elevating the overall experience.

Getting Started: Integrating the Video Editor in React Native

Requirements

Before diving into the integration, ensure your environment meets these requirements:

React Native: 0.73+
iOS: 16+
Swift: 5.10 (Xcode 15.4)
Android: 7+

To get started, add the @imgly/editor-react-native package to your project:

npm install @imgly/editor-react-native

Setting Up the Editor

Once the package is installed, initialize the editor by importing the necessary modules and creating an instance of EditorSettingsModel. You’ll need a license key, which you can obtain from the IMG.LY dashboard.

Here’s how to set up and launch the video editor:

import IMGLYEditor, { EditorSettingsModel } from '@imgly/editor-react-native';

export const openVideoEditor = async (): Promise<void> => {
  const settings = new EditorSettingsModel({
    license: 'YOUR_LICENSE_KEY',
  });

  const result = await IMGLYEditor.openEditor(settings);
};

This launches the editor with the Video Editor preset, enabling users to trim, cut, and enhance their videos with filters, text overlays, stickers, and music.

Use Cases: Building a TikTok-Like Experience

Now that the video editor is integrated into your React Native app, let’s explore some key use cases and how to configure the editor to support them.

Short-Form Video Creation

For apps targeting TikTok-style short-form video content, prioritize features that simplify the editing process:

Timeline Control: Allow users to trim clips and sync overlays like stickers or music.
Filters & Effects: Offer filters to let users add certain moods to their videos with themes like retro, high contrast, or pastel tones.
Text & Stickers: Add captions and playful stickers for videos that are often consumed on mute.
Music & Audio: Let users add soundtracks or sound effects, with a library of trending music for inspirational background music.
Video Templates: Provide ready-made templates that users can customize for specific occasions or themes.

This setup allows influencers and brands to create professional-looking videos that they can share across social media platforms with ease.

Influencers and Marketing

If your app serves influencers or businesses, the ability to create polished, on-brand videos is key. Features to consider:

Branded Templates: Pre-designed templates aligned with specific brand aesthetics, simplifying content creation for marketing campaigns.
Watermarks: Add brand logos or watermarks to videos, ensuring creators and brands maintain visibility.

These features empower influencers and businesses to quickly generate content that’s ready for distribution across platforms.

E-commerce and User-Generated Content

Video editing is a powerful tool for e-commerce apps, enabling sellers and customers to create engaging product demos, unboxing videos, reviews and tutorials. Features like trimming, filters, and music tools make it easy to produce compelling content that enhances the shopping experience.

For example:

Vendors can create promotional videos for products.
Customers can share authentic reviews or tutorials, increasing trust and boosting sales.

Customization Options with CreativeEditor SDK

The React Native plugin offers a range of customization options to adapt the editor to your app’s needs:

UI Customization: Change themes, colors, and layouts to match your app’s branding.
Video Presets: Configure presets for specific use cases, such as limiting video length or optimizing for a particular resolution.
Custom Assets: Add unique filters, stickers, and fonts to create a personalized experience for your audience.
Templates: Use the CreativeEditor SDK Web UI to create custom templates, enabling users to start with professional-grade designs.

Conclusion

Integrating a video editor into your React Native appwill improve your UX, and help boost engagement, retention, and potential distribution of your product, whether you’re building a social media platform, an influencer tool, or an e-commerce app.
With CreativeEditor SDK, you can create a TikTok-like video editing experience or offer specialized tools for businesses and creators.

By following the steps in this guide, you can empower your users to create professional-quality video content directly within your app. Explore the full capabilities of the SDK by visiting the React Native documentation and getting started today.

Stay tuned for updates, and don’t hesitate to reach out with any questions.

Thanks for reading!

3,000+ creative professionals gain early access to new features and updates—don’t miss out, and subscribe to our newsletter.

A Modern Video Editor SDK for your Flutter App

Jan — Fri, 13 Sep 2024 11:26:15 GMT

Video has been the only type of content with steadily growing demand, and it will remain a staple and an expected medium for users to create and consume.

With platforms such as TikTok, Instagram Reels, and YouTube Shorts dominating and shaping user habits, the ability to create and edit videos directly within your app can significantly increase engagement.

In this post, we’ll show how to integrate a video editor into a Flutter app using CreativeEditor SDK and customize it for specific use cases, like creating TikTok-style short videos.

You can check out the code for the default and custom Flutter video editor on GitHub

Why add a Video Editor to Your Flutter App?

The last decade has made one thing clear: video content is king. Particularly, short-form dominates digital content. At the forefront of this trend are apps like TikTok, which made people more comfortable creating, editing, and sharing bite-sized clips.

These apps have capitalized on their ability to let users apply filters, and add audio overlays, text, and more, empowering users at all skill levels to make professional-looking videos with minimal effort.

A host of different use cases stand to profit from this trend by enhancing their user experience with video editing features, particularly marketing tech tooling and apps with a messaging or social media component.

In general, video creation lowers the threshold for user-generated content and can positively impact distribution and product engagement.

Flutter, Google’s UI toolkit for building apps across platforms from a single code base, is a perfect fit for IMG.LY’s Video Editor. What often motivates the choice to use Flutter is that It ensures performance, flexibility, and a seamless user experience on iOS and Android devices. Similarly, our SDK is designed to offer a native cross-platform experience, while relying on the same underlying graphics processing engine, ensuring consistency and interoperability.

Getting Started: Integrating the Video Editor in Flutter

Let’s dive into how to set up a video editor in your Flutter app using CreativeEditor SDK.

Requirements

Before we get into the details, here’s what you’ll need:

Flutter 3.16.0+
Dart 2.12.0+
iOS 16 or later (as the video editor is currently only available for iOS, Android coming soon)
Swift 5.10 and Xcode 15.4

First, ensure your pubspec.yml file includes the necessary dependencies:

dependencies:
  flutter:
    sdk: flutter
  imgly_editor: 1.34.0

Once your environment is set up, you’re ready to start coding.

Setting Up the Editor

After adding the imgly_editor dependency, import the package in your code:

import "package:imgly_editor/imgly_editor.dart";

The next step is to initialize the editor by providing an instance of EditorSettings. This setup requires a license key, which you can obtain from CreativeEditor SDK, and an optional userId that helps track monthly active users (MAUs) across different devices.

Here’s a basic example of how to open the video editor:

class VideoEditorSolution {
  /// Opens the editor.
  void openEditor() async {
    final settings = EditorSettings(
        license: "YOUR_LICENSE",
        userId: "YOUR_USER_ID"
    );

    const preset = EditorPreset.video;

		/// Open the editor and retrieve the editing result.
    final result = await IMGLYEditor.openEditor(
        preset: preset,
        settings: settings
    );
  }
}

This launches the editor with the video preset, enabling users to trim, cut, and add filters, text overlays, music, and more to their videos.

Use Cases: Building a TikTok-Like Experience

Now that we have integrated the basic editor configuration into your Flutter app, let’s revisit some prominent use cases and how to configure the editor to support them.

Short-Form Video Creation

For a short-form video, TikTok-like user experience, the most important factor is the ease of use of any video editor and content such as stickers or audio overlays.

The following SDK capabilities can help you achieve that goal:

Control Video Duration and Position in Time: Users should be able to trim video clips on a timeline and position them relative to other tracks, such as audio clips.
Filters & Effects: Filters are essential for setting the tone of the video. Whether users want a retro, high contrast, or soft pastel look, filters provide creative control over the visual style.
Text & Stickers: Allow users to add text captions—especially important for content often watched on mute. Stickers and emojis can add a playful element to videos.
Music & Audio: Like TikTok, you can allow users to add background music or sound effects. Providing a library of popular tracks or sound snippets can enhance the content creation experience.
Video Templates: Give your users a head start with customizable video templates for different themes, topics, and occasions. IMG.LY’s CE.SDK comes with a web video editor to help build and deliver these templates to your Flutter app. You can explore a range of different templates in our video editor demo.

By integrating these features, you’ll empower users to create engaging, shareable content, elevating your app’s video creation experience to the level of consumer-grade social media apps.

Influencers and Marketing

If your app is targeting influencers or businesses, the ability to quickly create branded video content is key. You can offer:

Branded Templates: Pre-made templates that align with a brand’s style, allowing users to easily drop in their video footage and text while maintaining brand consistency. Again, check out the web editor demo to see how to build video templates.
Watermarks: Adding a watermark or logo as a brand signifier to videos can help with brand visibility and make sure content always points back to the creator.

This setup allows influencers and brands to create professional-looking videos that they can share across social media platforms with ease.

eCommerce and User-Generated Content

Video is a powerful medium to showcase products, especially on marketplaces that can leverage user-generated content and enable vendors as well as customers to create product or review videos.

For example, users could create unboxing videos, product reviews, or tutorials using the same trimming, filter, and music tools offered in a TikTok-style editor. These videos can then be shared on social media or used within the app to enhance the shopping experience.

Likewise, showcasing products using authentic demo videos tap into familiar patterns of video consumption and can boost sales for vendors.

Customization Options with CreativeEditor SDK

We have designed the Flutter plugin to be adaptable to match your brand and use case-specific requirements. Soon the plugin will also support more advanced customization options such as the ability to activate or deactivate features by default.

UI Customization: Modify the look and feel of the editor to match your app’s branding. We provide options for theming, styling color palettes, and callbacks to hook into editor events.
Video Presets: You can create custom video presets based on your users’ needs. For instance, you may want to limit video length or optimize for certain resolutions, depending on the target platform.
Video Templates: You can use the CE.SDK web UI to generate any number of video templates including collages, text designs, and animations to give your users professional-looking starting points for their video designs.
Assets: Provide custom filters, fonts, and stickers to enhance the user experience. For niche apps, this can be an excellent way to engage users by offering content that feels unique to your community.

Conclusion

Integrating a video editor into your Flutter app will improve your UX, and help boost engagement, retention, and potential distribution of your product, whether you’re building a social media platform, an influencer tool, or an e-commerce app. With CreativeEditor SDK, you can create a TikTok-like video editing experience or offer specialized tools for businesses and creators.

By following the steps outlined in this post, you can bring professional-level video editing features to your users. Explore CE.SDK’s video capabilities and dive into the docs.

Stay tuned for more updates, and please reach out if you have any questions. Thank you for reading.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

CE.SDK v1.33 Release Notes

Neslihan — Thu, 29 Aug 2024 12:08:57 GMT

Welcome to the latest update of CE.SDK! We are bringing you new features that enhance your app’s creative capabilities and user experience. This release includes Configurations for a Design Editor or Photo Editor on iOS and Android, Plugins, Voiceover Capabilities, and more.

With this release, you can:

Integrate a Design Editor on iOS and Android

Get started with our new Design Editor configuration for iOS and Android. The out-of-the-box package is designed for easy creation and management of multipage designs.

Users can switch between editing and the page overview mode to review and adjust their work.

Keep the design process straightforward and efficient: the editor features a dock at the bottom, offering convenient access to essential tools like adding text, images, stickers, and uploading new assets. Additionally, you can insert a photo directly from your camera.

Key Features

Page Management
Create and manage your design pages within a template. It’s easy to create fun carousel posts, as seen on Instagram and LinkedIn, or create a series. Head to the ‘Pages’ icon on the top right bar. In the page overview, you can create, duplicate, and edit a page. Lastly, you can change the order of pages by moving them up and down.

Image Settings
Selecting an image lets you fine-tune adjustment settings, and add filters, effects, or blur. Easily crop, straighten, rotate, or flip images.

Try the Design Editor Configuration on iOS and Android.

Use Plugins to Customize UI and Add Quick Actions

We’re excited to introduce the first part of our plugin release for CE.SDK on the web: with this release, you can tailor the editor’s UI elements, reorder components, and add quick actions like background removal.

We’ve enabled extension points within key areas of the UI: the Canvas, Navigation bar, Inspector Bar, Dock, and Canvas Menu. Modify the appearance of existing features, register new components, and integrate quick actions. For example, you can add a “Background Removal” button directly in the Canvas Menu or create complex interactions like custom cut-out lines.

For more details on this first batch release and the upcoming plugins, please read our dedicated article.

Add Voiceovers to Videos on iOS

Add voiceover tracks directly within your mobile app, making your projects more dynamic and engaging. Whether creating a tutorial, narrating a slideshow, or adding commentary to a video, this feature allows you to seamlessly record, edit, and integrate voiceovers.

The popular feature, as enjoyed on TikTok and Instagram, is a staple for content creation and adds a personal touch to video creations.

To get started, navigate to “Add Audio”, select “Voiceover” section and start recording. You can preview the recording, re-record it, and add more audio, such as music. Adjust the audio levels for a balanced background track.

Try recording a voiceover with our Demo App for iOS.

Integrate a Photo Editor for iOS and Android

Our new Photo Editor Configuration is designed to offer an intuitive, out-of-the-box solution that bundles a range of key photo editing capabilities. Enhance individual photos, add text and shapes, or apply filters—this configuration is a kick-start to provide high-quality photo editing for iOS and Android with CE.SDK.

Key Features

Crop, Resize
Crop, straighten, flip, resize, and rotate your photos.

Adjustments
Fine-tune your photos by adjusting brightness, contrast, and other key settings, ensuring your images look their best.

Text, Shapes, and Stickers
Enhance your photos by adding and customizing text, shapes, and stickers. Whether you’re creating eye-catching titles, intricate shapes, or playful stickers—our configuration gives you the flexibility to adjust fonts, font colors, sizes, and positions to perfectly match your vision.

Effects, Filters, Blur, Blending Modes
Transform your images with fun effects, blur, and filters. Apply overlays to add depth and texture to your photos by adjusting blend modes and opacity to achieve the perfect effect.

Additionally, use our range of photo filters to set the mood, from warm and nostalgic tones to vibrant, modern effects, giving your images a distinctive and professional look.

Check our documentation to get started with our photo editor configuration, or try the photo editor on iOS and Android.

Improvements

Preview Audio on iOS and Android

We’ve made it easier to select the perfect audio clip for your projects. Listen to audio tracks directly within the interface before adding them to your video. Tap the play button of an audio clip in the asset library to preview it, and adjust your audio clip within the timeline.

Resize Pages

Resize pages in your web editor by simply grabbing corners and side handles, moving beyond the limitations of input fields or preset format sizes. This intuitive resizing integrates seamlessly with other adjustments, such as image cropping, ensuring a smooth workflow. Enable or disable this behavior as needed.

Create Rounded Corners

You can now easily create rounded corners for images on the web, iOS and Android. Within your designs, you can add a sleek and modern touch to your projects.

Drag and Select Multiple Elements

You can now click and drag with your mouse to draw a selection rectangle, instantly selecting all blocks within the covered area. This feature saves you time and effort, working with complex templates.

Thanks for reading! Don’t hesitate to reach out if you have any questions or need assistance.

Over 3,000 creative professionals gain early access to our new features, demos, and updates—don’t miss out, and subscribe to our newsletter.

Please note that these release notes also include improvements from previous versions.

Designing A Timeline For Mobile Video Editing

Frank — Tue, 20 Aug 2024 13:08:58 GMT

This article documents the most surprising insights from our journey building timeline user interface. We’ll explore everything from overcoming usability challenges, visualizing data, prioritizing information, and refining touch interactions.

IMG.LY’s CE.SDK for iOS now comes with a native video editor, built from scratch in SwiftUI. I was responsible for coding and polishing the timeline user interface, working with both our design team and our cross-platform engine team. When I started implementing the timeline, we had the basic design concept sketched out in Figma. However, for an advanced UI like this, nothing beats rapid prototyping and iteration between design and development to make it feel right.

I. Visualizing Time

Timelines are a well-known convention, at least to modern humans. Timelines show durations in time as segments in space, distributed along the horizontal axis, with multiple tracks for things that happen in parallel. This is a simple yet effective way of representing moments in time visually. As obvious as it may seem today, this flavor of data visualization first appeared only 250 years ago. One of the earliest known examples is Joseph Priestley’s A Chart of Biography, published in 1765. Here is Priestley’s A New Chart of History from 1769, which is even more impressive:

Video editing timelines are built on this exact principle of mapping time to space: A digital video composition consists of tracks for simultaneous audio and video layers. These tracks contain individual clips that can be rearranged, trimmed, and transformed. Navigating the video’s time dimension is as easy as scrolling left and right. The tracks and clips serve as landmarks for orientation, forming the mental model of what the video composition is.

Timelines map time to space

https://www.youtube.com/embed/cKcksLW95-Y?feature=oembed

Horizontal Direction

Modern iOS user interfaces appear flipped in right-to-left languages: the buttons and controls are in different horizontal positions. The layout follows the reading direction of the language. Even the familiar drill-down navigation animates to the left when drilling deeper into the hierarchy, and the Back button sits in the upper right corner, its chevron pointing to the right. So, naturally, one of the first things we checked was whether timelines should run right-to-left in writing systems like Arabic or Hebrew. According to a trustworthy source, this is not the case: we found an old version of Apple’s developer guides stating that editor timelines in software are universally left-to-right, even if the surrounding interface layout is flipped by the operating system for right-to-left system languages.

Cognitive Load vs. Discoverability

When designing our video timeline, we experimented with both the size of the clips and with the dimensions of the timeline itself. How much can you fit on a screen before it looks cluttered? How large can the controls be without being clumsy and inefficient? The only way to find out is to try it out right on the device.

Hiding features in dropdown menus is easy. Banishing them to separate screens is even easier. But hiding functionality hurts discoverability. Out of sight, out of mind. We think user interfaces in productivity apps should be the opposite. They should prioritize high information density and avoid extra navigation steps as much as possible.

We increased the density of the toolbar that sits below the timeline: Our goal was to make all the important features easily accessible, despite the small screen. We achieved this by changing the single floating Add button from the initial designs to a more granular set of buttons to add specific asset types, like stickers and text. This not only increases efficiency for frequent users, it also makes the capabilities of the video editor more obvious and approachable.

Adaptive Height

As space is at a premium on a small screen, we wanted to make the video preview as large as possible. We decided to allow the user to collapse the timeline at any time to focus on the video canvas. And even without manually minimizing the timeline, it doesn’t waste space: the timeline adjusts its height to fit snugly around its contents. It only grows as you add more clips.

Units and Scale

The numeric scale is an essential part of any timeline. It helps you see the in and out time of a clip at a glance, and the permanently visible ruler makes changes in the zoom level easy to understand. We added dots as subdivisions between the ruler’s numeric labels, which gives it a nice tool-like character. At full zoom—when the ruler shows 5-second intervals—we decided to use four dots in between, so there is one dot for each second. We think this detail adds a welcome sense of precision.

The clips themselves have a duration label when they are selected. If the duration is below 10, you get a fractional digit (e.g., 4.2s). For the ruler, we went with the same short format (5s, 10s, 30s, etc.) for all values below one minute, and the technical format (1:00, 1:30, etc.) for the rest. Of course, we use iOS’s built-in number formatters to get properly formatted and localized strings for our labels.

An important detail for the label that shows the current playback timecode: The seconds must always be rounded down. If the playhead sits between two seconds, the lower number must be displayed until the playhead crosses the indicator for the higher second. Simple rounding to the nearest integer would be confusing, because it doesn’t match our expectations when dealing with timelines. Five seconds do not count as five seconds until they have elapsed in their entirety.

II. Manipulating Time

What looks good is not guaranteed to feel good when you interact with it. A video editor timeline is more than just a visualization; it’s also a user interface. As such, it should be self-explanatory and provide constant feedback as the user manipulates it.

Fingertip-sized

The clips must be large enough to be easy to touch and at the same time detailed enough to give a decent overview. The size that can be easily tapped by most people is around one square centimeter, which matches the 44 × 44 pt default size of iPhone system controls. Our testing showed that we could go a little smaller. Editing video is a focused task, so the tap targets don’t need to be as robust as for tasks that happen while you, say, run for the bus. And all user interfaces should be scalable for accessibility anyhow, so we didn’t hard-code any sizes and we also made the dimensions adjustable. And, speaking of adapting to individual user needs: of course, all the text labels in our timeline are automatically scaled depending on your device settings.

Handles

Selected clips in the timeline get a bold blue frame with large handles on either side. These handles are used to trim the clip’s length. For comfortable tapping and dragging, the handles must be larger than their visual size. In fact, we made the hit area more than twice as wide as the visual appearance suggests.

Early on in our design process, we realized that the trim handles had to sit on the outside of the clip, extending beyond the left and right edge. This made it hard to see the clip’s actual size, so we added little grooves: little triangular indents that point to the start and end of the selected clip.

Some clips in the timeline can be extended to any duration, like text or stickers. Other clips have a finite footage duration, like video and audio clips. To make this difference visible, clips with a finite footage duration reveal a ghost outline in the background while they are resized, featuring marching ants. The handle icon changes from an outward-pointing chevron to a vertical line when the clip can’t be made longer.

Scrolling and Zooming

A rather invisible detail was one of the trickiest things to get right: the coordination of all the different finger gestures on the timeline.

Horizontal Scrolling: Timeline scrolls horizontally to move in time.
Vertical Scrolling: Timeline scrolls vertically providing room for tracks.
Clip Movement: Dragging horizontally on a selected clip moves it.
Duration Adjustment: Dragging the trim handles adjusts the clip’s duration.
Pinch-to-Zoom: Pinching with two fingers changes the zoom level of the visualization.

All these gestures must behave as expected and not interfere with each other. We found it unexpectedly difficult to fine-tune and harmonize these interactions with pure SwiftUI, so we used some proven legacy iOS techniques to get it right. If you’re interested in the technical details of our solution with SwiftUIIntrospect and UIGestureRecognizers, check the source code.

Snapping

Video editing must feel precise, even on a small screen. When you trim or move clips, they should snap to other clips, to the playhead, and to the start and the end of the timeline. To show which position the selected clip snaps to, we designed animated dotted lines. We use the iPhone’s haptic engine to provide subtle feedback for snapping, which really helps to make the interaction feel natural—like a notch snapping into a detent. Something that we didn’t get right at first: Snapping should only respect those snapping points that are currently visible in the viewport. Of course! Otherwise, a partially visible clip may snap to something offscreen, which is very confusing.

Scrubbing Preview

For editing a video clip to feel precise, you need to be able to see exactly where you’re trimming. But the playback position in our mobile timeline can’t change while you’re trimming, because that would mean scrolling the timeline—which would move the very clip you’re trimming. The trim preview needs to be independent of the composition’s playback position. We solved this by adding a temporary overlay with its own playback time to the canvas. When you start dragging one of the handles on either side of the clip, the overlay shows the exact trim position of the video clip. When you release, the preview returns to the playback position.

Try it out!

“The only way to experience an experience is to experience it,” said interaction designer Bill Moggridge, and we couldn’t agree more! If you’re curious how our new video timeline feels, download our iOS app and try editing your own videos in our brand-new timeline experience.

If you’re a developer, check out the video editor documentation. With IMG.LY’s SDK, you can add all of these video editing capabilities to your own apps in a breeze.

How to Load Stripe Data into Google BigQuery

Sascha — Thu, 18 Jul 2024 10:04:26 GMT

At IMG.LY, we recognize that leveraging data is essential for driving innovation and growth. To optimize our data for reporting, we consolidate multiple data sources, including Stripe billing and financial data, into Google BigQuery.

IMG.LY is the leading provider of creative editing SDKs for video, photo, and design templates. While this article may not directly relate to media creation, we believe in empowering developers through knowledge sharing. Let’s dive in.

Until now, we’ve relied on Fivetran to fetch our data from Stripe and store it in Google BigQuery. Fivetran uses Stripe’s API, calling each endpoint, iterating over all resources, and storing the results in BigQuery (or any other supported data warehouse). While this generally works well, issues can arise. For instance, we sometimes create Stripe Subscriptions using inline pricing with the price_data parameter. This generates a new Price object in Stripe on-the-fly and immediately sets it to active: false. Consequently, the Price object is not returned by Stripe API’s price endpoint, leading to missing data in our warehouse. Although Fivetran’s support was exceptional in resolving this issue within a day, it highlighted a potential flaw in relying solely on ETL services for data extraction.

Recently, Stripe introduced Data Pipeline, its own service for transferring Stripe data into a data warehouse. This ensures complete, reliable data without needing a third-party service to read Stripe’s API. Additionally, you can receive test environment data and access several tables not available via the API. For a comprehensive summary of the available data, refer to Stripe’s official data schema.

Currently, Stripe supports only Snowflake and Amazon Redshift as data warehouses. However, they’ve recently added the option to deliver data as Parquet files into Google Cloud Storage (GCS). The next step for us was to import this data into Google BigQuery.

Setting Up Stripe Data Pipeline with Google Cloud Storage

Stripe is renowned for its excellent developer experience, and this beta feature is no exception. Enabling it within the Stripe Dashboard is quick, and the documentation is straightforward. After following the instructions and enabling the feature, it takes a while for data to appear in GCS. Once available, a complete data dump is provided every 6 hours, structured as follows:

At the root level, Stripe creates a folder representing the date and time of the latest transfer, e.g., 2024071600 (YYYYMMDDHH), representing the 12 am push on July 16, 2024.
One level deeper, there are two folders: livemode and testmode, representing live and test data, respectively.
Each folder contains one folder per data table, e.g., subscriptions or invoices. Additionally, a coreapi_SUCCESS file indicates successful data transfer to your GCS bucket and readiness for consumption.
Within the table folders are several Parquet files containing the actual data for each table.

Loading the Data from Google Cloud Storage into Google BigQuery

There are multiple ways to transfer data from GCS to BigQuery. We opted for the following approach:

Using Google Cloud Scheduler to publish a message to Google Pub/Sub every 6 hours at 1 am, 7 am, 1 pm, and 7 pm.
Creating a Google Cloud Function that listens for new messages on the above Pub/Sub topic. When a message is received, it triggers a Node.js script that loads the most recent data from GCS into BigQuery and deletes it from GCS.

Let’s delve into the details.

Create a Google Cloud Scheduler Job

First, create a new Cloud Scheduler job here with the following configuration:

Name: Choose a name for this job.
Region: The region is not crucial for this task; we used europe-west3 since most of our services are in Germany.
Frequency: We want the job to run every 6 hours at 1 am, 7 am, 1 pm, and 7 pm. Stripe publishes data every 6 hours, but it takes time to transfer it to GCS. We chose 1 hour later than Stripe’s push time, so our value is 0 1,7,13,19 * * *.
Timezone: Choose ‘Coordinated Universal Time (UTC)’.
Target type: Choose ‘Pub/Sub’.
Select a Cloud Pub/Sub topic: Select or create a new Pub/Sub topic using the default configuration. This is used to trigger the Cloud Function.
Message body: For this task, we don’t look at the contents of the message, as such the content of this value doesn’t matter. We opted for a simple load string.

Finally, click ‘Create’ to set up the scheduler. Now, a message is published to the selected Pub/Sub topic every 6 hours. Next, we need to respond to this message.

Create a Google Cloud Function

Create a Google Cloud Function triggered by Pub/Sub here with the following configuration:

Environment: Choose ‘2nd gen’.
Function name: Choose a name for this Cloud function.
Region: Select the region for the function, typically europe-west3 for our services.
Trigger type: Choose ‘Cloud Pub/Sub’.
Cloud Pub/Sub topic: Select the Pub/Sub topic created in the previous step.

Adjust the ‘Runtime, build, connections and security settings’ based on your Cloud setup and the required processing power for Stripe data. Generally, the following settings work well:

Memory allocated: ‘512 MiB’
CPU: ‘1’
Timeout: ‘540’
Minimum number of instances: ‘0’ (to ensure the function shuts down when not in use)
Maximum number of instances: ‘1’
Service account: Use or create a service account with permissions to access the GCS bucket where Stripe data is stored and the BigQuery datasets to load the data.
Ingress settings: Choose ‘Allow internal traffic only’.

Click ‘Next’ to provide the function’s code. Select:

Runtime: ‘Node.js 20’
Source code: ‘Inline Editor’
Entry point: loadStripeData

In the package.json, add the BigQuery and Cloud Storage Node.js packages:

In the index.js, add the following code:

This script does the following:

For each environment (live and test), it searches for the latest folder containing a coreapi_SUCCESS file.
For each table, it groups all related Parquet files and loads them into the BigQuery table using WRITE_TRUNCATE, which overwrites existing data. Note that the location is specified as EU, matching our BigQuery dataset and GCS bucket location. Adjust this parameter if your data is elsewhere.
If all files for an environment are loaded without errors, the files are deleted from GCS. This step is optional; if you prefer to keep a backup, you can omit this part.

Click ‘Deploy’ to deploy your Cloud function.

Create BigQuery Datasets

The final step is to create two datasets in Google BigQuery. Open Google BigQuery, click on the three dots next to your project’s name, and select ‘Create dataset’. Enter a name and choose a location matching your GCS bucket’s location. Repeat this process for the test dataset.

Conclusion

Following these steps will ensure your Stripe data is imported into BigQuery and automatically updated every 6 hours. However, as Data Pipeline for GCS is still in beta, there are some limitations. For example, the schema of the Parquet files lacks type annotations for timestamps, so all timestamps in BigQuery are represented as INTEGER instead of TIMESTAMP.

Additionally, some tables, such as subscription_item_change_events, are not currently transferred when syncing with Google Cloud Storage, although this issue is expected to be resolved soon. Meanwhile, we continue to use Fivetran in conjunction with the above method to sync Stripe data to Google BigQuery and plan to fully migrate to Data Pipeline once it exits the beta phase.

Thank you for reading!

3,000+ creative professionals gain exclusive access and hear of our releases first—subscribe to our newsletter and never miss out.

Background Removal in the Browser Using ONNX Runtime with WebGPU

Daniel — Tue, 11 Jun 2024 16:26:17 GMT

TL;DR: Using ONNX Runtime with WebGPU and WebAssembly leads to 20x speedup over multi-threaded and 550x speedup over single-threaded CPU performance. Thus achieving interactive speeds for state-of-the-art background removal directly in the browser.

Removing background from an image is a typical job to be done in creative editing. We have come a long way from manually knocking out the background from an image to full automation with Neural Networks.

Most state-of-the-art background removal solutions work by offloading the task to the server with a GPU as it was simply infeasible to run the NN on the client.

However, running background removal directly in the browser offers several advantages over server-side processing:

Reduced server load and infrastructure costs by offloading heavy lifting to the client.
Enhanced scalability by distributing the workload across client devices.
Easier compliance with data protection and security policies by not transferring data across a network to a server.
Offline processing without needing a reliable internet connection.

It caters to a wide range of use cases, including but not limited to:

E-commerce applications that need to remove backgrounds from product images in real time.
Image editing applications that require background removal capabilities for enhancing user experience.
Web-based graphic design tools that aim to simplify the creative process with in-browser background removal.

In general, two factors influence the feasibility of running background removal directly on the client.

The execution performance, and
the download size of the Neural Network.

The performance or overall runtime is the major factor to be useful in interactive applications, if a user has to wait several minutes or hours for a neural network to execute, this is in many cases far too long in terms of good user experience. From experience, there are three factors to consider.

The initial first-time execution. The major factor is that neural networks come with the drawback of generally being several MB to GB in size, thus the time to download the neural network into the browser cache is considerable. In subsequent browser page reloads this has no impact anymore.
The neural network or session initialization time, cannot be cached and has to run with every reload of the page in the browser.
The neural network or session inference time, largely depends on the longest path inside the neural network and most importantly the execution time of each operator in the neural network.

Towards Real-time Background Removal in the Browser

Neural networks are commonly trained in frameworks like PyTorch, which is a neural network library for Python, as such not usable directly in the browser. The best option to run neural networks directly in the browser is converting the neural network into the Open Neural Network Exchange (ONNX) format, which is a widely supported standardized format by Microsoft used extensively in the industry.

These ONNX-formatted neural networks can then be reconverted into a platform-specific format or directly executed by a supported runtime.

The ONNX Runtime by Microsoft is a high-performance inference engine designed to run ONNX models across various platforms and languages. One notable feature is ONNX Runtime Web, which allows JavaScript developers to execute ONNX models directly in the browser. ONNX Runtime Web offers several execution providers for hardware acceleration. For instance, its WebAssembly execution provider enhances CPU execution performance using multiple Web Workers and SIMD instructions.

More importantly, starting from version 1.7.0, ONNX Runtime Web includes official support for the WebGPU execution provider. WebGPU is a modern web API that enables developers to utilize GPU power for high-performance computations, offering a significant performance boost over CPU-based in-browser machine learning. WebGPU support has been available by default since Chrome 113 and Edge 113 on Mac, Windows, and ChromeOS, and Chrome 121 on Android. For the latest browser support updates, you can track them here.

Implementation Details

For the implementation of the open-source package @imgly/background-removal-js, we started the journey with a neural network implementation in PyTorch written in Python. The network was then converted to ONNX. You can see our live showcase here.

The original model was using 32-bit floating point (fp32) precision, which is fine, but results in a file size of 168 MB after converting it to ONNX. As mentioned earlier, the size of the network has a large impact on perceived first-time execution performance as the download time tends to be longer than the execution time, but more to that later.

To reduce the size of the model, we converted the model to use fp16 (16-bit floating point) and QUINT8 (Quantized 8-bit) datatypes. Thus, effectively reducing the size to half (84MB) and a fourth (42MB) of the original size. Additionally to the download size, the operators used will be converted corresponding to the datatype and different data types and depending on the hardware may lead to speed improvements or even deteriorating due to specialized hardware being used or not being present.

Note, that while that sounds great, the conversion has a potential negative impact on the quality of the output as we are removing information in the neural networks and since we are working with images, artifacts might become visible and the quality of the resulting background mask is reduced.

Evaluation

When dealing with neural networks in the browser we can identify three different scenarios

First Use
First Run
Consecutive Runs

The first-use scenario occurs when the web application is started the first time. The neural network has to be downloaded from the server into the browser sandbox, as neural networks are several 10-1000 MB in size this is not neglectable.

The first run assumes that the neural network is already present in the browser cache, however, when the neural network ought to be used, the network has to initialize before execution.

In consecutive runs, the neural network is already in memory and the execution time is largely determined by the neural network depth and operators only.

First Use – Neural Network Download

While download or time is a large factor in the first-time execution time, this is largely dependent on the available network bandwidth, as such it is not part of the evaluation but in order to get an idea here are the expected download times for the fp32 (168MB) and fp16 (84MB) neural network subject to various common network bandwidth:

Network Bandwidth	Filesize	Download Time
10 Mbps	84 MB	67s s
100 Mbps	84 MB	6.7 s
1 Gbps	84 MB	0.67 s
10 Gbps	84 MB	0.067

Network Bandwidth	Filesize	Download Time
10 Mbps	168 MB	134.4 s
100 Mbps	168 MB	13.4 s
1 Gbps	168 MB	1.34 s
10 Gbps	168 MB	0.13 s

Based on the assumption that the median download bandwidth is ~100 Mbps, we see that it’s in the 5-15 second range.

First Run – Neural Network Initialization / Compilation

Before the neural network can be executed it has to be initialized. Initialization includes several execution provider-specific steps. Most prominent are the time to upload or convert the data to the execution provider and execution provider-specific ONNX graph optimization passes. This all adds to the first run experience. We have evaluated the average session initialization time for the CPU (WASM) and WebGPU provider on a MacBook Pro 13” from 2024 with an Apple M3 Max 16 cores to get an idea of the general impact on first run execution time:

Device	Datatype	Session Initialization Time
CPU (WASM)	fp32	~320ms
CPU (WASM)	fp16	~320ms
WebGPU	fp32	~400ms
WebGPU	fp16	~200ms

Note, that session initialization is not negligible and adds significant runtime overhead and might be subject to additional optimization possibilities like caching the optimized model.

Consecutive Runs – Neural Network Execution

Independent of the download and initialization time, we evaluated different execution providers on a MacBook Pro 13” from 2024 with an Apple M3 Max 16 cores. While this is top-end consumer hardware, the general trends will probably apply to various hardware configurations. As a reference, we choose the single thread performance with the neural network running on the CPU with 1 worker thread and SIMD disabled.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
CPU (WASM)	No	1	fp32	~53000 ms	1.0 x
CPU (WASM)	No	16	fp32	~6300 ms	8.4 x
CPU (WASM)	Yes	1	fp32	~15000 ms	3.5 x
CPU (WASM)	Yes	16	fp32	~2000 ms	26.5 x

The data above reveals that running the neural network without any acceleration such as SIMD and threading in the browser results in almost 53s runtime, and as such is for most interactive use cases too slow to use. Due to the optimizations of the ONNX Runtime that uses SIMD and threads, we can achieve an overall speedup of roughly ~26 times compared to the baseline performance. Thus decreasing the session runtime to around 2s and making it usable for interactive applications.

As mentioned before, we neglected the download time and size of the network. Leveraging the fp16 model compression, we re-ran the benchmarks and got similar results as before, but with half the bandwidth required to download the network for the first time application. The general visual performance of the fp16 and fp32 is similar or visually not perceivable. The quint8 model led to artifacts and as such unusable for visual processing, thus we excluded from the following benchmark.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
CPU (WASM)	No	1	fp16	~55000 ms	1.0 x
CPU (WASM)	No	16	fp16	~7300 ms	7.2 x
CPU (WASM)	Yes	8	fp16	~15000 ms	3.5 x
CPU (WASM)	Yes	16	fp16	~2300 ms	23.9 x

The results are a little worse than the fp32 version, which might be because the fp16 datatype has no direct specialized hardware in modern CPUs like the M3 Max CPU, and as such additional fp16 to fp32 conversion operations have to be performed.

Our final benchmark measures the WebGPU performance, which led to the following impressive results. To have a fair comparison and understanding of the impact of the WebGPU technology, we compare it with the best CPU version with 16 threads and SIMD enabled with the fp32 model, which was around ~2 s.

Device	SIMD	Threads	Datatype	Session Runtime	Speedup
WebGPU	Not applicable	Not applicable	fp32	~120ms	16.6 x
WebGPU	Not applicable	Not applicable	fp16	~100ms	20.0 x

The WebGPU performance varies around 16 to 20 x improvements over the best CPU execution time. The GPU has specialized hardware for fp16 instructions, as such the model performs better than the fp32 model. Also, note that the session initialization time is also reduced due to half the required bandwidth to upload the network data to the GPU.

Therefore, the first run of the network will take ~300 ms and consecutive runs will be ~100 ms, leading to near real-time performance in the browser.

Note, that the WebGPU performance is an astonishing 550 times faster than the single thread, with no SIMD performance.

Try the live implementation in our background removal showcase.

Conclusion

WebGPU is a major leap in establishing the browser as a factor to be reckoned with as a true Application platform. With IMG.LY, we are striving to leverage modern technology to make design tools accessible, this includes on-device execution, on-premise, but also on-cloud execution of design tools leveraging neural networks.

As a next step, we will port background removal to all our supported platforms, ONNX Runtime seems the best choice, as it is already available for all the potential platforms we support.

Furthermore, we are evaluating the feasibility of in-browser background removal for videos to be included with our video-editing suits.

Try our tools today to see how we help bring unique creative editing experiences to any application. Or contact us to discuss your project.

Unleash Creativity with CE.SDK’s New Plugin System

Daniel — Mon, 03 Jun 2024 08:42:25 GMT

At IMG.LY, we have always believed that a superb design editor should be effortlessly customizable and extensible. We are thrilled to roll out a brand-new Plugin system for CE.SDK in the upcoming months—to take creative editing and feature development to the next level.

Starting with one-click features like background removal or vectorizers, smart design tools like QR codes or subtitle generators, and deeply interactive features like generative AI for text and images; all these tools can be used or built by customers soon.

Additionally, our upcoming Plugin system will bring you unparalleled autonomy by making feature development for our SDK accessible and offering extensive options to reconfigure our editor’s UI.

Start exploring our Plugin System rollout now to immediately benefit from upcoming features.

Built for Modification

Let’s dive deep into some of the opportunities CE.SDK plugins help unlock.

Unlocking the AI Revolution

While the AI transformation is already fully underway, much of the tech is still not very accessible to product builders, often requiring deep technical knowledge to get started. At the same time, AI features become significantly more valuable when integrated with other editing functionalities in workflows or automation. With our plugins, we aim to make it effortless to leverage innovation and put it to use.

Boosting Customer Autonomy

Key to our success is providing maximum flexibility and autonomy to our customers about product decisions. Ultimately, you shouldn’t depend on our product roadmap; rather, you should be able to add features when you like. While our SDKs are already highly configurable, plugins allow tailoring the whole user experience not only on a look & feel level but through custom functionalities and editing experiences.

Accelerate Product Expansion

Many ecosystems witnessed explosive growth in added value to the user after releasing plugin mechanisms. Currently, only IMG.LY core developers can contribute to the SDK. We have started to extend this to solution engineers and even designers on our team who don’t have much knowledge of the inner workings of the SDK. Ultimately, we will push this more and more into a community of contributors, making the community’s innovation accessible to everyone.

Key Concepts

Three important concepts have driven the development of our Plugins:

Customizable Menu Bars
We are extending our API so that it allows easy hooking into various parts of the UI. Our editor has key components like the inspector, toolbar, and on-canvas menu. These are now all accessible through an API, so you can hook your feature anywhere in the editor.

UI Building Blocks to Provide Consistency
To reach a high level of consistency and speed up time for development, we will be providing out-of-the-box UI components such as buttons, sliders, text inputs, etc.

Escape Hatches
From experience, we know that sometimes unique functionality needs unique solutions, so we have added escape hatches to add custom elements via HTML whenever needed.

What Can You Build with our Plugin System?

Let’s explore some potential use cases of the plugin system.

Custom Actions

Adding custom actions is a great option to make simple third-party APIs accessible within the editor. This can be one-click edits such as background removal, vectorizers, or auto-enhancement for images. You can also add custom actions for text in combination with Large Language Models (LLM) to provide features like autocorrection and improved writing, etc.

Custom Tools

Some custom functionality will require more than just a single button, e.g., to generate AI images, background patterns, QR codes, or maps. In these cases, you’ll require sliders, text input, drop-downs, and many other UI elements. With plugins, you can easily create panels with your own UI to bring any custom tool to life.

Custom Assets & Presets

Apart from building custom tools, you can also bundle and group effects into presets and make them accessible in a custom panel. This is especially useful to simplify the design process or create standard design components: for example, providing beautiful text presets will enable your users to create instantly great text designs without any design knowledge.

Custom Libraries

Integrate third-party libraries, such as Unsplash, Getty Images, Pexels, or your own.

Custom Editor Behavior

Some of our customers asked us how they could move a functionality from one place in the editor to another. Let’s say you wish to move the function ‘move to front’ from the inspector to the canvas menu. This is not a problem! You can do this by using the internal API endpoints of our editor.

As for custom editor behavior, you can do far more than just move functionality. Here is a demo of layer lists we built as a plugin.

Custom User Feedback

Additionally, you can enhance the canvas with overlays, which are useful for providing alerts, instructions, or feedback directly on the canvas.

What’s Next

We are now rolling out Plugins and building an initial set of features through Plugins ourselves—available for the web first, and mobile SDKs will follow. Keep your eyes peeled for our next releases and don’t hesitate to get in touch with us to learn more about plugins, and how your product benefits from integration without losing time and resources.

Thank you for reading. Join 3,000+ creative professionals—subscribe to our newsletter for updates on new features, plugins, early access, and more!

CE.SDK v1.25 Release Notes

Neslihan — Tue, 07 May 2024 11:43:59 GMT

Welcome to the latest update from our team! Since our last release, we have been working to bring more creative capabilities to your app and improve user experience. With this release, you can:

Recolor Images like T-shirt Mockups

Replace colors in any image with our new recoloring feature. Select a color you want to change, choose its replacement, and apply the changes. Recoloring is ideal for tasks like altering the color of specific elements, such as t-shirts in product photos.

Simply choose your image, navigate to Effects, and select “Recolor”. Choose the color you want to change (Source Color), and the color you want to replace it with (Target Color). For a more refined result, adjust “Smoothness”, “Color Match”, and “Brightness Match”.

This feature is available on Web, Android, and iOS platforms.

Check our documentation on filters and effects to learn more.

Create Transparency with a Green Screen

Easily make green screen backgrounds transparent in your images and videos on the Web, and for images on Android and iOS. Our Green Screen Effect is tailored to identify and eliminate green backdrops—or any other color you specify. Perfect for content creators and videographers who need to isolate subjects without manually editing backgrounds.

Select an image or video clip with a color background you wish to remove. Then choose “Green Screen” in the Effects, and our effect handles the rest. Fine-tune the effect to your liking.

User Interface Improvements

In our latest update, we’ve focused on enhancing the usability and functionality of our interface. Here’s what’s new:

Audio Preview

Choosing the right audio clip is now easier with our new Audio Preview feature. Preview audio clips directly within the interface and make the perfect selection for your projects.

Selection Frame Improvements

We’ve upgraded our selection frames with new icons, hover effects, enhanced drag functionality, and shadow rendering. These improvements make manipulating elements within our platform more intuitive and visually appealing. Whether you’re rearranging components or resizing elements, our refined handling ensures a smoother interaction.

Please note that these release notes also include improvements from v1.24.

Thanks for reading. Join over 3000 specialists with powerful apps, and subscribe to our newsletter. We keep you in the loop with brand-new features, early access, and updates.

CE.SDK v1.19 Release Notes

Neslihan — Fri, 22 Dec 2023 14:26:48 GMT

Since our last release, we’ve been crafting new features to empower your users’ creative journey. Today, we’re thrilled to introduce CE.SDK v1.19! With this release, you can:

Unify Design Power with Block Unification

Introducing Block Unification for CE.SDK on Mobile and Web—a feature that simplifies design versatility. Videos, images, shapes, and stickers now seamlessly come together as one Graphic Block, easily modified within a unified inspector. All supported settings for your selected block, including strokes, filters, and adjustments, are conveniently displayed in a single, user-friendly space.

Unified Graphic Block
Experience a singular Graphic Block supporting diverse fills (images, videos, colors, gradients) and shapes. Switch fill types (video, image, color) effortlessly, enabling dynamic designs in one place.

Easy Fine-Tuning
Block Unification integrates effects, filters, adjustments, and blur uniformly across all design elements. Elevate your designs with precision and consistency.

A Treat for Users and Developers
For developers, Block Unification means streamlined code maintenance. The consolidation of design blocks into one Graphic Block simplifies development, allowing more focus on crafting exceptional design experiences with IMG.LY.

Enhance Interaction with Mouse Wheel Support

In our latest update, our engine now directly registers mouse wheel events on the canvas, eliminating the need for manual camera manipulation. This seamless integration brings a host of new features, including smooth zooming, centered around the cursor, and support for pinch gestures on multitouch trackpads. Enjoy a more intuitive and unified interaction experience across various inputs, simplifying and enriching your design process.

Migrate to CE.SDK v1.19

The structural changes of CE.SDK v1.19 bring a more composable and powerful editing experience. To benefit from our new features, and to attain to licensing changes, head to our documentation on migrating to v.1.19 for an easy transition.

Wait. There’s more. As we explore the extensive capabilities of our SDK, we’re unveiling new showcases to demonstrate your possibilities. Plus, stick around until the end for an exciting announcement that will elevate your video creation experience!

New Showcase: Automated Resizing

In our new live showcase for web, effortlessly generate size variations of your designs and seamlessly scale your marketing materials across platforms. The automated resizing feature empowers you to easily leverage marketing materials without requiring the involvement of a design team, ensuring efficiency and adaptability.

New Showcase: Version History

Discover the convenience of Version History, enabling you to monitor changes made to each design and seamlessly restore past versions when needed. Explore our straightforward Version History showcase for a practical design experience.

Outlook: Offer TikTok-Inspired Video Editing

Video remains the reigning champion for attracting and retaining your audience. Yet, building a video editor from the ground up is a pain.

To simplify your journey and save valuable time and resources, we are excited to unveil a groundbreaking SDK for creating captivating short-form videos in January 2024!

Empower your users to arrange video, audio, text, and graphics seamlessly on a sleek video timeline. Our new camera introduces popular features like Voiceover, Zoom, Tap to Record, and more. Plus, bring the beloved Split Screen Modes for Reactions and Duets, akin to TikTok and Instagram!

Be the first to access our new IMG.LY SDK—join our waitlist now!

Thanks for reading! Subscribe to our newsletter to stay in the loop with the latest news and updates.

CE.SDK v1.17 Release Notes

Neslihan — Tue, 17 Oct 2023 12:48:42 GMT

In the ever-evolving landscape of app development, staying ahead of the curve is crucial. Since our last release, we’ve been crafting new features to empower your users’ creative journey. Today, we’re thrilled to introduce CE.SDK v1.17. With this release, you can:

Support WebP File Formats

Interfaces: Engine
Platforms: All

We’re delighted to announce that our editor now supports WebP images, delivering you an array of benefits. WebP is gaining popularity for its unique ability to strike the perfect balance between high image quality and smaller file sizes. This means your visuals will look stunning while optimizing website performance.

Capture and Export Videos with Camera for Android

Interfaces: Engine
Platforms: Android

Video content creation has become the lifeblood of today’s app ecosystem. Social entertainment, user-generated content, and the demand for immersive experiences have transformed the way we engage with audiences.

We’re excited to introduce Camera Recording for Android, seamlessly integrating your live camera preview with our engine’s extensive capabilities. This means you can apply a wide range of effects, strokes, and drop shadows to your live camera feed, all while ensuring it harmoniously blends into the composition of your scene.

This feature allows you to seamlessly capture, edit, and export high-quality videos directly within your applications. Whether you’re building a social platform, a video editing app, or a storytelling tool, this feature will redefine your users’ video creation experience. Learn how to use your camera in our Engine for Android in our documentation.

Stay tuned for more exciting updates and the full reveal: CE.SDK v1.17 is just the beginning of a new era in app-driven video content.

Thank you for being a part of this journey with us. Subscribe to our newsletter to ensure you never miss out on updates! ?

Elegant Event Handling in Kotlin - A Refactoring Walkthrough

Sven — Fri, 29 Sep 2023 07:00:53 GMT

In the world of software development, code refactoring is the hero that rescues us from tangled and inefficient code. In this article, we’ll embark on an adventure to revamp Kotlin code handling diverse events. Our mission? To enhance performance and style, making the code sleeker, more maintainable, and a joy to work with.

What We Aim to Achieve

On this journey to transform Kotlin event handling, our goal is to refine our code to be more efficient, readable, and maintainable. We’re introducing a variety of improvements, including:

Replacing a convoluted when statement with a HashMap for lightning-fast (O(1)) performance.
Infusing syntactic sweetness with inline functions and reified type parameters.
Employing delegated properties for cleaner dependency injection.
Adhering to the Single Responsibility Principle by enabling multiple specialized event handler functions.

Step 1: The Starting Line

Our adventure begins with a glance at the original code. This codebase manages a variety of block events through a function named handleBlockEvent and an event handler function called onEvent. Let’s unveil the original code:

open fun onEvent(event: Event) {
    // ...
    handleBlockEvent(engine, getBlockForEvents(), checkNotNull(assetsRepo.fontFamilies.value).getOrThrow())
}

fun handleBlockEvent(engine: Engine, block: DesignBlock, fontFamilyMap: Map<String, FontFamilyData>, event: BlockEvent) {
    when (event) {
        BlockEvent.OnDelete -> engine.delete(block)
        BlockEvent.OnBackward -> engine.sendBackward(block)
        BlockEvent.OnDuplicate -> engine.duplicate(block)
        BlockEvent.OnForward -> engine.bringForward(block)
        BlockEvent.ToBack -> engine.sendToBack(block)
        BlockEvent.ToFront -> engine.bringToFront(block)
        BlockEvent.OnChangeFinish -> engine.editor.addUndoStep()
        is BlockEvent.OnChangeBlendMode -> onChangeBlendMode(engine, block, event.blendMode)
        is BlockEvent.OnChangeOpacity -> engine.block.setOpacity(block, event.opacity)
        is BlockEvent.OnChangeFillColor -> onChangeFillColor(engine, block, event.color)
        // and so on...
    }
}

sealed class BlockEvent : Event {
    object OnChangeFinish : BlockEvent
    object OnForward : BlockEvent
    object OnBackward : BlockEvent
    object OnDuplicate : BlockEvent
    object OnDelete : BlockEvent
    object ToFront : BlockEvent
    object ToBack : BlockEvent
    data class OnChangeBlendMode(val blendMode: BlendMode) : BlockEvent
    data class OnChangeOpacity(val opacity: Float) : BlockEvent
    data class OnChangeFillColor(val color: Color) : BlockEvent
    // and so on...
}

To use the original code, you’d typically call the onEvent function with a specific event:

onEvent(BlockEvent.OnChangeFillColor(Color.RED))

This would then trigger the handleBlockEvent function to deal with the event at hand. Now, let’s embark on our first refactoring adventure.

Step 2: Unveiling HashMaps and Payloads for Peak Performance

In our first act of refactoring, we introduce a trusty HashMap to map each event type to its corresponding action. This heroic move eliminates the need for the convoluted when statement, making our code more efficient. We also unveil a payload mechanism to convey essential data to the event handlers.

Behold the refactored code:

abstract class EventsHandler<Payloads>(
    val fillPayload: (cache: Payloads) -> Unit
) {
    abstract val payloadCache: Payloads
    private val eventMap = mutableMapOf<KClass<out Event>, Payloads.(event: Event) -> Unit>()

    fun handleEvent(event: Event) {
        eventMap[event::class]?.let {
            it.invoke(payloadCache.also { fillPayload(it) }, event)
        }
    }

    operator fun <EventType : Event> set(event: KClass<out EventType>, lambda: Payloads.(event: EventType) -> Unit) {
        eventMap[event] = lambda as Payloads.(event: Event) -> Unit
    }
}

class BlockEventsHandler(fillPayload: (cache: BlockEventsHandler.Payloads) -> Unit) : EventsHandler<BlockEventsHandler.Payloads>(fillPayload) {
    class Payloads {
        lateinit var engine: Engine
        lateinit var block: DesignBlock
        lateinit var fontFamilyMap: Map<String, FontFamilyData>
    }
    override val payloadCache: Payloads = Payloads()

    init {
        it[BlockEvent.OnDelete::class] = { engine.delete(block) }
        it[BlockEvent.OnBackward::class] = { engine.sendBackward(block) }
        it[BlockEvent.OnDuplicate::class] = { engine.duplicate(block) }
        it[BlockEvent.OnForward::class] = { engine.bringForward(block) }
        it[BlockEvent.ToBack::class] = { engine.sendToBack(block) }
        it[BlockEvent.ToFront::class] = { engine.bringToFront(block) }
        it[BlockEvent.OnChangeFinish::class] = { engine.editor.addUndoStep() }
        it[BlockEvent.OnChangeBlendMode::class] = { onChangeBlendMode(engine, block, it.blendMode) }
        it[BlockEvent.OnChangeOpacity::class] = { engine.block.setOpacity(block, it.opacity) }
        it[BlockEvent.OnChangeFillColor::class] = { onChangeFillColor(engine, block, it.color) }
        // and so on...
    }
}

private val blockEventHandler = BlockEventsHandler {
    it.engine = engine
    it.block = getBlockForEvents()
    it.fontFamilyMap = checkNotNull(assetsRepo.fontFamilies.value).getOrThrow()
}

open fun onEvent(event: Event) {
    // ...
    blockEventHandler.handleEvent(event)
}

A Performance Boost

By harnessing the power of a HashMap, we’ve turbocharged our event handling. The time complexity for handling an event is now a lightning-fast (O(1)), a monumental improvement over the (O(n)) time complexity of the ponderous when statement. While our payload mechanism adds a dollop of syntactic sugar. It enables us to bundle all the necessary data into a single object, making our code more legible and maintainable.

? Note: Using a HashMap instead of a large when() statement provides a significant performance improvement. It can be up to 40 to 150 times faster. However, explaining the details would exceed the scope of this blog post. Therefore, I will cover it, along with other Kotlin performance puzzles, in a future blog post.

While the refactored code remains as simple as before:

onEvent(BlockEvent.OnChangeFillColor(Color.RED))

This still triggers the handleEvent method in BlockEventsHandler, which in turn performs the appropriate action based on the event type. The BlockEvent itself is a data object containing all event details, and it serves as the lambda parameter.

A Note on Payloads

The payload creation is a dynamic lambda function that’s executed each time an event is handled. This ensures that all variables not part of the event are consistently up-to-date. Given that we’re dealing with a single thread per event handler, caching the payload is entirely secure.

Step 3: Adding Syntactic Sweetness with Infix Functions

In our next act, we elevate our syntax to a new level of expressiveness and readability. We introduce an infix function called to, allowing us to map an event class to its corresponding action elegantly.

Witness the updated code:

abstract class EventsHandler<Payloads>(
    val fillPayload: (cache: Payloads) -> Unit
) {
    infix fun <Payloads, EventType : Event> KClass<out EventType>.to(lambda: Payloads.(event: EventType) -> Unit) {
        eventMap[event] = lambda as Payloads.(event: Event) -> Unit
    }
    // ... (rest of the code remains the same)
}

class BlockEventsHandler(
    manager: EventsManager,
    override val fillPayload: (cache: TextBlockEventsHandler) -> Unit
) : EventsHandler<TextBlockEventsHandler>(manager) {
    lateinit var engine: Engine
    lateinit var block: DesignBlock
    lateinit var fontFamilyMap: Map<String, FontFamilyData>

    init {
        BlockEvent.OnDelete::class to {
            engine.delete(block)
        }
        BlockEvent.OnBackward::class to {
            engine.sendBackward(block)
        }
        BlockEvent.OnDuplicate::class to {
            engine.duplicate(block)
        }
        BlockEvent.OnForward::class to {
            engine.bringForward(block)
        }
        BlockEvent.ToBack::class to {
            engine.sendToBack(block)
        }
        BlockEvent.ToFront::class to {
            engine.bringToFront(block)
        }
        BlockEvent.OnChangeFinish::class to {
            engine.editor.addUndoStep()
        }
        BlockEvent.OnChangeBlendMode::class to {
            onChangeBlendMode(engine, block, it.blendMode)
        }
        BlockEvent.OnChangeOpacity::class to {
            engine.block.setOpacity(block, it.opacity)
        }
        BlockEvent.OnChangeFillColor::class to {
            onChangeFillColor(engine, block, it.color)
        }
        // ...
    }
}

Syntactic Sweetness and Performance

The introduction of the to infix function adds a sprinkle of syntactic sweetness that enhances code expressiveness and enables a more natural usage. This makes it crystal clear what each event is all about. And fear not, the performance remains at a blazing-fast (O(1)), thanks to our trusty HashMap.

Flexibility in Syntax

While the to keyword is used here, feel free to substitute it with other terms like handle, trigger, or anything that best suits your context. Flexibility is the name of the game.

Step 4: Embracing Inline Functions for Elegance

However, this is still not perfect because the ::class breaks smooth reading.

So let’s do it differently. Let us try to introduce a more elegant way to register an event. Let us eliminate the need to specify ::class every time we register an event handler will make our code more concise and readable.

This is made possible by an inline function with a verified type parameter that maintains the class reference at runtime.

To do this, we extend the EventsHandler class with this new register function:

class EventsHandler(
    register: EventsHandler.() -> Unit,
) {
    inline fun <reified EventType : BaseEvent> register(noinline lambda: (event: EventType) -> Unit) : Any {
        this[EventType::class] = lambda
        return lambda
    }
   // ... (rest of the code remains the same)
}

The New Syntax

This is what registering an event handler looks like with the new syntax:

register<BlockEvent.OnChangeLineWidth> {
    engine.block.setWidth(block, engine.block.getFrameWidth(block))
    engine.block.setHeight(block, it.width)
}

Much better, right? The new syntax is more concise, eliminates redundancy, and is type-safe because the reified type parameters ensure that the event type is known at compile-time and runtime, eliminating the need for unsafe casting.

Step 5: Elevating `register` to an Extension Function for Highlighting

To improve code readability, we’ll make a subtle but effective step by converting the register function from a EventsHandler class function, into an EventsHandler extension function.

Sounds stupid! So why?

This small change improves code readability by highlighting the register keyword through syntax highlighting from a Kotlin extension function. This will make it much more colorful, which improves readability.

Updated `EventsHandler` Class

The EventsHandler class remains largely unchanged, but the register function is now outside the class and transformed into an extension function for the EventsHandler class:

class EventsHandler(
    register: EventsHandler.() -> Unit,
) {
    // ... (rest of the code remains the same)
}

inline fun <reified EventType : BaseEvent> EventsHandler.register(noinline lambda: (event: EventType) -> Unit) : Any {
    this[EventType::class] = lambda
    return lambda
}

By simply shifting register out of the class, the EventsHandler class definition now stands out with distinctive syntax highlighting. It’s a clever trick that doesn’t impact runtime or compile performance, since it’s an inline operation anyway.

**register**<BlockEvent.OnChangeLineWidth> {
    engine.block.setWidth(block, engine.block.getFrameWidth(block))
    engine.block.setHeight(block, it.width)
}

Step 6: Eliminating `lateinit` Variables with Delegated Properties

Now, it’s time to address the enigmatic lateinit variables and the somewhat convoluted fillPayload mechanism. Let us introduce a cleaner approach, using delegated properties and lambda functions to inject dependencies.

Let’s add an Inject class to wrap a normal lambda as delegable:

class Inject<Type>(private val inject: () -> Type) {
    operator fun getValue(thisRef: Any?, property: KProperty<*>): Type = inject()
}

With this newfound power, our event handler code becomes cleaner and more intuitive. It takes on the style of Jetpack Compose’s declarative syntax:

fun EventsHandler.textBlockEvents(
    engine: () -> Engine,
    block: () -> DesignBlock,
    fontFamilyMap: () -> Map<String, FontFamilyData>,
) {
    // Inject the dependencies
    val engine by Inject(engine)
    val block by Inject(block)
    val fontFamilyMap by Inject(fontFamilyMap)

    // Event handling logic here
    // ...
}

Whenever one of the variables is accessed, the lambda is called, and you always get the current variable.

Also, the creation of the “payload” becomes more straightforward, clean, and type-safe. It kinda looks like passing a variable:

private val eventHandler = EventsHandler {
    textBlockEvents (
        engine = ::engine,
        block = ::getBlockForEvents,
        fontFamilyMap = { checkNotNull(assetsRepo.fontFamilies.value).getOrThrow() },
    )
}

Looks and feels like magic! Pretty cool, right?

Step 7: Multiple Event Handlers for Single Responsibility Principle

In our grand finale, we harness the newfound flexibility from our previous changes to register multiple event handler functions. Each event handler registration function now has a specific topic, aligning perfectly with the Single Responsibility Principle (SRP).

Enhanced Event Handler Registration

We can now register multiple event handler functions within the same EventsHandler instance. Each function can specialize in handling a particular type of event, making the code more modular and manageable. Behold the grand design:

private val eventHandler = EventsHandler {
    cropEvents(
        engine = ::engine,
        block = ::getBlockForEvents,
    )
    blockEvents (
        engine = ::engine,
        block = ::getBlockForEvents,
    )
    textBlockEvents (
        engine = ::engine,
        block = ::getBlockForEvents,
        fontFamilyMap = { checkNotNull(assetsRepo.fontFamilies.value).getOrThrow() },
    )
    // ...
}

fun EventsHandler.blockEvents(
    engine: () -> Engine,
    block: () -> DesignBlock
) {
    val engine: Engine by Inject(engine)
    val block: DesignBlock by Inject(block)

    register<BlockEvent.OnDelete> { engine.delete(block) }

    register<BlockEvent.OnBackward> { engine.sendBackward(block) }

    register<BlockEvent.OnDuplicate> { engine.duplicate(block) }

    register<BlockEvent.OnForward> { engine.bringForward(block) }

    register<BlockEvent.ToBack> { engine.sendToBack(block) }

    register<BlockEvent.ToFront> { engine.bringToFront(block) }

    register<BlockEvent.OnChangeFinish> { engine.editor.addUndoStep() }

    register<BlockEvent.OnChangeBlendMode> {
        if (engine.block.getBlendMode(block) != it.blendMode) {
            engine.block.setBlendMode(block, it.blendMode)
            engine.editor.addUndoStep()
        }
    }

    register<BlockEvent.OnChangeOpacity> { engine.block.setOpacity(block, it.opacity) }
}

fun EventsHandler.cropEvents(
    engine: () -> Engine,
    block: () -> DesignBlock
) {
    val engine: Engine by Inject(engine)
    val block: DesignBlock by Inject(block)
    // ... (event handling logic for cropping events)
}

fun EventsHandler.textBlockEvents(
    engine: () -> Engine,
    block: () -> DesignBlock,
    fontFamilyMap: () -> Map<String, FontFamilyData>,
) {
    val engine by Inject(engine)
    val block by Inject(block)
    val fontFamilyMap by Inject(fontFamilyMap)
    // ... (event handling logic for text block events)
}

While the triggering and its API remain unchanged, and no extra parameters need to be passed:

open fun onEvent(event: Event) {
    eventHandler.handleEvent(event)
}

Final Words

As we conclude our journey through Kotlin code refactoring, we’ve unlocked the secrets to enhanced performance and style. By embracing techniques such as HashMaps, infix functions, and inline functions with reified type parameters, we’ve elevated our code to new heights. The benefits are clear: improved efficiency, readability, and adherence to the Single Responsibility Principle. Armed with these tools, you’re now ready to embark on your own coding adventures, transforming messy code into elegant masterpieces.

If you’d like to try it out, I’ve created a working example code on the Kotlin Playground.

Thank you for accompanying, and happy coding! Never miss out on updates and subscribe to our newsletter.

CE.SDK v1.16 Release Notes

Neslihan — Tue, 26 Sep 2023 14:26:37 GMT

It’s getting colorful! Since our last release, we’ve been busy crafting new features and enhancements to empower your user’s creative journey. We are excited to announce that CE.SDK v1.16 is now available.

With this release, you can:

Maintain Color Consistency with CMYK & Libraries

Interfaces: Engine, Editor
Platforms: Web

We understand the importance of precise color reproduction and the need for brand consistency. That’s why we’re enhancing your editing experience with greater color control.

You can now seamlessly integrate CMYK and Pantone color spaces, along with other customized color libraries, including your brand’s unique colors.

This ensures that your designs not only meet but exceed expectations across all mediums, delivering a consistent and visually engaging experience. It also helps eliminate color inconsistencies and product returns, while ensuring color accuracy.

Learn more about adding color libraries to CE.SDK in our documentation.

Improve Web Interaction with Touch Gestures

Interfaces: Engine, Editor
Platforms: Web

With this update, we’re bringing a more interactive and responsive experience to our web applications. Let your users experience seamless multitouch interactions, even on mobile web. This ensures that the content and functionality of our web applications remain accessible and user-friendly across various devices.

Thank you for reading! Never miss out on updates and subscribe to our newsletter.

CE.SDK v1.14 Release Notes

Neslihan — Fri, 25 Aug 2023 15:37:00 GMT

Since our last release, we’ve been hard at work developing new features and improvements to enhance your creative editing experience. We are excited to announce that CE.SDK v1.14 is now available!

With this release, you can:

Generate Thumbnails Fast for Smooth Performance

Interfaces: Engine
Platforms: All

We have made significant enhancements to the way video thumbnails are generated, resulting in a smoother and more efficient workflow. Our intelligent engine now renders thumbnails seamlessly across multiple frames, ensuring there are no delays or interruptions while you work. Instead of slow JPEG compression, we have implemented a faster method using raw pixel buffers that maintain high-quality results.

If you’re using web applications, the thumbnail width adapts to fit your screen, providing a seamless and enjoyable user experience. Additionally, our dynamic thumbnail generation adjusts based on the size of your clips and zoom level, optimizing resources and improving overall performance.

Enhance Templating with Improved Placeholders

Interfaces: Engine, Editor
Platforms: All

Introducing an enhanced placeholder behavior for text and image content, allowing for seamless editing and replacement. With the new PlaceholderBehavior concept, you can define whether a block should initially look and behave as a placeholder or allow editing/replacement. This feature provides a smooth editing experience for users and offers flexibility for template designers to differentiate between placeholder and desired content.

Bring Creative Editing to Your macOS App

We’re excited to announce the addition of macOS platform support to our lineup. Extend your application’s reach to macOS users and provide a seamless editing experience across different platforms. Get started, and initialize our engine in your macOS app with our documentation.

Expand Editing Capabilities: macOS & Catalyst Binding

With this new feature, we provide a comprehensive solution for developers looking to enhance their iOS applications with our CE.SDK’s editing capabilities. Learn how to initialize our creative engine in your iOS app in our documentation.

Thank you for reading! Never miss out on updates and subscribe to our newsletter.

IMG.LY Announces Open Source JavaScript Library for In-Browser Background Removal

Jan — Wed, 28 Jun 2023 11:38:49 GMT

UPDATE October 2023: We’ve recently introduced Node.js support for the @imgly/background-removal npm package. This enhancement enables you to remove backgrounds from images not only in the user’s browser but also on your server. This opens up the potential for batch processing and allows you to bypass memory constraints, especially for larger images.

Just before the summer lull engulfs us all, we have something exciting to share! We are thrilled to announce the release of @imgly/background-removal, an innovative npm package that empowers developers to seamlessly remove backgrounds from images directly in the browser.

Gone are the days of relying on server-side processing or sacrificing data privacy. With IMG.LY’s Background Removal, you can now harness the power of in-browser background removal with ease. Let’s dive into the key features that make this library truly exceptional:

In-Browser Background Removal

Our one-of-a-kind solution performs the entire background removal process directly in the user’s browser, eliminating the need for additional server costs. By leveraging the computing power of the local device, users can enjoy a fast and efficient background removal process that streamlines their workflow.

Data Protection

Rest assured that your images and sensitive information remain secure within your own devices. As IMG.LY’s Background Removal runs entirely in the browser, there are no data transfers to external servers, ensuring robust data privacy and alleviating any concerns you may have.

You can experience it in action on our background removal showcase.

Seamless Integration with IMG.LY’s CE.SDK

IMG.LY’s Background Removal seamlessly integrates with IMG.LY’s CE.SDK, making it easier than ever to incorporate powerful in-browser image matting and background removal capabilities into your projects. Boost your creative endeavors with this intuitive integration.

The Neural Network (ONNX model) and WebAssembly (WASM) files used by IMG.LY’s Background Removal are hosted on UNPKG, making them readily available for download to all users of the library. However, if you prefer to host the data on your own servers, our library also supports custom asset serving. The choice is yours!

Background removal is often the first step in any creative workflow involving image composition. Whether you are developing e-commerce applications that need real-time background removal, enhancing user experience in image editing applications, or simplifying the creative process with web-based graphic design tools, IMG.LY’s Background Removal is your go-to solution.

Empower Yourself with IMG.LY’s Background Removal

Whether you are a professional developer or a hobbyist, this open source JavaScript library empowers you to deliver impressive applications and services with ease. Join the growing community of developers who are revolutionizing the way background removal is done.

Get started with @imgly/background-removal today by visiting our official npm package page and our GitHub repository. You’ll find comprehensive documentation, examples, and everything you need to unlock the full potential of in-browser background removal.

Update: Node.js Support

We have just shipped Node.js support for the @imgly/background-removal npm package. In addition to the users’ browser, you can now remove backgrounds from images on your server, making it possible to perform batch processing or bypass memory constraints for larger images.

Thank you for your support, and we can’t wait to see the incredible projects you’ll create with IMG.LY’s Background Removal. Stay tuned for more updates and enhancements as we continue to improve and expand this powerful library. Happy coding!

Don’t forget to subscribe to our newsletter to stay in the loop with the latest news and updates.

Learning – IMG.LY Blog

How to Build a Short Video Generator Using CE.SDK

Scope

Technical Overview

Setup

Implementation

1. Generate The Script

2. Generate All Assets

3. Generate The Video

4. Add A Video Editor

Conclusion

CE.SDK v1.44 Release Notes

Simplify Your Video Editing Workflow

Get Vibrant Colors with P3/Wide Gamut Support

Tailor the Android Canvas Menu to Your Workflows

Improvements

Upcoming

How I Built a Short Video Generator with AI & CE.SDK in One Day

Why a Short Video Generator?

Enter CE.SDK

The Ground Rules

The Tools & Workflow

Building the App: Divide and Conquer

Key Takeaways

Next Steps

A Modern Vue.js Video Editor: Setup Guide

Why You Should Add a Video Editor to Your Vue.js App?

How to Integrate a Video Editor in Vue.js

Requirements

Create the Video Editor Component

Use the Component

Use Cases for the Vue.js Video Editor

Automated Marketing

Informative Videos for E-Learning

Unique Videos for Lead Outreach

Social Media Video Content Generation

Digital Asset Management

Customizing your CreativeEditor SDK Instance

Conclusion

A Modern React Video Editor: Integration Guide

Why Integrate a Video Editor in Your React Web App?

Getting Started: Adding a Video Editor in React

React Video Editor Use Cases

Customizing your CreativeEditor SDK Instance

Conclusion

A Modern Angular Video Editor: Setup Guide

Why Add a Video Editor to an Angular Application?

How to Set Up a Video Editor in Angular

Create a Video Editor Component

Use the Component

Use Cases the Your Angular Video Editor

Video Messages for Marketing Outreach

Educational Videos for E-Learning Platforms

Video Content for Social Media

Showcase Products on E-Commerce Applications

Smart and Automated Workflows

Customizing your CreativeEditor SDK Instance

Conclusion

A Modern Video Editor SDK for Your React Native App

Why Add a Video Editor to Your React Native App?

Getting Started: Integrating the Video Editor in React Native

Requirements

Setting Up the Editor

Use Cases: Building a TikTok-Like Experience

Short-Form Video Creation

Influencers and Marketing

E-commerce and User-Generated Content

Customization Options with CreativeEditor SDK

Conclusion

A Modern Video Editor SDK for your Flutter App

Why add a Video Editor to Your Flutter App?

Getting Started: Integrating the Video Editor in Flutter

Requirements

Setting Up the Editor

Use Cases: Building a TikTok-Like Experience

Short-Form Video Creation

Influencers and Marketing

eCommerce and User-Generated Content

Customization Options with CreativeEditor SDK

Conclusion