Batch Processing

In the browser, batch processing means automating the same CreativeEngine workflow while the tab stays open. Instead of the user editing/exporting items one by one, your front-end:

1. Loops through a dataset.
2. Produces a series of outputs.

This guides helps you understand how the CE.SDK can work in a batch process workflow.

What You’ll Learn#

• Two different batch processing approaches:

  • Sequential
  • Parallel

• How to batch:

  • Templates population with data.
  • Exports to different formats (PNG, JPEG, PDF, MP4).
  • Thumbnails generation.

• How to optimize memory usage.

Batch Processing Strategies#

You can run batch operations in two ways:

• Sequential: a single engine loop.
• Parallel: multiple workers spinning up.

The following examples show both approaches when running a batch export in the browser:

// ... downloadBlob logic
//Start the engine and download the scene
const engine = await CreativeEngine.init({ license: LICENSE_KEY });

for (const record of records) {
  await engine.scene.loadFromString(record.scene);
  const blob = await engine.block.export(engine.scene.getPages()[0], 'image/png');
  await downloadBlob(blob, `${record.id}.png`);
}

engine.dispose();

const workers = [new Worker('worker.js'), new Worker('worker.js')];

await Promise.all(
  records.map((record, idx) =>
    workers[idx % workers.length].postMessage({ type: 'PROCESS', record })
  )
);

The following table summarizes the pros and cons of each approach:

How To Batch Template Population#

For this operation, you generate personalized outputs at scale by combining:

• Templates
• Structured data

Set the Data Sources#

Batch workflows can use a variety of data sources to populate a template, such as:

• CSV files with parsing libraries
• JSON from REST APIs
• Databases (SQL, NoSQL)
• Stream data

The following examples show how to set three different data sources:

await fetch('path/to/dataset.json').then((r) => r.json());

await fetch('https://api.example.com/dataset').then((r) => r.json());

// Define key variables
let textVariables = {
  first_name: '',
  last_name: '',
  address: '',
  city: '',
};

Update the Template#

You can automate template population, update media, and show conditional content based on data. Find some examples in existing guides:

Batch Export the Design#

The CE.SDK provides a set of format options when exporting the edited designs:

Check all the export options in the Export section .

Batch Thumbnail Generation from Static Scenes#

The export feature allows you to automate thumbnails generation by tweaking the format and the size of the design, for example:

// Example: Real-time thumbnail generation
const thumbnailEngine = await CreativeEngine.init({ container: null });

async function generateThumbnail(sceneData) {
  await thumbnailEngine.scene.loadFromString(sceneData);
  const page = thumbnailEngine.scene.getPages()[0];

  // Generate small preview
  const thumbnail = await thumbnailEngine.block.export(page, 'image/jpeg', {
    targetWidth: 200,
    targetHeight: 200,
    quality: 0.7,
  });

  return thumbnail;
}

Read more about thumbnails generation in the Engine guide .

The CE.SDK also provides over 20 pre-designed text layouts to apply on thumbnails. Check the relevant guide to use them.

Batch Thumbnail Generation from Video Scenes#

Extract representative frames from videos efficiently, and automate this action using the dedicated CE.SDK features:

The following code shows how to generate thumbnails from a video:

import CreativeEngine from '@cesdk/engine';

const engine = await CreativeEngine.init({ license: LICENSE_KEY });
await engine.scene.loadFromURL('/assets/video-scene.scene');

const [page] = engine.scene.getPages();
const videoBlock = engine.block
  .getChildren(page)
  .find((child) => engine.block.getType(child) === 'video');

if (videoBlock) {
  const videoFill = engine.block.getFill(videoBlock);
  await engine.block.setPlaybackTime(videoFill, 4.2);

  const thumbnail = await engine.block.export(page, {
    mimeType: 'image/png',
    targetWidth: 640,
    targetHeight: 360
  });

  await downloadBlob(thumbnail, 'scene-thumb.png');
}

engine.dispose();

The preceding code:

1. Loads a scene containing a video.
2. Seeks to 4.2 s.
3. Exports the page as a PNG.
4. Saves the thumbnail.

Optimize Memory Usage#

Every export produces and accumulates:

• Blobs
• URLs
• Engine state

Proper cleanup ensures batch processes complete without resource exhaustion. Without proper cleanup, the browser might:

• Hits memory ceiling.
• Crash.
• Slow down.

Consider the following actions to avoid exhausting the client:

Treat cleanup as part of each loop iteration, by either:

• Freeing resources after each item.
• Chunking resources, by loading smaller parts of your datasets at a time.

Apply Error Handling#

Batch runs often work with large records of data. Some factors can make the job crash, such as:

• A malformed asset
• Timeouts

When your job encounters one of these errors, you can proactively avoid the job’s failure using the following patterns:

• Catch errors inside each loop iteration.
• Log failing records so you can retry them later.
• Decide whether to keep going or stop when an error happens.
• Collect a summary of all failures for post-run review.

For example, the preceding code to generate thumbnails now handles errors gracefully to avoid crashes:

import CreativeEngine from '@cesdk/engine';

let engine;
try {
  engine = await CreativeEngine.init({ license: LICENSE_KEY });
  await engine.scene.loadFromURL('/assets/video-scene.scene');

  const [page] = engine.scene.getPages();
  if (!page) throw new Error('Scene has no pages.');

  const videoBlock = engine.block
    .getChildren(page)
    .find((child) => engine.block.getType(child) === 'video');
  if (!videoBlock) throw new Error('No video block found.');

  const videoFill = engine.block.getFill(videoBlock);
  if (!videoFill) throw new Error('Video block is missing its fill.');

  await engine.block.setPlaybackTime(videoFill, 4.2);

  const thumbnail = await engine.block.export(page, {
    mimeType: 'image/png',
    targetWidth: 640,
    targetHeight: 360
  });

  await downloadBlob(thumbnail, 'scene-thumb.png');
} catch (error) {
  console.error('Failed to generate thumbnail', error);
} finally {
  engine?.dispose();
}

Use Retry Logic#

Some errors are temporary due to factors such as:

• Network hiccup
• Rate limits
• Busy CDN

To avoid saturating the related service, you can use smart retries after a short delay. If the error persist:

1. Double the delay.
2. Retry
3. Double again the delay exponentially after each retry.

This strategy allows you to identify temporary failures that could be resolved later.

For API failures, consider using circuit breaking patterns that:

• Pause the calls on repeated errors.
• Test again after a delay.

Check the Input Data Before Processing#

Lightweight checks can help you with:

• Catching bad inputs early.
• Preventing waste of time and compute on batches that’ll fail.

Add checks before:

• Launching the CE.SDK.
• Loading scenes.
• Exporting large scenes.

The following table contains some checks examples:

For example, the following data validation function checks:

• The record type
• The id
• The HTTPS template URL
• The presence of variants

It throws descriptive errors if any of these elements are missing.

function validateRecord(record) {
  if (typeof record !== 'object' || record === null) {
    throw new Error('Record must be an object');
  }
  if (typeof record.id !== 'string') {
    throw new Error('Missing record id');
  }
  if (!record.templateUrl?.startsWith('https://')) {
    throw new Error('Invalid template URL');
  }
  if (!Array.isArray(record.variants) || record.variants.length === 0) {
    throw new Error('Record requires at least one variant');
  }
  return true;
}

Batch Process on Production#

When running on production, enhance browser-based batch processes with architecture and UX decisions that help the user run the workflow, such as:

• User-initiated batches: keep work tied to explicit user actions; show confirmation dialogs for large jobs.
• Chunked processing: split datasets into small slices (for example, 20 records) to avoid blocking the main thread.
• Resource caps: document safe limits (for example, 50–100 exports per session) and enforce them in the UI.
• Persistence: use localStorage or IndexedDB to cache progress so reloads can resume work.

Monitor the Process#

Give users visibility inside the tab and send lightweight telemetry upstream.

• Render UI elements that show the state, such as:

  • Progress bars
  • Per-item status chips

• Send fetch calls to your backend for:

  • Error logs
  • Aggregated stats

• When a chunk fails:

  1. Show in-app notifications/snackbars.
  2. Offer retries.

For example, the following code:

• Structures logging.
• Renders it with timestamps.

function reportBatchMetrics(batchMetrics) {
  const entry = {
    timestamp: new Date().toISOString(),
    ...batchMetrics,
  };
  console.table([entry]);
  return fetch('/api/logs', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(entry),
  });
}

Troubleshooting#

Debugging Strategies#

Effective troubleshooting techniques for batch processing in web apps include:

• Retry with small batches.
• Console log detailed error information.
• Isolate problematic items.

Next Steps#

• Headless Mode - Learn headless engine operation basics
• Design Generation - Automate single design generation workflows
• Export Designs - Deep dive into export options and formats
• Text Variables - Work with dynamic text content in templates
• Source Sets - Specify assets sources for each block.