Skip to main content
Contact SalesFree Trial
Contact SalesFree Trial
Platform:
Language:

Control Audio & Video

In this example, we will show you how to use the CreativeEditor SDK's CreativeEngine to configure and control audio and video through the block API.

Setup#

This example uses the headless CreativeEngine. See the Setup article for a detailed guide. To get started right away, you can also access the block API within a running CE.SDK instance via cesdk.engine.block. Check out the APIs Overview to see that illustrated in more detail.

Offset and Duration#

The offset determines when a block becomes active during playback on the scene's timeline, and the duration decides how long this block is active. Page blocks are a special case in that they have an implicitly calculated offset that is determined by their order and the total duration of their preceding pages. As with any audio/video-related property, not every block supports these properties. Use hasTimeOffset and hasDuration to check.

  • hasTimeOffset(id: number): bool returns true if the block has a time offset property.
  • setTimeOffset(id: number, TimeOffset: number): void sets the time offset of the given block in seconds.
  • getTimeOffset(id: number): number returns the time offset of the given block in seconds.
  • hasDuration(id: number): bool returns true if the block has a duration property.
  • setDuration(id: number, duration: number): void sets the playback duration of the given block in seconds.
  • getDuration(id: number): number returns the playback duration of the given block in seconds.
  • getTotalSceneDuration(id: number): number returns the total scene duration in seconds.

Trim#

You can select a specific range of footage from your audio/video resource by providing a trim offset and a trim length. The footage will loop if the trim's length is shorter than the block's duration. This behavior can also be disabled using the setLooping function.

  • hasTrim(id: number): bool returns true if trim is applicable to this block.
  • setTrimOffset(id: number, offset: number): void sets the trim offset of the given block in seconds.
  • getTrimOffset(id: number): number returns the trim offset of the given block in seconds.
  • setTrimLength(id: number, length: number): void sets the trim length of the given block in seconds.
  • getTrimLength(id: number): number returns the trim length of the given block in seconds.

Playback Control#

You can start and pause playback and seek to a certain point on the scene's timeline. There's also a solo playback mode to preview audio and video blocks individually while the rest of the scene stays frozen. Finally, you can enable or disable the looping behavior of blocks and control their audio volume.

  • setPlaying(id: number, enabled: bool): void sets whether the block should be playing during playback.
  • isPlaying(id: number): bool returns true if the block is playing during playback.
  • setSoloPlaybackEnabled(id: number, enabled: bool): void sets whether the block should play while the rest of the scene remains paused.
  • isSoloPlaybackEnabled(id: number): bool returns true if the solo playback mode is active for the given block.
  • hasPlaybackTime(id: number): bool returns true if the block has a playback time property.
  • setPlaybackTime(id: number, time: number): void sets the playback time of the given block in seconds.
  • getPlaybackTime(id: number): number returns the playback time of the given block in seconds.
  • hasPlaybackControl(id: number): bool returns true if the block supports playback control.
  • setLooping(id: number, looping: bool): void sets whether the block should restart from the beginning again or just stop.
  • isLooping(id: number): bool returns true if the block is looping.
  • setMuted(id: number, muted: bool): void sets whether the block's audio is muted.
  • isMuted(id: number): bool returns true if the block is muted.
  • setVolume(id: number, volume: bool): void sets the volume of the block in the range of [0, 1].
  • getVolume(id: number): bool returns the volume of the block in the range of [0, 1].

Resource Control#

Until an audio/video resource referenced by a block is loaded, properties like the duration of the resource aren't available, and accessing those will lead to an error. You can avoid this by forcing the resource you want to access to load using forceLoadAVResource.

  • async forceLoadAVResource(id: number): Promise<void> returns a promise that resolves once the resource has finished loading.
  • getAVResourceTotalDuration(id: number): number returns the duration in seconds of the video or audio resource referenced by the given block.

Thumbnail Previews#

For a user interface, it can be helpful to have image previews in the form of thumbnails for any given video resource. For videos, the engine can provide a single frame using getVideoFillThumbnail. Or you can obtain an atlas. That is a two-dimensional grid of frames taken at an evenly spaced interval across the entire video from left to right, top to bottom. Finally, the engine can generate a complete page composition atlas over time. The video editor uses these to visually represent and preview pages in the timeline. They can be regenerated whenever an edit is made to a page.

  • async getVideoFillThumbnail(videoFill: number, thumbnailHeight: number): Promise<Blob> returns a thumbnail encoded as JPEG. The thumbnail width will be calculated from the video's aspect ratio.
  • async getVideoFillThumbnailAtlas(videoFill: number, numberOfColumns: number, numberOfRows: number, thumbnailHeight: number): Promise<Blob> returns a thumbnail atlas encoded as JPEG. The width will be calculated from the video's aspect ratio.
  • async getPageThumbnailAtlas(videoFill: number, numberOfColumns: number, numberOfRows: number, thumbnailHeight: number): Promise<Blob> returns a thumbnail atlas encoded as JPEG. The width will be calculated from the page's aspect ratio.
File:
// Offset and Duration
engine.block.hasTimeOffset(audio);
engine.block.setTimeOffset(audio, 2);
engine.block.getTimeOffset(audio); /* Returns 2 */


engine.block.hasDuration(page);
engine.block.setDuration(page, 10);
engine.block.getDuration(page); /* Returns 10 */


engine.block.getTotalSceneDuration(scene);


// Trim
engine.block.hasTrim(videoFill);
engine.block.setTrimOffset(videoFill, 1);
engine.block.getTrimOffset(videoFill); /* Returns 1 */
engine.block.setTrimLength(videoFill, 5);
engine.block.getTrimLength(videoFill); /* Returns 5 */


// Playback Control
engine.block.setPlaying(scene, true);
engine.block.isPlaying(scene);


engine.block.setSoloPlaybackEnabled(videoFill, true);
engine.block.isSoloPlaybackEnabled(videoFill);


engine.block.hasPlaybackTime(scene);
engine.block.setPlaybackTime(scene, 1);
engine.block.getPlaybackTime(scene);


engine.block.hasPlaybackControl(videoFill);
engine.block.setLooping(videoFill, true);
engine.block.isLooping(videoFill);
engine.block.setMuted(videoFill, true);
engine.block.isMuted(videoFill);
engine.block.setVolume(videoFill, 0.5); /* 50% volume */
engine.block.getVolume(videoFill);


// Resource Control
await engine.block.forceLoadAVResource(videoFill);
engine.block.getAVResourceTotalDuration(videoFill);


// Thumbnail Previews
const videoThumb = await engine.block.getVideoFillThumbnail(videoFill, 128);
const videoThumbAtlas = await engine.block.getVideoFillThumbnailAtlas(videoFill, 4, 4, 128);
const pageThumbAtlas = await engine.block.getPageThumbnailAtlas(page, 4, 4, 128);