How to Edit Videos
In addition to static designs, CE.SDK also allows you to create and edit videos. Working with videos introduces the concept of time into the scene, which requires you to switch the scene into the "Video" mode.
In this mode, each page in the scene has its own separate timeline within which its children can be placed. The "playback/time" property of each page controls the progress of time through the page.
In order to add videos to your pages, you can add a block with a "//ly.img.ubq/fill/video" fill. As the playback time of the page progresses, the corresponding point in time of the video fill is rendered by the block.
You can also customize the video fill's trim in order to control the portion of the video that should be looped while the block is visible.
//ly.img.ubq/audio blocks can be added to the page in order to play an audio file during playback.
The playback/timeOffset property controls after how many seconds the audio should begin to play, while the duration property defines how long the audio should play. The same APIs can be used for other design blocks as well, such as text or graphic blocks.
Finally, the whole page can be exported as a video file using the block.exportVideo function.
Explore a full code sample on Stackblitz or view the code on Github.
A Note on Browser Support#
Video mode heavily relies on modern features like web codecs. A detailed list of supported browser versions can be found in our "Supported Browsers" documentation. Please also take note of possible restrictions based on the host platform browsers are running on.
Setup#
This example uses the headless Creative Engine. See the Setup article for a detailed guide.
To get started right away, you can also access the block API within a running CE.SDK instance via cesdk.engine.block.
Check out the APIs Overview to see that illustrated in more detail.
Creating a Video Scene#
First, we create a scene that is set up for video editing by calling the scene.createVideo() API. Then we create a page, add it to the scene and define its dimensions. This page will hold our composition.
Setting Page Durations#
Next, we define the duration of the page using the setDuration(block: number, duration: number): void API to be 20 seconds long. This will be the total duration of our exported video in the end.
Adding Videos#
In this example, we want to show two videos, one after the other. For this, we first create two graphic blocks and assign two 'video' fills to them.
Creating a Track#
While we could add the two blocks directly to the page and and manually set their sizes and time offsets, we can alternatively also use the track block to simplify this work. A track automatically adjusts the time offsets of its children to make sure that they play one after another without any gaps, based on each child's duration.
Tracks themselves cannot be selected directly by clicking on the canvas, nor do they have any visual representation.
We create a track block, add it to the page and add both videos in the order in which they should play as the track's children. Next, we use the fillParent API, which will resize all children of the track to the same dimensions as the page.
The dimensions of a track are always derived from the dimensions of its children, so you should not call the setWidth or setHeight APIs on a track, but on its children instead if you can't use the fillParent API.
By default, each block has a duration of 5 seconds after it is created. If we want to show it on the page for a different amount of time, we can use the setDuration API.
Note that we can just increase the duration of the first video block to 15 seconds without having to adjust anything about the second video. The track takes care of that for us automatically so that the second video starts playing after 15 seconds.
If the video is longer than the duration of the graphic block that it's attached to, it will cut off once the duration of the graphic is reached. If it is too short, the video will automatically loop for as long as its graphic block is visible.
We can also manually define the portion of our video that should loop within the graphic using the setTrimOffset(block: number, offset: number): void and setTrimLength(block: number, length: number): void APIs. We use the trim offset to cut away the first second of the video and the trim length to only play 10 seconds of the video. Since our graphic is 15 seconds long, the trimmed video will be played fully once and then start looping for the remaining 5 seconds.
We can control if a video will loop back to its beginning by calling setLooping(block: number, looping: boolean): void. Otherwise, the video will simply hold its last frame instead and audio will stop playing. Looping behavior is activated for all blocks by default.
Audio#
If the video of a video fill contains an audio track, that audio will play automatically by default when the video is playing. We can mute it by calling setMuted(block: number, muted: boolean): void.
We can also add audio-only files to play together with the contents of the page by adding an 'audio' block to the page and assigning it the URL of the audio file.
We can adjust the volume level of any audio block or video fill by calling setVolume(block: number, volume: number): void. The volume is given as a fraction in the range of 0 to 1.
By default, our audio block will start playing at the very beginning of the page. We can change this by specifying how many seconds into the scene it should begin to play using the setTimeOffset(block: number, offset: number): void API.
By default, our audio block will have a duration of 5 seconds. We can change this by specifying its duration in seconds by using the setDuration(block: number, duration: number): void API.
Exporting Video#
You can start exporting the entire page as a video file by calling exportVideo(). The encoding process will run in the background. You can get notified about the progress of the encoding process by the progressCallback. It will be called whenever another frame has been encoded.
Since the encoding process runs in the background the engine will stay interactive. So, you can continue to use the engine to manipulate the scene. Please note that these changes won't be visible in the exported video file because the scene's state has been frozen at the start of the export.