How to Use the Camera
Other than having pre-recorded video in your scene you can also have a live preview from a camera in the engine. This allows you to make full use of the engine's capabilities such as effects, strokes and drop shadows, while the preview integrates with the composition of your scene. Simply swap out the
VideoFill of a block with a
PixelStreamFill. This guide shows you how the
PixelStreamFill can be used in combination with a camera.
This example uses the headless CreativeEngine. See the Setup article for a detailed guide.
To get started right away, you can also access the
block API within a running CE.SDK instance via
Check out the APIs Overview to see that illustrated in more detail.
We create a video scene with a single page. Then we create a
PixelStreamFill and assign it to the page. To demonstrate the live preview capabilities of the engine we also apply an effect to the page.
To not waste expensive compute time by transforming the pixel data of the buffer itself, it's often beneficial to apply a transformation during rendering and let the GPU handle this work much more efficiently. For this purpose the
PixelStreamFill has an
orientation property. You can use it to mirror the image or rotate it in 90° steps.
This property lets you easily mirror an image from a front facing camera or rotate the image by 90° when the user holds a device sideways.
We use the
MediaDevices.getUserMedia() API to prompt the user for permission to use a media input video device. This can be a camera, video recording device, or a screen sharing service. To access frames of the
MediaStream we create an HTML video element. Once the video meta data is available we adjust the page size to the match the video resolution and zoom the camera to include the entire page.
Updating the Fill#
We use the video's
requestVideoFrameCallback to update the contents of the
PixelStreamFill. The callback is fired whenever a new video frame is available. We use the engine's
setNativePixelBuffer method to update the fill with the new video frame. This method is optimized to avoid unnecessary copies of the pixel data. The method accepts either an
HTMLVideoElement or an
HTMLCanvasElement as input.