Record Video

This guide covers creating a video scene with a pixel stream fill, controlling feed orientation, accessing the camera, and updating the fill with video frames.

Create a Video Scene with Camera Fill#

To display a camera feed, create a video scene and set up a page with PixelStreamFill. The pixel stream fill accepts live pixel data that you provide frame by frame:

engine.scene.createVideo();
const stack = engine.block.findByType('stack')[0];
const page = engine.block.create('page');
engine.block.appendChild(stack, page);

const pixelStreamFill = engine.block.createFill('pixelStream');
engine.block.setFill(page, pixelStreamFill);

engine.block.appendEffect(page, engine.block.createEffect('half_tone'));

We create a video scene, add a page to the stack, and assign a PixelStreamFill to receive camera data. The PixelStreamFill acts as a container that displays whatever pixel data you send to it. You can apply any CE.SDK effect to process the camera feed in real-time.

Control Feed Orientation#

The fill/pixelStream/orientation property controls how the camera feed is displayed. Use UpMirrored to horizontally flip the image, which is common for front-facing cameras to create a natural mirror-like selfie view:

// Horizontal mirroring
engine.block.setEnum(
  pixelStreamFill,
  'fill/pixelStream/orientation',
  'UpMirrored'
);

Available orientation values:

These orientations let you rotate or flip the feed without expensive CPU transformations.

Access Camera with Browser APIs#

Request camera access using the browser’s navigator.mediaDevices.getUserMedia() API. Create an HTML video element to receive the MediaStream, then update the page dimensions to match the video:

navigator.mediaDevices.getUserMedia({ video: true }).then(
  (stream) => {
    const video = document.createElement('video');
    video.autoplay = true;
    video.srcObject = stream;
    video.addEventListener('loadedmetadata', () => {
      engine.block.setWidth(page, video.videoWidth);
      engine.block.setHeight(page, video.videoHeight);
      engine.scene.zoomToBlock(page, 40, 40, 40, 40);
      // Continue with frame updates...
    });
  },
  (err) => {
    console.error(err);
  }
);

The facingMode option in getUserMedia() lets you request the front-facing camera ('user') or rear camera ('environment') on mobile devices.

Update Fill with Video Frames#

Use requestVideoFrameCallback() to efficiently sync frame updates with the video’s frame rate. Call engine.block.setNativePixelBuffer() in each callback to send the current video frame to CE.SDK:

const onVideoFrame = () => {
  engine.block.setNativePixelBuffer(pixelStreamFill, video);
  video.requestVideoFrameCallback(onVideoFrame);
};
video.requestVideoFrameCallback(onVideoFrame);

The setNativePixelBuffer() method accepts either an HTMLVideoElement or HTMLCanvasElement, providing flexibility for different video processing workflows. Using requestVideoFrameCallback instead of requestAnimationFrame ensures frame updates are synchronized with the video’s actual frame rate.

Troubleshooting#

Camera Not Appearing#

Verify camera permissions are granted in the browser. Check that the video element has autoplay set to true. Ensure the loadedmetadata event fires before accessing video dimensions.

Mirrored or Rotated Incorrectly#

Check the fill/pixelStream/orientation enum value. Front-facing cameras typically need UpMirrored. Mobile devices may require rotation adjustments based on device orientation.

Effects Not Rendering#

Confirm effects are appended using engine.block.appendEffect(). Verify the effect block was created successfully with engine.block.createEffect().

Performance Issues#

Ensure requestVideoFrameCallback is used instead of requestAnimationFrame for better frame synchronization. Check that only one callback loop is running. Consider reducing effect complexity for smoother preview.

API Reference#