Web Development – IMG.LY Blog

CE.SDK v1.13 Release Notes

Neslihan — Wed, 19 Jul 2023 13:08:50 GMT

Since our last release, we’ve been busy crafting new features and enhancements to empower your creative editing journey. Moreover, we have changed the configuration of CreativeEngine and CreativeEditor SDK to make it more versatile. We are excited to announce that CE.SDK v1.13 is now available!

Enhance Video Composition: Duration and Time Offset Control

Interfaces: API
Platforms: All

Whether you want to add a captivating video overlay or include background music, the possibilities are endless. What’s included:

Time Offset and Duration control: Give each block a specific time offset and duration, allowing them to appear and disappear at desired intervals. For example, you can overlay a sticker on your video for a brief moment, creating eye-catching effects.
API integration: Currently, this feature can be controlled via API, providing flexibility and control for advanced customization.

We are thrilled to announce that this engaging feature will soon be available across all our user interfaces. Learn how to configure video and audio in documentation.

Elevate iOS Video Experience with More Codecs

Interfaces: Touch UI
Platforms: iOS

With CE.SDK’s H.265 support for iOS, you can deliver high-quality video playback on iOS devices while enjoying the benefits of improved video compression and reduced file sizes. Enhance your users’ experience with faster video streaming, enhanced storage efficiency, and exceptional visual quality.

Easily Select Blocks in Complex Designs

Interfaces: Default UI, Studio UI, Touch UI, API
Platforms: All

Selecting blocks in complex designs can be tedious and time-consuming. But with our refined selection behavior, we’ve made it effortless. With this enhancement, you can now effortlessly select and interact with elements in your designs, even in the most intricate compositions. Our advanced algorithm considers the shape of the block and the transparency of the fill, ensuring precise and intuitive selections.

Simplify Group Interaction and Enhance Mobile Usability

Interfaces: Default UI, Studio UI, Touch UI
Platforms: All

Simplify the management and editing of groups in your designs with our latest enhancements. Experience a seamless workflow on both desktop and mobile devices, as you navigate and manipulate group elements. With clear hover-states and selection visualizations, group relationships become more apparent, allowing for precise editing and arrangement of elements. Your users can achieve the perfect composition for their designs with ease.

Configure via API

Interfaces: All
Platforms: Web

We’re thrilled to introduce a more versatile approach to configuring the engine with our latest update. Now, you have the power to fine-tune every aspect of the engine’s settings using comprehensive API calls. This enhanced configurability empowers you to customize your editor precisely according to your requirements. Embrace the flexibility and control to create an editing experience that aligns perfectly with your vision. Learn more about the changes in detail and how to migrate to CE.SDK v1.13 in our documentation.

Outlook: Easily Remove Backgrounds in Your Browser

@imgly/background-removal is a powerful open-source JavaScript library that allows seamlessly background removal of images directly in the browser. With its unique features and capabilities, this package offers enhanced productivity, cost-efficiency, and data privacy while achieving stunning results. Try it out in our background removal showcase. This highly requested feature will soon be available within CE.SDK

Thank you for reading! Never miss out on updates and subscribe to our newsletter.

IMG.LY Partners with Getty Images to Revolutionize Image Integration

Klaudia — Wed, 03 May 2023 13:07:31 GMT

We are thrilled to announce our latest partnership with Getty Images, the world’s leading provider of images, clips, and illustrations for visual communication. This collaboration has brought forth the Getty Images Library SDK - a bridge that seamlessly integrates image searching, licensing, and editing into your application. This revolutionary toolkit grants access to millions of stock images and enables your teams and users to browse, license, and edit images without ever leaving your application or website.

By integrating Getty Images Library SDK into your application, you can save precious resources, streamline your workflows, and ensure legal compliance with a broad range of licensed stock images at your fingertips. Imagine not having to switch between different platforms to search, license, and edit images – all while staying in your application.

Empower your Developers and Users

At IMG.LY, we are passionate about delivering powerful editing tools with an intuitive interface, providing a creative outlet for users of all skill levels. Teaming up with Getty Images, we strive to provide a toolkit that empowers content creators to communicate their message in the most efficient and effective way possible, while upholding Getty Images’ commitment to quality and diversity in visual images.

Designed by developers, for developers, the integration is quick and easy to set up. Once integrated, users can effortlessly browse through images, license the visuals they want, and make quick edits such as cropping or rotating images, all within the same application.

We are excited to share this news and look forward to working with Getty Images to bring tools and empower creative expression!

Get Started

Learn more about the Getty Images Library SDK, how to integrate it, or contact Getty Images to obtain a license key.

How To Build a Video Editor With Wasm in React

Antonello — Mon, 16 Jan 2023 16:31:57 GMT

You may think that video processing can only be performed on a server, but this is not true. Thanks to WebAssembly (Wasm), you can run high-performance code written in C or C++ in your browser. This opens up a lot of possibilities and gives you the ability to build a client-side video editor. Let’s now learn how to create a Wasm-based video editor in React.

Follow this tutorial and learn how to build the application below:

This React video editor allows users to upload a video, select a portion of it, convert it to GIF, and download the resulting image file – all of this in your browser.

What is WebAssembly?

WebAssembly (also called Wasm) is a new type of code that modern web browsers can run and understand. Specifically, Wasm provides new functionality to web development and brings significant performance benefits. This applies to both frontend and backend, since Wasm can be used by both clients and servers.

Wasm code should not be written by hand. This is because Wasm is a binary instruction format. So, Wasm is designed to be a compilation target for source languages such as C, C++, Rust, and others. Learn more about Wasm on the WebAssembly MDN Web Docs page.

Wasm provides a way to run code written in several languages on the Web at near-native speed. This means that WebAssembly forms can be easily imported and executed by a Web client or server application. In other words, you can use WebAssembly functions via JavaScript and achieve results that were not previously possible on a browser, especially in terms of performance.

What is `ffmpeg.wasm`?

As stated on the GitHub page of the project, ffmpeg.wasm is a WebAssembly and JavaScript port of FFmpeg. If you are not familiar with it, FFmpeg is an open-source software suite that includes several libraries and programs for managing audio, video, streams, and other media files. All of these tools can be invoked and executed via the ffmpeg command-line instruction.

Therefore, ffmpeg.wasm enables video and audio processing in JavaScript-based web applications. In detail, it enables video and audio recording, format transcoding, video and audio editing, and video scaling. Even though it is a WebAssembly-based library, you can use it in your JavaScript code just like any other npm library. This is the power of Wasm.

Since it transpiles to Wasm, you can take advantage of ffmpeg.wasm directly in your browser without performance concerns. This means that ffmpeg.wasm opens the door to client-side audio and video processing. Let’s now learn how to use ffmpeg.wasm to build a client-side video editor in React!

Building a Video Editor in React with WASM

In this step-by-step tutorial, you will learn how to build a Wasm-based video editor in React with ffmpeg.wasm. This React video editor application will allow you to upload a video, trim it, and convert it to GIF in the browser, without using any backend functionality or external API.

Clone the GitHub repository that supports the tutorial and try the video editor application by launching the following commands:

git clone https://github.com/Tonel/video-editor-wasm-react
cd video-editor-wasm-react
npm install
npm run start

Let’s waste no more time and see how to build a video editor application in React.

Prerequisites

This is the list of libraries the video editor application you are about to build depends on:

Node.js and npm 8+ and higher
React 18.2+
[ffmpeg.wasm](https://github.com/ffmpegwasm/ffmpeg.wasm)
[antd](https://www.npmjs.com/package/antd) >= 4.3
[video-react](https://www.npmjs.com/package/video-react) >= 0.15

antd is one of the most popular UI libraries for React. In this tutorial, its [Slider](https://ant.design/components/slider/) component will be used to implement the video cutting feature. So, any other UI library including a slider component with a range option will do.

video-react is one of the most advanced and reliable HTML5 video players for React. You can replace it with any other React video player library.

Initializing a React Project

Let’s initialize a new Create React App React project called video-editor-wasm-react with the command below:

npx create-react-app video-editor-wasm-react

Your video-editor-wasm-react directory should now contain the following files:

video-editor-wasm-react
├── README.md
├── node_modules
├── package.json
├── .gitignore
├── public
│   ├── favicon.ico
│   ├── index.html
│   ├── logo192.png
│   ├── logo512.png
│   ├── manifest.json
│   └── robots.txt
└── src
    ├── App.css
    ├── App.js
    ├── App.test.js
    ├── index.css
    ├── index.js
    ├── logo.svg
    ├── reportWebVitals.js
    └── setupTests.js

Enter the video-editor-wasm-react folder in your terminal and launch a React local server with the commands below:

cd video-editor-wasm-react
npm start

Now, visit the [http://localhost:3000/](http://localhost:3000/) page in your browser, and you should be seeing the default Create React App screen.

Before installing ffmpeg.wasm, you need to get your server ready. As stated in the official documentation, fmmpeg.wasm depends on [SharedArrayBuffer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer). This means that only browsers that support SharedArrayBuffer can run ffmpeg.wasm. You can find a complete list of all browsers here.

In the most popular browsers, SharedArrayBuffer is only available to cross-origin isolated webpages. To enable this, you need to set the following headers in your server hosting the React application:

Cross-Origin-Embedder-Policy: require-corp
Cross-Origin-Opener-Policy: same-origin

In a Create-React-App application, you achieve this by creating a src/setupProxy.js file and making sure it contains the following code:

module.exports = function (app) {
    app.use(function (req, res, next) {
        res.setHeader("Cross-Origin-Opener-Policy", "same-origin")
        res.setHeader("Cross-Origin-Embedder-Policy", "require-corp")
        next()
    })
}

Your local development server is now ready to use ffmpeg.wasm!

Installing the Project’s Dependencies

It is now time to add the aforementioned required dependencies to your project.

First, install ffmpeg.wasm with the following command:

npm install @ffmpeg/ffmpeg @ffmpeg/core

Now, let’s install Antd Design:

npm install antd

Finally, it is time to install Video-React:

npm install video-react redux

Note that video-react also requires [redux](https://www.npmjs.com/package/redux) to work.

You have now everything you need to start developing your React video editor!

Uploading a Video to the Video Editor

Create a VideoUpload component as below:

// src/components/VideoUpload.js

import { Button, Upload } from 'antd';

function VideoUpload({ disabled, onChange = () => {}, onRemove = () => {} }) {
  return (
    <>
      <Upload
        disabled={disabled}
        beforeUpload={() => {
          return false;
        }}
        accept="video/*"
        onChange={(info) => {
          if (info.fileList && info.fileList.length > 0) {
            onChange(info.fileList[0].originFileObj);
          }
        }}
        showUploadList={false}
      >
        <Button>Upload Video</Button>
      </Upload>
      <Button
        danger={true}
        disabled={!disabled}
        onClick={() => {
          onRemove(undefined);
        }}
      >
        Remove
      </Button>
    </>
  );
}

export default VideoUpload;

As you can see, this component uses the Antd Design Upload component to allow users to upload a video file. Then, a “Remove” button gives users the ability to remove the uploaded video. Note that when the Upload component is enabled, the “Remove” button is disabled and vice versa. This way, users can deal with only one video at a time.

Playing a Video in React

You now need a VideoPlayer component to play the uploaded video. You can build it as below:

// src/components/VideoPlayer.js

import {
  BigPlayButton,
  ControlBar,
  LoadingSpinner,
  Player,
  PlayToggle,
} from 'video-react';
import 'video-react/dist/video-react.css';
import { useEffect, useState } from 'react';

export function VideoPlayer({
  src,
  onPlayerChange = () => {},
  onChange = () => {},
  startTime = undefined,
}) {
  const [player, setPlayer] = useState(undefined);
  const [playerState, setPlayerState] = useState(undefined);

  useEffect(() => {
    if (playerState) {
      onChange(playerState);
    }
  }, [playerState]);

  useEffect(() => {
    onPlayerChange(player);

    if (player) {
      player.subscribeToStateChange(setPlayerState);
    }
  }, [player]);

  return (
    <div className={'video-player'}>
      <Player
        ref={(player) => {
          setPlayer(player);
        }}
        startTime={startTime}
      >
        <source src={src} />
        <BigPlayButton position="center" />
        <LoadingSpinner />
        <ControlBar autoHide={false} disableDefaultControls={true}>
          <PlayToggle />
        </ControlBar>
      </Player>
    </div>
  );
}

This is a simple video-react-based video player component, which only allows playing or stop the video file received with the src prop.

Cutting a Video in React

Your goal is to allow users to select a portion of the video and watch it before converting it into GIF. Implement this with the logic below:

import { Slider } from "antd"
import { sliderValueToVideoTime } from "../utils/utils"

function VideoEditor() {
  const [sliderValues, setSliderValues] = useState([0, 100])

  // logic to handle videoFile, videoPlayer, and videoPlayerState...

  useEffect(() => {
      const min = sliderValues[0]
      // when the slider values are updated, updating the
      // video time
      if (min !== undefined && videoPlayerState && videoPlayer) {
          videoPlayer.seek(sliderValueToVideoTime(videoPlayerState.duration, min))
      }
  }, [sliderValues])

  useEffect(() => {
      if (videoPlayer && videoPlayerState) {
          // allowing users to watch only the portion of
          // the video selected by the slider
          const [min, max] = sliderValues

          const minTime = sliderValueToVideoTime(videoPlayerState.duration, min)
          const maxTime = sliderValueToVideoTime(videoPlayerState.duration, max)

          if (videoPlayerState.currentTime < minTime) {
              videoPlayer.seek(minTime)
          }
          if (videoPlayerState.currentTime > maxTime) {
              // looping logic
              videoPlayer.seek(minTime)
          }
      }
  }, [videoPlayerState])

  return (
          // other video editor components...
          <Slider
            disabled={!videoPlayerState}
            value={sliderValues}
            range={true}
            onChange={(values) => {
              setSliderValues(values)
            }}
            tooltip={{
              formatter: null,
            }}
          />
        // ...
}

This component equips users with the ability to select a portion of the uploaded video with the Antd Design Slider component and watch it in loop, before converting it into a GIF file.

Specifically, the logic here uses the sliderValueToVideoTime() function below to transform the slider values into time coordinates:

// src/utils/utils.js

export function sliderValueToVideoTime(duration, sliderValue) {
    return Math.round(duration * sliderValue / 100)
}

Note that the actual video processing slider will be performed by the next component through ffmpeg.

Converting a Video to GIF in React

To convert a video to GIF, create a VideoConversionButton component as follows:

// src/components/VideoConversionButton.js

import { Button } from 'antd';
import { fetchFile } from '@ffmpeg/ffmpeg';
import { sliderValueToVideoTime } from '../utils/utils';

function VideoConversionButton({
  videoPlayerState,
  sliderValues,
  videoFile,
  ffmpeg,
  onConversionStart = () => {},
  onConversionEnd = () => {},
  onGifCreated = () => {},
}) {
  const convertToGif = async () => {
    // starting the conversion process
    onConversionStart(true);

    const inputFileName = 'gif.mp4';
    const outputFileName = 'output.gif';

    // writing the video file to memory
    ffmpeg.FS('writeFile', inputFileName, await fetchFile(videoFile));

    const [min, max] = sliderValues;
    const minTime = sliderValueToVideoTime(videoPlayerState.duration, min);
    const maxTime = sliderValueToVideoTime(videoPlayerState.duration, max);

    // cutting the video and converting it to GIF with an FFMpeg command
    await ffmpeg.run(
      '-i',
      inputFileName,
      '-ss',
      `${minTime}`,
      '-to',
      `${maxTime}`,
      '-f',
      'gif',
      outputFileName
    );

    // reading the resulting file
    const data = ffmpeg.FS('readFile', outputFileName);

    // converting the GIF file created by FFmpeg to a valid image URL
    const gifUrl = URL.createObjectURL(
      new Blob([data.buffer], { type: 'image/gif' })
    );
    onGifCreated(gifUrl);

    // ending the conversion process
    onConversionEnd(false);
  };

  return <Button onClick={() => convertToGif()}>Convert to GIF</Button>;
}

export default VideoConversionButton;

This component uses ffmpeg.wasm to launch an ffmpeg command. This command takes care of clipping the video to the time limits defined by sliderValues and converting it to GIF. Note that all these operations are performed client-side in the browser.

In detail, the ffmpeg -ss flag defines the time from which to start reading the video file, while -to defines the end time. Then, the gif option specifies that you want to produce a GIF file. Learn more about the flags and options offered by ffmpeg here.

The resulting GIF image is stored in the output.gif memory file, loaded as a valid "image/gif" Blob, and finally converted to a valid image URL with the [URL.createObjectURL()](https://developer.mozilla.org/en-US/docs/Web/API/URL/createObjectURL_static) native function.

Putting It All Together

Now, let’s see the full code of the VideoEditor component:

// src/components/VideoEditor.js

import { createFFmpeg } from '@ffmpeg/ffmpeg';
import { useEffect, useState } from 'react';
import { Slider, Spin } from 'antd';
import { VideoPlayer } from './VideoPlayer';
import { sliderValueToVideoTime } from '../utils/utils';
import VideoUpload from './VideoUpload';
import VideoConversionButton from './VideoConversionButton';
const ffmpeg = createFFmpeg({ log: true });

function VideoEditor() {
  const [ffmpegLoaded, setFFmpegLoaded] = useState(false);
  const [videoFile, setVideoFile] = useState();
  const [videoPlayerState, setVideoPlayerState] = useState();
  const [videoPlayer, setVideoPlayer] = useState();
  const [gifUrl, setGifUrl] = useState();
  const [sliderValues, setSliderValues] = useState([0, 100]);
  const [processing, setProcessing] = useState(false);

  useEffect(() => {
    // loading ffmpeg on startup
    ffmpeg.load().then(() => {
      setFFmpegLoaded(true);
    });
  }, []);

  useEffect(() => {
    const min = sliderValues[0];
    // when the slider values are updated, updating the
    // video time
    if (min !== undefined && videoPlayerState && videoPlayer) {
      videoPlayer.seek(sliderValueToVideoTime(videoPlayerState.duration, min));
    }
  }, [sliderValues]);

  useEffect(() => {
    if (videoPlayer && videoPlayerState) {
      // allowing users to watch only the portion of
      // the video selected by the slider
      const [min, max] = sliderValues;

      const minTime = sliderValueToVideoTime(videoPlayerState.duration, min);
      const maxTime = sliderValueToVideoTime(videoPlayerState.duration, max);

      if (videoPlayerState.currentTime < minTime) {
        videoPlayer.seek(minTime);
      }
      if (videoPlayerState.currentTime > maxTime) {
        // looping logic
        videoPlayer.seek(minTime);
      }
    }
  }, [videoPlayerState]);

  useEffect(() => {
    // when the current videoFile is removed,
    // restoring the default state
    if (!videoFile) {
      setVideoPlayerState(undefined);
      setSliderValues([0, 100]);
      setVideoPlayerState(undefined);
      setGifUrl(undefined);
    }
  }, [videoFile]);

  return (
    <div>
      <Spin
        spinning={processing || !ffmpegLoaded}
        tip={!ffmpegLoaded ? 'Waiting for FFmpeg to load...' : 'Processing...'}
      >
        <div>
          {videoFile ? (
            <VideoPlayer
              src={URL.createObjectURL(videoFile)}
              onPlayerChange={(videoPlayer) => {
                setVideoPlayer(videoPlayer);
              }}
              onChange={(videoPlayerState) => {
                setVideoPlayerState(videoPlayerState);
              }}
            />
          ) : (
            <h1>Upload a video</h1>
          )}
        </div>
        <div className={'upload-div'}>
          <VideoUpload
            disabled={!!videoFile}
            onChange={(videoFile) => {
              setVideoFile(videoFile);
            }}
          />
        </div>
        <div className={'slider-div'}>
          <h3>Cut Video</h3>
          <Slider
            disabled={!videoPlayerState}
            value={sliderValues}
            range={true}
            onChange={(values) => {
              setSliderValues(values);
            }}
            tooltip={{
              formatter: null,
            }}
          />
        </div>
        <div className={'conversion-div'}>
          <VideoConversionButton
            onConversionStart={() => {
              setProcessing(true);
            }}
            onConversionEnd={() => {
              setProcessing(false);
            }}
            ffmpeg={ffmpeg}
            videoPlayerState={videoPlayerState}
            sliderValues={sliderValues}
            videoFile={videoFile}
            onGifCreated={(girUrl) => {
              setGifUrl(girUrl);
            }}
          />
        </div>
        {gifUrl && (
          <div className={'gif-div'}>
            <h3>Resulting GIF</h3>
            <img
              src={gifUrl}
              className={'gif'}
              alt={'GIF file generated in the client side'}
            />
            <a
              href={gifUrl}
              download={'test.gif'}
              className={'ant-btn ant-btn-default'}
            >
              Download
            </a>
          </div>
        )}
      </Spin>
    </div>
  );
}

export default VideoEditor;

Since ffmpeg takes time to load, you need to let users know that they need to wait a while before they can use the video editor component. Also, ffmpeg operations take time to execute. This wait logic is implemented by using the Antd Design [Spin](https://ant.design/components/spin/) component.

Also, note that by passing the log: true option to the createFFmpeg() function, ffmpeg will log useful debug info in the console. For example, after ffmpeg gets loaded, you should be able to see the following lines in the browser console:

[info] use ffmpeg.wasm v0.11.5
createFFmpeg.js:43 [info] load ffmpeg-core
createFFmpeg.js:43 [info] loading ffmpeg-core
createFFmpeg.js:43 [info] ffmpeg-core loaded

What the VideoComponent does is use all the components presented earlier to allow users to upload a video, select a portion through a slider, convert it to GIF, and download the resulting GIF file.

Et voilà! You just learned how to build a client-side video editor in React without using any backend functionality!

Commercial Alternative

VideoEditor SDK by IMG.LY provides powerful video editing features, including cropping and trimming videos in your project. You will receive staples of video editing, including straightening videos, filters, brightness, color adjustments, and more.

Conclusion

Building a client-side video processing and editing application that uses only the browser possible. With WebAssembly, you can run high-performance code written in C or C++ in JavaScript. This allows you to perform complex operations at lightning speed directly in the browser, without having to delegate the work to a server.

In this tutorial, you learned how to use ffmpeg.wasm to build a video editor in React. ffmpeg.wasm is the Wasm port of ffmpeg and enables video and audio processing and editing directly in the browser. Using it to build a web application to upload a video, cut it, and export it to GIF requires only a few lines of code, and here we have seen how to implement such a video editing application.

If your app goes beyond merely displaying video, and you want to allow your users to also edit video or create video based templates in the browser, explore our VideoEditor SDK for the Web – a performant video editing solution based on WASM and WebCodecs. Try the editor for yourself in action with our showcases.

Thanks for reading! Let us know what you think on Twitter! To stay in the loop, subscribe to our Newsletter.

CE.SDK v1.9.0 Release

Neslihan — Tue, 13 Dec 2022 08:21:50 GMT

We are happy to shine light on more new features since our CE.SDK v1.8.0 Release. To build powerful features for your application, we are excited to hear your feedback and suggestions.

CE.SDK v1.9.0 is out now, and it includes:

Video Support for Web
Video Timelines
Web Audio Support
Native iOS Platform Support
Swift Package
Common Touch Gesture Support
Apparel UI Showcase
Spot and Named Colors
More Showcases

Video Support for Web

We now support adding and editing videos in your web browser. You can apply video fills, add blocks to the scene, and combine any other block to create videos and stories with CE.SDK. For now this feature is only supported in Chromium based browsers.

Video Timelines

Video Timelines and Trimming is coming to our Studio and Design UIs. This adds complete client-side video editing in the browser to our User Interfaces.

Web Audio Support

In addition to video support, we also introduce a new block type for audio. You can use audio blocks to add audio files in MP3 and M4a formats to your scene. Like design blocks, you can add and modify them via the Block APIs and attach them to a scene. Note that audio blocks have no visual representation on the canvas, but only in the timeline.

We dedicate our first implementation to the web. However, native mobile implementations will follow soon.

Native iOS Support

With this feature, we support developing native iOS applications with our SDK. It comes with full Swift API support and many integration examples.

Additionally, CE.SDK v1.9.0 comes with Common Touch Gesture to improve the editing experience on touchpads and mobile devices. That includes gestures like pinch, spread, drag, rotate, tap, and double tap.

Spot and Named Colors

With this release we are extending our support for print use cases by adding spot colors which enable the use of custom color mixes in the printing process. Consult the guides on creating spot colors for more information.

More Showcases

To explore CE.SDK capabilities firsthand, and to look into cases that might be comparable to yours, we provide more showcases on our website.

Easily build a

Mobile UI
Photo UI for easy photo editing
Post & Greeting Card UI to provide templates and easy adaption for print
Apparel UI to let users create fantastic apparel designs
Apparel UI Mobile.

Thanks for reading! Let us know what you think on Twitter! To stay in the loop, subscribe to our Newsletter.

How to Apply Filters and Effects to an Image in Flutter

Natalia — Tue, 15 Nov 2022 10:51:25 GMT

Introduction

Having previously discussed how Flutter deals with resizing and various overlays, in this tutorial, we will talk about a more popular type of image manipulation; applying filters and effects. At least since the ubiquity of social media platforms such as Instagram, Snapchat or more recently TikTok, applying various filters and effects has become an expected feature of any application dealing with image manipulation.

Applying Filters and Effects in Flutter demo application

In this tutorial, we go through the process of applying filters and effects to images in Flutter. Like the example given in the video, we will examine the options for developing an image editor and the effects available in the Flutter framework. The Git repository supporting this article can be cloned with the command provided below:

git clone git@github.com:nataliakzm/Applying_Filters_and_Effects_to_Images_Flutter.git

One of the leading solutions proposed by the Flutter team on this issue was the ImageFiltered class. Overall, the widget deals with major image manipulations (from blurring to matrix transformation, rotation etc.), transforming and rearranging image pixels. To create an ImageFilter, you need to follow a particular structure and apply the ImageFilter class to its child elements:

Moreover, in order to start working with ImageFiltered within your Flutter application, first, don’t forget to import this library on the top of your .dart file.

const ImageFiltered({
  super.key,
  required this.imageFilter,
  super.child,
  this.enabled = true,
}) : assert(imageFilter != null);

import 'dart:ui';

Then, ImageFiltered offers several filter choices for image processing, depending on the specification of your app or visual asset.

So, for example, with the help of ImageFilter.blur or the so-called Gaussian blur, the pixels of the image can be blurred, which can, for instance, be used to create a graphic element that can serve as a background image.
On the other hand, two other filters such as ImageFilter.dilate and ImageFilter.erode affect the pixels’ value by either enlarging them to the max significance within the specified range along the x and y axes or by shrinking pixels to their minimum values;
You can also easily rotate, scale and change your asset in other ways by transforming its matrix with ImageFilter.matrix;
Finally, ImageFilter.compose is handy for merging two filters and combining their effects.

At this point, the purpose of each effect seems much clearer, but we still miss some crucial steps to implement these filters in your Flutter application. Let’s see some real examples:

ImageFilter.blur constructor

To apply a blurring effect on your image, you have to call an ****ImageFiltered.blur(), and adjust the values of sigmaX and sigmaY arguments for blurring the image. Note that the asset must be called as a child element of the filter to achieve the desired effect.

Container(
		child: Center(
				child: Column(
						mainAxisAlignment: MainAxisAlignment.center,
						children: [
      // Blur an Image
      ImageFiltered(
					imageFilter: ImageFilter.blur(sigmaY: 5, sigmaX: 5),
					child: Image.asset("assets/logo.png"))

ImageFilter.matrix constructor

The situation is slightly different with ImageFilter.matrix filter since the effect can be initialized by one of two methods – Float64List.fromList or Matrix4.rotationZ . Thus, for the first method, the user needs to directly provide the matrix as a Float64List as it is done below:

// Matrix Transformation
ImageFiltered(
		imageFilter: ImageFilter.matrix(Float64List.fromList([
			1, 0.5, 0.0, 0.0,
			0.0, 1, 0.0, 0.0,
			0.0, 0.0, 1.0, 0.0,
			0.0, 0.0, 0.0, 1.0,
    ])),
		child: Image.asset("assets/logo.png", scale: 3.5)) // Note that scale is optional and depends on the image size

While the Matrix4 class simply provides rotation in Z directions:

// Matrix4 Rotation
ImageFiltered(
		imageFilter: ImageFilter.matrix(Matrix4.rotationZ(0.2).storage),
		child: Image.asset("assets/logo.png", scale: 6)) //Note that scale is optional and depends on the image size

ColorFiltered Effects

Alternatively, Flutter has several options to integrate some filters without calling any extra library. One such example is the ColorFilter function executed within the widget of the same name. ColorFiltered processes two colors and outputs only one of them, which is eventually displayed at the top of the layer.

 // Black & White Effect
ColorFiltered(
		colorFilter: const ColorFilter.mode(Colors.grey, BlendMode.saturation),
		child: Image.asset("assets/logo.png"))

Users can also engage with several construction methods: from directly calling the sRGB gamma curve with ColorFilter.linearToSrgbGamma() widget or its inverse prototype with ColorFilter.srgbToLinearGamma() to transforming the color by a 5x5 matrix within ColorFilter.matrix() function or simply applying the blend mode with a given color specified in ColorFilter.mode().

Final Thoughts

In conclusion, Flutter’s image processing capabilities are limited by a small variety of effects and often depend on the developer’s ability to handle complex concepts like matrix or sRGB curves. In this article, we reviewed some of the main image processing techniques proposed by the Flutter team and examined how to integrate them into photo editing apps. However, it is not always possible to allocate several days to calculate the correct matrix for transforming just one image when it comes to processing a large number of graphic assets. In such cases, you may want to think about using the PhotoEditor SDK for your next Flutter app. Just like in the interactive example below, PhotoEditor SDK has many different effects and filters that will suit any of your requests.

Applying Filters via PhotoEditor SDK

To learn more, check out our article on PhotoEditor SDK and how to set it up in your app and official PE.SDK guides. You can also follow our guide on how to integrate video editor for Flutter into your app. For any of your questions that might occur in the process, you can contact our Support, who will be glad to help you.

How to Draw on Images in React Native

Natalia — Thu, 20 Oct 2022 13:09:38 GMT

It has become a commonplace requirement for many applications to manipulate images in one way or another before users are ready to share them. Therefore, it is crucial to foresee end-user needs while working on an app involving any interaction with graphical content. Image annotation can be necessary in various cases, such as simple inspection apps, bug reporting software, or machine learning applications relying on manual user input.

In this article, we are shedding light on how to draw on images in React Native with the help of a simple HTML5

Enable the characteristics of the
Set the thickness and color of the brush instrument;
Define at what moment your brush needs to be activated and when the drawing process will finish;
Start drawing!

Let’s first clear up the purpose of

Canvas Integration for Images

<canvas> is an HTML component commonly used for creating and manipulating graphics, animation, and other visualization features. This element provides JavaScript APIs enabling image overlays and handling user input to make drawing possible. Moreover, while we have several methods for image manipulation with

<! -- HTML example of using canvas: -->
<canvas id="myCanvas" width="200" height="100"></canvas>

You might wonder why you cannot easily substitute <canvas> with some lines of HTML and CSS in your code. The answer lies on the surface: <canvas> creates a single flattened graphic rather than multiple components lying on top of each other (i.e., typically the output of HTML and CSS).

Now, let’s see how to implement this powerful element in your React Native application:

Drawing with Canvas

import React, { useEffect } from 'react';

// CanvasContext here is used for drawing onto the canvas
import { useCanvas } from './CanvasContext';

export function Canvas() {
  const { canvasRef, prepareCanvas, startDrawing, finishDrawing, draw } =
    useCanvas();

  useEffect(() => {
    prepareCanvas();
  }, []);

  return (
    <canvas
      onMouseDown={startDrawing}
      onMouseUp={finishDrawing}
      onMouseMove={draw}
      ref={canvasRef}
    />
  );
}

We start by enabling the characteristics of our <canvas> container, which are meant to render the CanvasContext component presented below.

The size of our future canvas is determined with prepareCanvas, which includes both the height (canvas.height) and width (canvas.width) of the HTML element. For the drawing process itself, it is necessary to assign not only the thickness and color of the brush instrument (context) but also take into account its activation (startDrawing with onMouseDown) and how the user can finish a stroke (finishDrawing with onMouseUp).

import React, { useContext, useRef, useState } from 'react';

// enabling drawing on the blank canvas
const CanvasContext = React.createContext();

export const CanvasProvider = ({ children }) => {
  const [isDrawing, setIsDrawing] = useState(false);
  const canvasRef = useRef(null);
  const contextRef = useRef(null);

  // defining width & height of the canvas
  const prepareCanvas = () => {
    const canvas = canvasRef.current;
    canvas.width = window.innerWidth * 2;
    canvas.height = window.innerHeight * 2;
    canvas.style.width = `${window.innerWidth}px`;
    canvas.style.height = `${window.innerHeight}px`;

    // defining the thickness and colour of our brush
    const context = canvas.getContext('2d');
    context.scale(2, 2);
    context.lineCap = 'round';
    context.strokeStyle = 'black';
    context.lineWidth = 5;
    contextRef.current = context;
  };

  const startDrawing = ({ nativeEvent }) => {
    const { offsetX, offsetY } = nativeEvent;
    contextRef.current.beginPath();
    contextRef.current.moveTo(offsetX, offsetY);
    setIsDrawing(true);
  };

  const finishDrawing = () => {
    contextRef.current.closePath();
    setIsDrawing(false);
  };

  const draw = ({ nativeEvent }) => {
    if (!isDrawing) {
      return;
    }
    const { offsetX, offsetY } = nativeEvent;
    contextRef.current.lineTo(offsetX, offsetY);
    contextRef.current.stroke();
  };

  // Once the canvas is cleared it return to the default colour
  const clearCanvas = () => {
    const canvas = canvasRef.current;
    const context = canvas.getContext('2d');
    context.fillStyle = 'white';
    context.fillRect(0, 0, canvas.width, canvas.height);
  };

  return (
    <CanvasContext.Provider
      value={{
        canvasRef,
        contextRef,
        prepareCanvas,
        startDrawing,
        finishDrawing,
        clearCanvas,
        draw,
      }}
    >
      {children}
    </CanvasContext.Provider>
  );
};

export const useCanvas = () => useContext(CanvasContext);

Drawbacks of the Canvas Method

Despite the canvas component offering powerful features, it is also quite demanding to use in its original form. For example, it may be problematic to adjust the properties of the brush to ensure that you can draw with smooth movements. Thus, implementing any solution for this issue would further increase the coding time and reduce the compatibility between different frameworks. Also, separately defining canvas and brush properties overcomplicates the code and requires advanced sources for successful integration into your React Native app. Although some temporary solutions aim to facilitate this process (such as react-native-image-draw or react-native-sketch-draw), none provides a permanent solution with clear documentation and guidance.

If you wonder about a magic door to avoid this coding nightmare, you can always choose a commercial, all-in-one solution like PhotoEditor SDK. Implementing this SDK only takes a few lines of code, saves time and expenses in building an app, and you can expect great support from the developers. This way, you can skirt the hassle of coding an editor and instead focus on creating a great product.

PhotoEditor SDK Integration for Drawing on Your Images

To get started integrating the PhotoEditor SDK in your React Native app, refer to this guide from the official documentation. You can then ensure comfortable image interaction for your users with the optimized brush tool, as in the example below.

Conclusion

This article aimed to examine image editing instruments suitable for React Native framework. The HTML <canvas> element could have big potential as it provides fundamental tools for creating graphics or animation. However, its application is time-consuming and requires comprehensive knowledge in the field. Therefore, if you want to guarantee a smooth and easy integration of image drawing tools for your app, consider using advanced software like PhotoEditor SDK.

CE.SDK v1.8.0 Release

Neslihan — Tue, 18 Oct 2022 14:48:05 GMT

Since our CE.SDK v1.7.0 Release, we implemented more highly requested features. Today, we shine light on them. As always, we are excited to hear your feedback and suggestions to build powerful features for your application. Our Product Roadmap is the best place to make your case and be the first one to know of future releases.

CE.SDK v1.8.0

This release includes:

Node.js Platform Support
Common Touch Gesture Support for iOS
Emoji Support ?
Image Straightening in Advanced & Default UI
Multi Selection Rotation
Outlook

Node.js Platform Support

Since the release of our Node Package, you can run a headless version of CE.SDK on the server. Now with this release, you have the full power of our Creative Engine at your fingertips, enabling you to programmatically generate images on the server or command line, render scenes created with the web SDK, and implement any creative automation workflow.

Common Touch Gesture Support for iOS

We know that a mobile presence and functionality is crucial for your application. If users find it difficult to complete a task or are frustrated by poor usability, they could quickly bid adieu to your app and ditch your hard work for a competitor. So with this update, we add common touch gesture to improve the editing experience on touchpads and mobile devices. This includes gestures like pinching, spreading, dragging, rotating, tap and double tapping.

Emoji Support

Sometimes, a party face says more than a thousand words ?: Emojis have become an essential part of modern communication and are omnipresent, especially on mobile platforms, but on desktop as well. We added Emoji rendering support for more fun edits and colorful messaging.

Image Straightening in Advanced & Default UI

While cropping your image, you can now also straighten, flip and rotate it! This feature gives you full control over your image in one place – available in both Advanced and Default User Interfaces.

Multi Selection Rotation

For an accelerated design and editing process, the engine and user interface now allow rotating several blocks at once. To multi-rotate, simply hold your Shift key while selecting the blocks you would like to move simultaneously.

Outlook

Native iOS Platform Support: With this feature, we will add support for developing native iOS applications with the SDK. It will come with full Swift API support and many integration examples.

Video for Web with CreativeEditor SDK: Users will be able to edit and create fantastic videos based on templates in their browser with our SDK in your application soon! Sign up for early access.

Thanks for reading! To stay in the loop, feel free to subscribe to our Newsletter.

How To Build a Video Player in JavaScript

Paul — Tue, 27 Sep 2022 17:02:49 GMT

Probably a decade ago, it was impossible to play video or audio inside your browser without any third-party services such as Flash or Silverlight. You needed to install a plugin and only play your media while using it, so as you can see, it was very uncomfortable, with low speed and high delays. Nowadays, we have JavaScript with the new version of HTML5. With these new technologies and tools, we can stream our video much quicker, easier, and without any latency. To do it, you will need only a simple tag and give a link to your video stored on your computer. The simple attribute controls will give you a default video player built into the browser. It’s elementary and doesn’t have many features, so if you want to stream a video on your website in a more professional way using your video player, you’ll need to use JavaScript. And we’ll teach you how to do it in this article!

By the end of this guide, you’ll have something similar to this, so if you’re excited, keep reading and follow this tutorial step-by-step!

See the Pen How to build a video player in Javascript by Paul Knulst (@paulknulst) on CodePen.

Setting Up the Project

Assuming you are working with UNIX system (or have Git BASH on Windows) you can create all three files that are necessary to build a video player in JavaScript with this command:

mkdir video-player
cd video-player
touch index.html script.js style.css

To add a simple video player to our application, we have to add the following code to our index.html:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>How to build a video player in Javascript</title>
        <link rel="stylesheet" href="style.css" />
    </head>
    <body>
        <div class="player">
            <video class="video" controls>
                <source
                    src="<https://ftp.f1nalboss.de/data/imgly/videoplayer/testvideo.mp4>"
                    type="video/mp4"
                />
                <source
                    src="<https://ftp.f1nalboss.de/data/imgly/videoplayer/testvideo.mp4>"
                    type="video/webm"
                />
                <p>No HTML5 video supported</p>
            </video>
        </div>
        <script src="script.js"></script>
    </body>
</html>

Within the above code, the <video> element uses a remote video from my FTP. You can either use my default video or add any video from your local computer by adjusting the src attribute. HTML5 specification supports three different video formats, and the snippet used multiple <source> tags to make the videos available in MP4 and WebM. Furthermore, the <p> tag is used to display pre-defined content to user agents that do not support the video element.

The HTML5 <video> tag accepts several native attributes. For example, the controls attribute displays the standard player controls when added or set to true. You can find out more about all video attributes here.

Before continuing, you should apply all styles that are needed within this tutorial by populating your style.css with all styles from this CodePen. Save and open your index.html and load it within the browser to see the embedded video player as seen below:

Customize the Video Player With JavaScript

To customize the video player, you first have to remove the controls attribute that displays Play, Pause, Volume, etc. because you will implement your own custom controls within this tutorial. Now, check your browser, you will recognize that the controls are gone, and you cannot play the video anymore.

Add Play and Pause

To enable play and pause the video, you have to add a new button to the index.html:

<div class="controls">
    <button
            class="controls__btn playPauseBtn"
            title="Toggle Play"
            >
        ►
    </button>
</div>

Afterward, open your script.js and enable functionality by adding this code:

const videoContainer = document.querySelector(".video-container");
const playPauseBtn = document.querySelector(".playPauseBtn");
function togglePlay() {
  if (videoContainer.paused || videoContainer.ended) {
    videoContainer.play();
  } else {
    videoContainer.pause();
  }
}
function updatePlayBtn() {
  playPauseBtn.innerHTML = videoContainer.paused ? "►" : "❚❚";
}
playPauseBtn.addEventListener("click", togglePlay);
videoContainer.addEventListener("click", togglePlay);
videoContainer.addEventListener("play", updatePlayBtn);
videoContainer.addEventListener("pause", updatePlayBtn);

Within this javascript code, first, the video-container element and the playPauseBtn is selected (Line 1 and 2). Then two functions are defined: togglePlay() and updatePlayBtn(). togglePlay() is used to stop and start the video based on its actual state. updatePlayBtn is used to switch between the Icon which is shown within the video player.

In the last part of the snippet, a click event listener is added to the playPauseBtn that executes the togglePlay() function. Next, three click event listeners are added to the videoContainer that executes togglePlay() on mouse click and also executes updatePlayBtn based on the video’s state.

Now you can reload your index.html and should be able to play and pause the video by either clicking the video or the button:

Add Progress Bar

Next, a progress bar will be implemented to show the current timestamp of the video when played. First, add a div tag to the index.html which will act as the progress bar:

<div class="controls">
    <div class="progress">
    	<div class="progress__filled"></div>
    </div>
    // ..
</div>

Then open the script.js and add the following snippet:

const progress = document.querySelector(".progress");
const progressBar = document.querySelector(".progress__filled");

function handleProgress() {
  const progressPercentage = (videoContainer.currentTime / videoContainer.duration) * 100;
  progressBar.style.flexBasis = `${progressPercentage}%`;
}

function jump(e) {
  const position = (e.offsetX / progress.offsetWidth) * videoContainer.duration;
  videoContainer.currentTime = position;
}

videoContainer.addEventListener("timeupdate", handleProgress);
progress.addEventListener("click", jump);
let mousedown = false;
progress.addEventListener("mousedown", () => (mousedown = true));
progress.addEventListener("mousemove", (e) => mousedown && jump(e));
progress.addEventListener("mouseup", () => (mousedown = false));

In this snippet, the progress container and the progress__filled element will be selected, and two functions will be added: handleProgress() and jump(e). handleProgress() will be responsible for updating the progress bar. The jump(e) function is used to enable clicking on the progress bar to jump to the position within the video.

The last part contains all event listeners that are needed for the progress bar. The handleProgress() will be called on every timeupdate event. Also, clicking anywhere on the progress bar will call the jump(e) method and the video will jump to the position. Additionally, mousedown, mousemove, and mouseup will be used to enable sliding through the video while holding the mouse button down on the progress bar.

Closing Notes

Congratulations! If you followed the tutorial, you learned how to implement your own video player and add custom controls using JavaScript. Now, you can use my CodePen and start implementing more controls like volume control, keyboard shortcuts, or skip controls to build your own customized video player.

If your app goes beyond merely displaying video, and you want to allow your users to also edit video or create video based templates in the browser, check out our Video Editor for Web!

CE.SDK v1.7.0 Release and Roadmap

Malte — Mon, 08 Aug 2022 12:07:47 GMT

Today, we are thrilled to highlight the best new features and changes in CE.SDK since our v1.6.0 Release. We have also published our Product Roadmap for CE.SDK to share our vision and direction for upcoming releases. This roadmap will let you know ahead of time what features you can expect from us and accelerate your product planning and development. We are excited to hear your feedback and suggestions to build more powerful features for your application!

CE.SDK v1.7.0

This release includes:

Auto Close Option in Web UI Configuration for Asset Libraries
Custom URI Resolver
Improving Mobile Touch Support
Rotation of Groups and Hierarchies
Custom Asset Source UI
Disconnecting UI Variants from Roles
Asset Source ApplyAsset API
Blur API
Effects API
Image Fills API
Image Straighten API
Improved Group Selection and API
Scopes API
Shadows API
Zoom API

Auto Close Option in Web UI Configuration for Asset Libraries

There are no one-size-fits-all solutions for the behavior of asset library panels after an action was triggered. It depends on the use case and might vary between the panels and the available screen space. Thus, this is now configurable.

Custom URI Resolver

Custom URI Resolvers give you full control over how URIs should be resolved. You can create custom storage backends and even resolve different resolution images in different instances. Find more information in our documentation.

Improving Mobile Touch Support

Input handling was streamlined to handle both touch and mouse input efficiently. On touch devices, all handles maintain the recommended minimum size for touch input and are intelligently hidden to minimize overlap. Thanks to this mechanism, resizing and rotating are always a breeze.

Rotation of Groups and Hierarchies

The engine and user interface add options for rotating groups and design blocks with children. This allows changing the rotation of multiple elements at once quickly.

Custom Asset Source UI

Instead of fixed entry points such as “Images,” “Stickers,” or “Shapes,” we want to enable custom and flexible entry points. Until today, we tied them to a block type, which is currently only relevant when users add an asset to the scene.

Moving from Roles to Abilities: Disconnecting UI Variants from Roles

Before the change, the Advanced UI was tied to the creator role and the Default UI was tied to the Adopter Role. This is now no longer the case. The scopes and editing options are still controlled by the abilities of the current role.

Shadows in Advanced UI

With this update, we roll out the long-awaited ability to apply shadows to elements in our Advanced UI. Realistic drop shadow effects add more depth to your designs with a few clicks – no other editor is needed.

Blur API

We introduce new APIs to modify the blur for images and other blocks. This change allows programmatic access to add blur effects in automation cases and when building your own UI.

Effects API

We introduce new APIs to modify the effect stack for shapes and other blocks. This allows programmatic access to effects like adjustments, LUT filters, and many more.

Image Fills API

Currently, you can only access and use images with the image block. However, pages or vector shapes could use more than just a color or gradient fill. Therefore, we broaden the concept of fills by allowing every block to have an image fill, e.g., pages. This change will enable building a more natural photo- or video-editor experience. Documentation.

Image Straighten API

We are extending the Image Crop API to allow straightening, flipping, and rotating images during cropping. This change allows more fine-grained control.

Improved Group Selection and API

There are multiple options to select blocks inside a group:

selecting a group first and with each click select one block deeper in the hierarchy
selecting an element under the cursor directly, neglecting the hierarchy

Previously, we defaulted to different behaviors depending on the user’s role. Now, this is configurable and also customizable during runtime via APIs. Therefore, it’s up to the implementor how the selection should work.

Scopes API

We introduce new APIs to modify the scopes for blocks. This update allows programmatic access to change the scopes for users and allows control over the possible edits a user can make to a scene.

Shadows API

We introduce new APIs to modify shadows for images and other blocks. This change allows programmatic access to add shadow effects to be used in automation cases and when you build your UI.

Zoom API

We introduce new APIs to modify the current zoom level. This allows programmatic access to change the current zoom easily for automation use-cases or for building your own user interface.

Product Roadmap

As mentioned before, we are happy to publish our Product Roadmap to let you in on our new features and enhancements for CE.SDK. This way, we can elaborate our decisions and plans for the product and how you can benefit while allowing you to give us feedback and indicate the importance of particular features.
Now that you know what’s coming down the pike, you can prepare for the release of significant improvements and new features such as Video Support for Web or Native iOS Support well in advance.

How To Build a Canva Clone with CE.SDK

Antonello — Wed, 27 Jul 2022 06:43:59 GMT

Canva has popularized image editing, and user expectations of creative capabilities have increased accordingly. If your web or mobile application includes any design functionality - for book covers, t-shirt designs, or social media content - it has to be on par with the design experience offered by prosumer creation tools such as Canva.

Allowing users to edit images through an easy user interface, define high-quality templates, and give them the ability to share them with the community is a great way to boost engagement, add virality, and potentially new revenue streams.

You might think this requires a lot of time and effort, but that is not the case. CreativeEditor SDK makes it dead simple to build a Canva-like design editor in minutes. I’ll show you how!

Follow this step-by-step tutorial and learn how to implement a Canva clone in React with CE.SDK. At the end of this tutorial, you will achieve the following result:

Try the final result live on CodeSandbox.

What is Canva?

Canva is a free graphic design tool for image editing featuring a drag-and-drop interface for composing different elements on a canvas. You can easily create flyers, invitations, business cards, and more with professionally designed templates. You can think of it as a basic version of Photoshop that anyone can use.

You can create Canva templates in a web browser or on the official app for iOS or Android, share them and collaborate easily. Templates make Canva a powerful tool - that is also true for CE.SDK, which makes it a perfect harbor to set sail for new horizons and create an excellent editor. Before we learn how to replicate Canva, let us evaluate the SDK in question.

What is CE.SDK?

CreativeEditor SDK (CE.SDK) is a fully customizable, user-friendly design editor tool. You can integrate it easily into your application with just a few lines of code and benefit from the template-based editor in no time.

The role-specific editing UI focuses heavily on content adaptation. CE.SDK offers two modes: in Creator Mode, you can create a design from scratch or customize existing templates. Once ready, creators can share their templates and decide which elements others can change and to what degree. Then, users can import the template and customize it in Adopter Mode (Default UI).

Try CE.SDK live demos or start a free trial.

The Creator Mode is the most powerful and least restrictive role in CE.SDK. It empowers you to unleash your creativity and create custom templates for every imaginable use case. Add, move, modify, and delete elements as you wish and define constraints. Further elevate your designs with photo editings, such as filters, effects, and background removal.

The well-rounded text editor also allows you to define variables and programmatically give them a value using the CE.SDK Variable API. For example, you may introduce a {{Name}} Variable and have CESDK import names from a database. This automation is perfect for batch-processing designs, such as greeting cards. Based on this principle, we have built an example NFT Art Collection Generator in React with CE.SDK.

In the Creator Mode, you also can define placeholders. Turning an element into a placeholder will let you determine if it can be deleted, styled, or duplicated by the adopter.

This is what the CE.SDK Creator Mode looks like:

As you can see, you can use CE.SDK to easily create a wedding invitation card template. This is just an example, and you can find other cool sample templates for greeting cards, flyers, postcards, and business cards on showcase page.

What Adopter Mode Offers You

You can test the adopter view of the wedding invitation card template shown above here.

This is what the Adopter Mode looks like:

As shown above, CE.SDK’s Adopter Mode enables you to add and modify colors, text, and images based on the constraints enabled by the creator of the template. This mode provides the Adopter users with a simple interface, narrowed down to the properties that they are allowed to change. This prevents adopters from accidentally changing or deleting parts of a design that should not be modified.

Implement a Canva Clone with CE.SDK

Let’s now learn how to use CE.SDK to build a Canva clone in React.

Prerequisites

This is the list of all the prerequisites for the Canva clone application you are going to build:

Add @cesdk/cesdk-js to your project’s dependencies with the following command:

npm install @cesdk/cesdk-js

Initialize a React Project

You can try out the Canva clone by cloning the GitHub repository supporting the article and running the demo web application with the following commands:

git clone https://github.com/Tonel/canva-clone-cesdk-imgly
cd canva-clone-cesdk-imgly
npm i
npm start

Otherwise, follow this step-by-step tutorial and learn how to build the Canva clone with CE.SDK by yourself.

First, you need to initialize a canva-clone-cesdk-imgly React project. You can do it with Create React App by launching the command below:

npx create-react-app canva-clone-cesdk-imgly

Now, the canva-clone-cesdk-imgly folder will have the following file structure:

canva-clone-cesdk-imgly
├── README.md
├── node_modules
├── package.json
├── .gitignore
├── public
│   ├── favicon.ico
│   ├── index.html
│   ├── logo192.png
│   ├── logo512.png
│   ├── manifest.json
│   └── robots.txt
└── src
    ├── App.css
    ├── App.js
    ├── App.test.js
    ├── index.css
    ├── index.js
    ├── logo.svg
    ├── reportWebVitals.js
    └── setupTests.js

Move into the canva-clone-cesdk-imgly folder and start a local development server with:

cd canva-clone-cesdk-imgly
npm start

Reach http://localhost:3000/ in your browser and make sure you see the default Create React App screen below:

You are now ready to code!

Build the Canva Clone Component with CE.SDK

If you want your image editing component to provide a Canva-like experience, it must include key features, such as template and custom resource management. Let’s see how to implement them all with CE.SDK.

Configure CE.SDK to use templates

Managing templates in CE.SDK is easy. All you have to do is configure the set of predefined templates loaded by CE.SDK on initialization, as explained in the official documentation on adding templates. You can achieve this as below:

useEffect(() => {
  let cesdk;

  let config = {
    // initializing CE.SDK with a few templates
    presets: {
      templates: {
        postcard_1: {
          label: 'Postcard Design',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_1.png`,
        },
        postcard_2: {
          label: 'Postcard Tropical',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_2.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_2.png`,
        },
        business_card_1: {
          label: 'Business card',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.png`,
        },
        instagram_photo_1: {
          label: 'Instagram photo',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_photo_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_photo_1.png`,
        },
        instagram_story_1: {
          label: 'Instagram story',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_story_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_story_1.png`,
        },
        poster_1: {
          label: 'Poster',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_poster_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_poster_1.png`,
        },
        presentation_4: {
          label: 'Presentation',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_presentation_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_presentation_1.png`,
        },
        collage_1: {
          label: 'Collage',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_collage_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_collage_1.png`,
        },
      },
    },
  };

  if (cesdkContainer.current) {
    CreativeEditorSDK.init(cesdkContainer.current, config).then((instance) => {
      cesdk = instance;
    });
  }

  return () => {
    if (cesdk) {
      cesdk.dispose();
    }
  };
}, [cesdkContainer]);

Thanks to the presets.templates, you can specify all the CE.SDK templates to be made available to users.
Note that the cesdkContainer variable stores the React reference to the div where to mount CE.SDK.

This is what the CE.SDK template section will look like:

As you can see, all eight templates loaded with presets.templates.

Integrate External Asset Sources Into CE.SDK

CE.SDK supports external asset sources, meaning you can give your users a vast library of resources to supercharge their creative workflows. Instead of spending time building a repository of images manually, you can provide your users with image resources retrieved dynamically by connecting to services like Unsplash and Airtable. Learn how to integrate external asset sources in CE.SDK, or see a live demo based on the Unsplash and Airtable integration.

Add Custom Images to CE.SDK

You can load custom resources and make them available to users with CE.SDK. For example, you would like to add this Flaticon below to the available images in CE.SDK:

All you need to do is define a new asset source as explained in the documentation page on Integrating Custom Resources into CE.SDK.

You can achieve this as follows:

useEffect(() => {
  // path to the local image to load into CE.SDK
  const customImagePath = `${window.location.protocol + '//' + window.location.host}/resources/programming.png`;

  let cesdk;

  let config = {
    // loading the business card template as default template
    initialSceneURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.scene`,
    // loading the external asset sources
    assetSources: {
      // loading a custom image into CE.SDK
      custom: {
        findAssets: () => {
          return {
            assets: [
              {
                id: 'custom-image-1',
                type: 'ly.img.image',
                locale: 'en',
                label: 'Programming',
                thumbUri: customImagePath,
                size: {
                  width: 512,
                  height: 512,
                },
                meta: {
                  uri: customImagePath,
                },
                context: {
                  sourceId: 'custom',
                },
                credits: {
                  name: 'Freepik',
                  url: 'https://www.flaticon.com/free-icon/programming_1208884?related_id=1208782&origin=search',
                },
              },
            ],
            currentPage: 1,
            total: 1,
            nextPage: undefined,
          };
        },
      },
    },
    // translating the labels associates with the external asset sources
    i18n: {
      en: {
        'libraries.custom.label': 'Custom',
      },
    },
    // initializing CE.SDK with a few templates
    presets: {
      templates: {
        // ...
        business_card_1: {
          label: 'Business card',
          scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.scene`,
          thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.png`,
        },
        // ...
      },
    },
  };

  if (cesdkContainer.current) {
    CreativeEditorSDK.init(cesdkContainer.current, config).then((instance) => {
      cesdk = instance;
    });
  }

  return () => {
    if (cesdk) {
      cesdk.dispose();
    }
  };
}, [cesdkContainer]);

The customImagePath variable stores the path to the local image file programming.png located in the public/resources folder of the React project. This variable is used in the findAssets() function. This function is important because it defines the complete asset source. In other words, you only need one function to implement the custom asset retrieval logic. Note that providing images with labels is useful to make them searchable through the CE.SDK UI.

Now, users will be able to use the image in their template.

Tie it Together

This is what the final CanvaClone component looks like:

import './CanvaClone.css';
import CreativeEditorSDK from '@cesdk/cesdk-js';
import React, { useEffect, useRef } from 'react';
import { findAirtableAssets } from './airtableAssetLibrary';
import { findUnsplashAssets } from './unsplashAssetLibrary';

function CanvaClone({
  // initializing Airtable as default external asset library
  assetLibrary = 'airtable',
}) {
  const cesdkContainer = useRef(null);

  useEffect(() => {
    const externalAssetSources = {
      ...(assetLibrary === 'airtable' && {
        airtable: {
          findAssets: findAirtableAssets,
          credits: {
            name: 'Airtable',
            url: 'https://airtable.com/shr4x8s9jqaxiJxm5/tblSLR9GBwiVwFS8z?backgroundColor=orange',
          },
        },
      }),
      ...(assetLibrary === 'unsplash' && {
        unsplash: {
          findAssets: findUnsplashAssets,
          credits: {
            name: 'Unsplash',
            url: 'https://unsplash.com/',
          },
          license: {
            name: 'Unsplash license (free)',
            url: 'https://unsplash.com/license',
          },
        },
      }),
    };

    // path to the local image to load into CE.SDK
    const customImagePath = `${window.location.protocol + '//' + window.location.host}/resources/programming.png`;

    let cesdk;

    let config = {
      // loading the business card template as default template
      initialSceneURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.scene`,
      // loading the external asset sources
      assetSources: {
        // loading the AirTable or Unsplash asset library
        ...externalAssetSources,
        // loading a custom image into CE.SDK
        custom: {
          findAssets: () => {
            return {
              assets: [
                {
                  id: 'custom-image-1',
                  type: 'ly.img.image',
                  locale: 'en',
                  label: 'Programming',
                  thumbUri: customImagePath,
                  size: {
                    width: 512,
                    height: 512,
                  },
                  meta: {
                    uri: customImagePath,
                  },
                  context: {
                    sourceId: 'custom',
                  },
                  credits: {
                    name: 'Freepik',
                    url: 'https://www.flaticon.com/free-icon/programming_1208884?related_id=1208782&origin=search',
                  },
                },
              ],
              currentPage: 1,
              total: 1,
              nextPage: undefined,
            };
          },
        },
      },
      // translating the labels associates with the external asset sources
      i18n: {
        en: {
          'libraries.airtable.label': 'Airtable',
          'libraries.unsplash.label': 'Unsplash',
          'libraries.custom.label': 'Custom',
        },
      },
      // initializing CE.SDK with a few templates
      presets: {
        templates: {
          postcard_1: {
            label: 'Postcard Design',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_1.png`,
          },
          postcard_2: {
            label: 'Postcard Tropical',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_2.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_postcard_2.png`,
          },
          business_card_1: {
            label: 'Business card',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_business_card_1.png`,
          },
          instagram_photo_1: {
            label: 'Instagram photo',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_photo_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_photo_1.png`,
          },
          instagram_story_1: {
            label: 'Instagram story',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_story_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_instagram_story_1.png`,
          },
          poster_1: {
            label: 'Poster',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_poster_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_poster_1.png`,
          },
          presentation_4: {
            label: 'Presentation',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_presentation_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_presentation_1.png`,
          },
          collage_1: {
            label: 'Collage',
            scene: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_collage_1.scene`,
            thumbnailURL: `https://cdn.img.ly/packages/imgly/cesdk-js/latest/assets/templates/cesdk_collage_1.png`,
          },
        },
      },
    };
    if (cesdkContainer.current) {
      CreativeEditorSDK.init(cesdkContainer.current, config).then(
        (instance) => {
          cesdk = instance;
        }
      );
    }

    return () => {
      if (cesdk) {
        cesdk.dispose();
      }
    };
  }, [cesdkContainer, assetLibrary]);

  return (
    <div className="caseContainer">
      <div className="wrapper">
        <div ref={cesdkContainer} className="cesdk"></div>
      </div>
    </div>
  );
}

export default CanvaClone;

The findAirtableAssets() and findUnsplashAssets() functions come from the GitHub repo associated with the Unsplash and Airtable integration showcase page.

These functions allow CE.SDK to dynamically retrieve resources from online services. A search for images in the CE.SDK UI will perform a query on Airtable or Unsplash and provides them to your users. With just a few lines of code, you are making massive resources available to your users.

Conclusion

Canva has rapidly become one of the most popular graphic design tools. It is so prevalent that users expect similar features in your web and mobile applications. Building a Canva clone from scratch would take months, but with CreativeEditor SDK, it only takes minutes.

In this article, we used CE.SDK to initialize an advanced design editor in React. We adopted its API to configure and customize a design editor to build a Canva clone. This would not be possible without the intuitive UI offered by CE.SDK and its advanced features, such as placeholders and the possibility to define validation rules to guide your users’ creation process.

Here we only scratched the surface of what is possible with CE.SDK – you can use its powerful image processing API to implement many more features and make your Canva clone more and more complex. This is only the beginning!

See how IMG.LY compares to Canva Connect API and check out all the use cases of CreativeEditor SDK or contact sales to learn more!

Content Moderation for User-Generated Designs with CE.SDK and Sightengine

Antonello — Mon, 13 Jun 2022 14:07:54 GMT

Creating content and sharing it with others has become one of the core features offered by online services. As users publish their content to your website, you want to consider implementing a system to prevent inappropriate posts. Ideally, content will be programmatically moderated before it is uploaded on your platform. That is what you will learn in this article!

Follow this step-by-step tutorial and learn how to integrate content moderation with Sightengine in your design process built with CreativeEditor SDK in React. At the end of this tutorial, you will achieve the following result:

Why You Should Moderate User-Generated Content

User-generated content represents a large portion of online content, whether text or multimedia. Empowering users with tools and features to create high-quality content is excellent. However, it is a common concern that potentially inappropriate content on your website will be associated with your brand. Naturally, you will want to avoid this.

A recent example from the web-to-print industry serves as a cautionary tale: the German e-commerce company Spreadshirt neglected to moderate content on its t-shirt printing platform, leading to a wave of outrage on social media. Even though the company reacted swiftly and stated an apology, Spreadshirt could have avoided most of the damage if it had implemented a system that automatically flags potentially harmful content. Of course, a religious or cultural symbol is not inherently inappropriate, but the misuse is.

It is highly advisable to take precautions to prevent your company from facing similar adversity. A common problem, however, is the costly and time-consuming manual moderation, epecially when controlling multimedia files like images.

Luckily, you can use CE.SDK and Sightengine to programmatically integrate moderation into your content design process without manual reviews.

What is CE.SDK?

CreativeEditor SDK (CE.SDK) is a powerful, easy-to-use, and fully customizable design editor. With just a few lines of code, CE.SDK provides your application with template-based workflows and automation.

The role-specific editing UI focuses on keeping content adaptation dead simple: in Creator Mode, you have the liberty to create a design from scratch or customize existing templates. When creators are ready to share a design for others to use and customize in Adopter Mode (Default UI), creators may decide which elements can be changed. See CE.SDK in action or try it for free.

What is Sightengine?

Sightengine is a convenient tool for automatic content moderation. It detects and filters unwanted content in photos, videos, and live streams through fast, accurate, easy-to-integrate, privacy-compliant APIs.

Sightengine offers you APIs to get moderation results instantly and effectively. The API-centric approach allows you to scale to up to tens of millions of images per month. You can have a look at the Sightengine demo or try it for free.

Integrate Content Moderation into CE.SDK

Let us look at how to implement an image editor with content moderation for your users. For this, you will learn how to use CE.SDK and React to implement an image design component and use the CE.SDK APIs to programmatically moderate the content with Sightengine on a Node.js server. You can try out a live showcase of content moderation with CE.SDK.

With the following steps, you can easily achieve this result with CE.SDK, React, Sightengine, and Node.js.

1. Get Started with CE.SDK

Integrate CreativeEditor SDK with React by using the official documentation. As you will see in the final snippet, this takes only a few lines of code.

Now that you have a working CE.SDK instance, your users may unleash their creativity by designing templates and sharing them with others. The powerful APIs ensure control at every step of the editing process and allow programmatically performing operations, such as content moderation.

2. Build a Content Moderation API with Sightengine

Let’s move on to content moderation with Sightengine. Keep in mind that any backend technology will work, but here you will implement a content moderation API based on Sightengine in Node.js.

2.1 Getting Started with Sightengine

First, you need a Sightengine account. If you do not have one yet, you can create a free account. Fill out the form, click Sign Up, and receive a verification email in your inbox. Click on the verification link and activate your account.

Then, navigate to API Keys on your Sightengine Dashboard page and retrieve your API user and secret credentials. You will need them to interact with the Sightengine moderation APIs.

2.2 Prerequisites

Here is a list of all prerequisites for the backend application you are going to build:

You can add express and node-fetch to your project’s dependencies with the following command:

npm install --save express node-fetch

2.3 Setting up a Node.js Express Server

Follow this step-by-step guide to set up a Node.js Express development server. You will need this to perform an API call to Sightengine to moderate your content server-side. Any Sightengine API call involves your Sightengine user and secret pair of credentials, and you do not want to expose them.

You may also want to store the result of the moderation API in your database, as well as other data. For example, you might be interested in keeping track of how many times a particular user has violated the moderation rules to warn or ban them.

2.4 Implementing a Content Moderation API in your Node.js Server

You can now implement a content moderation API in your Node.js Express server as follows:

    const fetch = require("node-fetch")

    router.get('/content-moderation', function (req, res) {
      if (!req.query.url) {
            res.status(404).send("Please provide an image url")
            return
        }

        // defining the Sightengine API parameters
        const params = {
            url: req.query.url,
            models: "nudity,wad,offensive",
            api_user: "YOUR_SIGHTENGINE_USER",
            api_secret: "YOUR_SIGHTENGINE_SECRET",
        }

        // generating the Sightengine API URL
        const queryString = new URLSearchParams(params).toString();
        const requestOptions = {
            method: "GET",
        }
        const sightengineUrl = "https://api.sightengine.com/1.0/check.json?" + queryString

        // calling the Sightengine moderation API
        const externalRes = await fetch(SightengineUrl, requestOptions)
        // retrieving the moderation result from the Sightengine API
        const moderationResultText = await externalRes.text()

        res.append("Access-Control-Allow-Origin", ["*"])
        res.append("Access-Control-Allow-Methods", "GET")
        res.append("Access-Control-Allow-Headers", "Content-Type")

        // returning the Sightengine moderation result text
        res.status(200).send(text)
    })

Take care of replacing YOUR_SIGHTENGINE_USER and YOUR_SIGHTENGINE_SECRET with your Sightengine pair of credentials retrieved before.

Now, launch your Node.js server and reach the following URL below to use Sightengine to moderate the images specified in the URL query parameter:

    http://localhost:3000/content-moderation?url=<your_image_url>

3. Moderate Images Made with CE.SDK

We will now use CE.SDK and Sightengine to programmatically moderate user-generated images. Before we can dive in, check the following requirements.

3.1 Prerequisites

This is the list of all the prerequisites you need to build for the frontend demo application:

You can add the @cesdk/cesdk-js library to your project’s dependencies with the following command:

npm install --save @cesdk/cesdk-js

Also, the demo application will use other libraries, although they are not strictly required to achieve the goals of this tutorial and can be omitted here.

3.2 Getting Started

You can try out the demo application by cloning the GitHub repository that supports this article with the following commands:

git clone https://github.com/Tonel/content-moderation-with-cesdk-and-sightengine
cd content-moderation-with-cesdk-and-sightengine
npm i
npm start

Otherwise, keep following this tutorial and build the demo application for content moderation by yourself.

First, initialize a React project called content-moderation-with-cesdk-and-sightengine by employing the Create React App command below:

    npx content-moderation-with-cesdk-and-sightengine

Now, the content-moderation-with-cesdk-and-sightengine folder should have the following file structure:

content-moderation-with-cesdk-and-sightengine
├── README.md
├── node_modules
├── package.json
├── .gitignore
├── public
│   ├── favicon.ico
│   ├── index.html
│   ├── logo192.png
│   ├── logo512.png
│   ├── manifest.json
│   └── robots.txt
└── src
   ├── App.css
   ├── App.js
   ├── App.test.js
   ├── index.css
   ├── index.js
   ├── logo.svg
   ├── reportWebVitals.js
   └── setupTests.js

Enter the content-moderation-with-cesdk-and-sightengine folder and start a local server with these two commands:

cd content-moderation-with-cesdk-and-sightengine
npm start

Visit http://localhost:3000/ in your browser, and you should now be seeing the default Create React App screen below:

3.3 Integrate the Moderation Check into your Design Process

Notice that the demo application you are about to implement is just a simplified version of the app you can find in this GitHub repository. This demo app integrates the Sightengine moderation functionality with the CE.SDK design capabilities.

Now, let’s move on to the main component where the integration takes place:

import CreativeEditorSDK from '@cesdk/cesdk-js';
import ValidationBox from '../ui/ValidationBox/ValidationBox';
import React, { useCallback, useEffect, useRef, useState } from 'react';
import { ReactComponent as RefreshIcon } from './refresh.svg';
import { moderateImages } from './moderationUtils';
import styles from './ImageModerator.module.css';
import classNames from 'classnames';

function selectBlock(cesdk, blockId) {
  // deselecting all blocks
  cesdk.engine.block
    .findAllSelected()
    .forEach((block) => cesdk.engine.block.setSelected(block, false));

  // selecting the block having the blockId passed as parameter
  cesdk.engine.block.setSelected(blockId, true);
}

const ImageModerator = () => {
  const cesdkContainerDiv = useRef(null);
  const cesdkRef = useRef(null);
  const [validationResults, setValidationResults] = useState([]);

  useEffect(() => {
    // initializing CE.SDK if it has not been initialized yet
    if (cesdkContainerDiv.current && !cesdkRef.current) {
      const config = {
        role: 'Adopter',
        theme: 'light',
        initialSceneURL: `${window.location.protocol + '//' + window.location.host}/cases/content-moderation/example.scene`,
        ui: {
          elements: {
            panels: {
              settings: true,
            },
            navigation: {
              action: {
                save: true,
              },
            },
          },
        },
        callbacks: {
          // calling this function when the
          // user presses the "Save" button
          onSave: () => runImageModerationCheck(),
        },
      };
      CreativeEditorSDK.init(cesdkContainerDiv.current, config).then(
        (instance) => {
          cesdkRef.current = instance;
          // running the moderation check on initialization
          runImageModerationCheck();
        }
      );
    }
    return () => {
      if (cesdkRef.current) {
        cesdkRef.current.dispose();
      }
    };
  }, [cesdkContainerDiv]);

  const runImageModerationCheck = useCallback(async () => {
    if (!cesdkRef.current) {
      return;
    }
    // retrieving the moderation results
    const validationResults = await moderateImages(cesdkRef.current);
    setValidationResults(validationResults);
  }, []);

  return (
    <div className={classNames(styles.wrapper, 'space-y-2')}>
      <div className={styles.header}>
        <div className={styles.headerDiv}>
          <div>
            <h3 className="h4" style={{ color: 'white' }}>
              Content Moderation
            </h3>
            <p className={styles.headerDivMessage}>
              Check images for compliance with your content guidelines before
              further processing and provide user feedback.
            </p>
          </div>
          <button
            onClick={() => runImageModerationCheck()}
            className={'button button--white space-x-2'}
          >
            <span>Validate Image Content</span>
            <RefreshIcon />
          </button>
        </div>
        <ValidationBox
          results={validationResults}
          onSelect={(blockId) => {
            selectBlock(cesdkRef.current, blockId);
          }}
        />
      </div>
      <div className={styles.cesdkWrapper}>
        <div ref={cesdkContainerDiv} className={styles.cesdk}></div>
      </div>
    </div>
  );
};

export default ImageModerator;

In the first useEffect() hook, CE.SDK is initialized, and a callback on the save action is registered. This will take care of calling the runImageModerationCheck() function when the user presses the CE.SDK Save button. For more information on how callbacks work in CE.SDK, see the official documentation on observing events.

runImageModerationCheck() retrieves the moderation results obtained from Sightengine and then stores it in the validationResults array variable. This is then passed to the ValidationBox component, which will show the results of the validation process to the end-user. None of this would be possible without the moderateImages() function from the moderationUtils.js file below:

export const moderateImages = async (cesdk) => {
  // retrieving all image blocks with a name
  const imageBlocksData = cesdk.engine.block
    .findByType('image')
    .map((blockId) => ({
      blockId,
      url: cesdk.engine.block.getString(blockId, 'image/imageFileURI'),
      blockType: cesdk.engine.block.getType(blockId),
      blockName: cesdk.engine.block.getName(blockId),
    }))
    .filter((block) => {
      return block.blockName !== '';
    });

  console.log('Moderation check launched...');

  // turning the moderation results on each image block into
  // a flat array
  const imagesWithValidity = await Promise.all(
    imageBlocksData.flatMap(async (imageBlockData) => {
      // retrieving the moderation results
      const imageModerationResults = await callSightengineModerationAPI(
        imageBlockData.url
      );

      console.log(imageBlockData.blockName, imageModerationResults);

      return imageModerationResults.flatMap((checkResult) => ({
        ...checkResult,
        ...imageBlockData,
      }));
    })
  );

  console.log('Moderation check completed');

  return imagesWithValidity.flat();
};

// a cache variable to avoid
// calling the Sightengine API when not necessary
const complianceCache = {};

const callSightengineModerationAPI = async (url) => {
  // if the url is already present in the cache, then
  // return the result previously retrieved
  if (complianceCache[url]) {
    return complianceCache[url];
  }

  // calling the Sightengine moderation API
  const response = await fetch(
    // your endpoint URL
    'https://europe-west3-img-ly.cloudfunctions.net/sightengineApiProxy?url=' +
      encodeURIComponent(url),
    {
      method: 'get',
      headers: {
        accept: 'application/json',
        'Accept-Language': 'en-US,en;q=0.8',
        'Content-Type': 'multipart/form-data;',
      },
    }
  );

  const results = await response.json();

  if (results.error) {
    console.error(results.error);
    return [];
  } else {
    const checkResults = [
      {
        name: 'Weapons',
        description: 'Handguns, rifles, machine guns, threatening knives...',
        state: percentageToState(results.weapon),
      },
      {
        name: 'Alcohol',
        description: 'Wine, beer, cocktails, champagne...',
        state: percentageToState(results.alcohol),
      },
      {
        name: 'Drugs',
        description: 'Cannabis, syringes, glass pipes, bongs, pills...',
        state: percentageToState(results.drugs),
      },
      {
        name: 'Nudity',
        description: 'Images that contain either raw nudity or partial nudity.',
        state: percentageToState(1 - results.nudity.safe),
      },
    ];

    // storing the Sightengine results in the cache
    complianceCache[url] = checkResults;
    return checkResults;
  }
};

const percentageToState = (percentage) => {
  // defining the custom moderation logic based on the
  // percentage returned by Sightengine
  if (percentage > 0.8) {
    return 'failed';
  } else if (percentage > 0.4) {
    return 'warning';
  } else {
    return 'success';
  }
};

Here, we use the CE.SDK reference variable to retrieve all images in the scene through the CE.SDK APIs. Then, these are converted into URLs and passed callSightengineModerationAPI() function, which takes care of calling the Sightengine moderation API, retrieving the results, using them to produce a new output, and returning it to the caller.

We used an IMG.LY endpoint here to evade putting together a backend application. In a real-world scenario, replace https://europe-west3-img-ly.cloudfunctions.net/sightengineApiProxy?url= with the URL of the /content-moderation endpoint of your backend application. Also, note that a cache system was set up to avoid calling the Sightengine API too many times and using up call credits.

This is what the demo application will look like after being launched and the moderation check has been executed:

As you can see, the Sightengine API correctly detects drugs and weapons. Next, remove the rightmost image containing a gun and click Save to re-run the moderation check.

Sightengine no longer detects weapons in the image, proving that the moderation process is working as expected.

Et voilà! You have learned to implement content moderation for user-generated images with CE.SDK and Sightengine.

Conclusion

A challenge of user-generated content is avoiding the risk of hosting and displaying offensive or outright illegal content. Inappropriate content affects your user community and brand reputation and may even cause legal action against you.

In this article, we used CE.SDK to initialize a fully-featured design editor in React. We employed its API to programmatically call a Node.js server using Sightengine to perform content moderation on images. This allows moderation of user-generated content automatically and prevents the upload of inappropriate content to your website.

We are only scratching the surface of what is possible with CE.SDK – you can implement any UI on top of its powerful image processing API and extend the core functionality with features such as placeholders and define validation rules to guide your users’ creation process.

Thanks for reading! We hope that you found this article helpful. Feel free to reach out on Twitter with any questions, comments, or suggestions.

CE.SDK v1.6.0 Release

Malte — Fri, 20 May 2022 09:21:48 GMT

Today, we are highlighting the best new features and changes in CE.SDK since our v1.5.0 Release six weeks ago. CE.SDK v1.6.0 is adding:

Linear Gradient Fills
Strokes, Outline, Insets, and Stroke Patterns
Blend Modes
Scale Groups, Multi-Selection, and Hierarchies
UI Template Library
Floating and Static Asset Library Panels
Placeholders API
Editing State APIs
API for Image Fit Modes

Linear Gradient Fills

With this update, we are introducing linear gradient fills. This feature creates a gradual blend between two colors, including transparency. Use the Color Picker to switch between solid color and linear gradient fill modes and determine the order of colors.

Strokes, Outline, Insets, and Stroke Patterns

This feature introduces strokes and stroke patterns. Strokes apply to any block, including images, text, and shapes. Use our APIs or UIs to set the stroke width, color, style, and more options affecting the stroke appearance.

Blend Modes

Blend modes are a popular feature in creative and image editing tools. They blend two blocks or layers to get different types of effects. This feature is available in both the API and the UI.

Scale Groups, Multi-Selection, and Hierarchies

Our engine and user interface now provide options to scale groups, multi-select and create design blocks with children. You can easily change the scaling of multiple elements at once. This also simplifies resizing and rescaling your template designs for other page and art board sizes.

UI Template Library

We are introducing the template library in all UIs. Allow users to switch between multiple templates while editing.

Floating and Static Asset Library Panels

For some screen sizes and use-cases, it might be tedious that the asset library floats over your canvas and covers your design. Therefore, you can now configure the asset library panel to be either floating or static. A static asset library won’t overlap with your design and adjusts to your screen size perfectly.

Placeholders API

CE.SDK 1.6 introduces an additional API that allows you to interact with all placeholders on the canvas. The API queries the number of placeholders in a template or document and lists all available placeholders in the current scene.

Editing State APIs

We extend the API to allow querying and modifying the current editing mode. That includes changing the on-canvas editing mode to Crop, Text, or others. You can also query the editor state for helpful information like the desired cursor type and rotation and the text cursor position. To observe these values more comfortably, an additional endpoint allows registering for callbacks whenever the editing state changes.

API for Image Fit Modes

The API now allows switching between fit modes for images:

Crop applies crop properties, and ignores the aspect ratio.
Cover resizes the image aspect-aware to fill the frame.
Contain allows resizing the image aspect-aware to fit into its frame.

How To Generate an NFT Art Collection With React Using CE.SDK

Antonello — Tue, 17 May 2022 13:48:25 GMT

In recent months NFTs have become increasingly popular and now represent a worldwide trend. We all have heard about NFTs at least once, but have you ever wondered how to generate an art collection easily and programmatically? That is what you will learn in this article!

Follow this step-by-step tutorial and learn how to build a trading card image generator representing your NFT art collection in React with CreativeEditor SDK. At the end of the article, you will achieve the following result:

What is an NFT Art Collection?

NFT stands for Non-Fungible Token and represents a non-interchangeable unit of data stored on a blockchain that can be traded and sold. We usually associate NFTs with physical or digital assets, such as photos, videos, or audio. You can think of an NFT as a digital work of art.

Consequently, an NFT art collection is nothing more than a set of NFTs associated with a physical or digital art piece.

What is CreativeEditor SDK?

CreativeEditor SDK (CE.SDK) is a fully customizable, easy-to-use, advanced design editor SDK that lets you add template-based workflows and automation to your app with just a few lines of code.

CE.SDK is based on a role-specific editing UI and focuses on keeping content adaptation dead simple – all of this, while empowering your creativity with a powerful authoring environment.

See CE.SDK in action or try it for free.

Build an NFT Art Collection Generator in React With CE.SDK

Now, let’s learn how to implement an NFT art collection generator using CE.SDK and React. In detail, you will learn how to use CE.SDK to define a dynamic template you can then adopt in React to generate your fancy NFT art collection of trading cards.

You can have a look at the final CE.SDK template live here.

Keep in mind that a CE.SDK template is not a static resource. That is because you can change it programmatically through the CE.SDK APIs, as you are about to learn.

1. Get started with CE.SDK

You can learn how to get started with CE.SDK in React by following the getting started guide of the official documentation. Notice that you do not need to set up an instance of CE.SDK to follow this article. You can follow this step-by-step tutorial.

2. Initialize a CE.SDK design

First of all, you need to create a CE.SDK template. As you are about to see, this is not that difficult. All you have to do is click on the “New Design” button.

Give your template a name, such as “NFT Trading Card Collection”.

Click on “Create”. This is what your blank template should look like:

Here you can unleash your creativity and create the NFT trading card template to your desire.

3. Make your CE.SDK design dynamic with variables and by setting placeholders

CE.SDK equips you with the ability to define variables. You can then give these variables a value in your frontend code. The CE.SDK variables can then be defined during text editing.

To add a variable to your design, click on “Text”, select a text element, place it on your page, double-click on it, and select “Add a variable”.

A CE.SDK variable has the following format:

{{variable_name}}

A variable value can be defined programmatically via the CE.SDK Variable API. Use them to set the dynamic text fields you want users to customize in the application layer. For example, you can fill them with data retrieved from an API called by the frontend.

Also, CE.SDK gives you the possibility to define placeholders. Placeholders can be created only by a user with Creator role and allow them to specify a set of constraints associated with the element declared as a placeholder. These constraints only apply to users having non-creator roles and define what operations such users can perform on that element.

You can define a placeholder by selecting an element, clicking Placeholder and then choosing the constraints you want the non-creator users to have for this element.

In this case, non-creator users are only allowed to style the name variable defined above.

Let’s assume you defined the trading card template of your dreams. You spent a lot of time on it, and you now want other users to use it to generate their own NFT art collection. All you have to do, is save, click on the “v” symbol near the Save button, and select “Share as Template”.

Now follow the instructions, and non-creator users will be able to access your template with the link the application provided you.

Users will be able to interact with your CE.SDK template based on the variables and placeholders you defined while creating your design.

Bear in mind that sharing your design as a template is a nice feature but not mandatory to achieve the goal. In case you want to use your design on your CE.SDK frontend instances, you only have to export it as a scene, and then load it through the CE.SDK Scene API.

To export a design as a scene, click on the “v” symbol, and in the dropdown, select “Export Design File”. You will automatically download the scene file to the directory according to your browser settings. The export feature allows you to download your scene. You can also import your own or another user’s scene.

You now have everything you need to build an NFT trading card collection generator in React! Let’s delve deeper into how to achieve this.

5. Use your CE.SDK scene in React to programmatically generate an NFT trading card art collection

5.1 Prerequisites

This is the list of all the prerequisites for the demo application you are going to build:

You can add the @cesdk/cesdk-js library to your project’s dependencies with the following command:

npm install --save @cesdk/cesdk-js

5.2 Getting started

You can try out the NFT art collection generator by cloning the GitHub repository supporting the article and launching the web application with the following commands:

git clone https://github.com/Tonel/how-to-generate-an-nft-art-collection-imgly
cd how-to-generate-an-nft-art-collection-imgly
npm i
npm start

Otherwise, keep following this tutorial and build the NFT trading card art collection generator by yourself.

First, initialize a React project called nft-art-collection-generator with Create React App with the command below:

npx create-react-app nft-art-collection-generator

Now, your nft-art-collection-generator should have the following file structure:

nft-art-collection-generator
├── README.md
├── node_modules
├── package.json
├── .gitignore
├── public
│   ├── favicon.ico
│   ├── index.html
│   ├── logo192.png
│   ├── logo512.png
│   ├── manifest.json
│   └── robots.txt
└── src
    ├── App.css
    ├── App.js
    ├── App.test.js
    ├── index.css
    ├── index.js
    ├── logo.svg
    ├── reportWebVitals.js
    └── setupTests.js

Now move into the nft-art-collection-generator folder and start a local server by launching these two commands:

cd nft-art-collection-generator
npm start

By visiting http://localhost:3000/ in your browser, you should now be able to see the default Create React App screen:

5.3 Building the NFT Art Collection Generator Component

You can use CE.SDK to implement an image generator based on your template as follows:

import CreativeEngine from '@cesdk/cesdk-js/cesdk-engine.umd.js';
import LoadingSpinner from '../LoadingSpinner/LoadingSpinner';
import { DOGS } from './data';
import styles from './NFTArtCollectionGenerator.module.css';
import React, { useEffect, useRef, useState } from 'react';

export default function NFTArtCollectionGenerator() {
  const engineRef = useRef(null);
  const [tradingCardImages, setTradingCardImages] = useState(
    new Array(DOGS.length).fill(null)
  );
  const [initialized, setInitialized] = useState(false);

  // initializing CE.SDK on first rendering
  useEffect(() => {
    const config = {
      page: {
        title: {
          show: false,
        },
      },
    };

    CreativeEngine.init(config).then(async (instance) => {
      // importing the CE.SDK scene
      await instance.scene.loadFromURL(
        `${window.location.protocol + '//' + window.location.host}/assets/cesdk-nft-doggo.scene`
      );
      engineRef.current = instance;
      setInitialized(true);
    });

    return function shutdownCreativeEngine() {
      engineRef?.current?.dispose();
    };
  }, []);

  async function generateTradingCards() {
    const cesdk = engineRef?.current;

    if (initialized && cesdk) {
      // setting the image loading state to true
      // to each image
      setTradingCardImages((oldImages) => [
        ...oldImages.map((image) => ({ ...image, isLoading: true })),
      ]);

      const newTradingCardImages = [];

      // iterating over each data entry to generate
      // its trading card with CE.SDK
      for (const dog of DOGS) {
        // giving each variable defined in the CE.SDK scene
        // a value based on the dog data
        cesdk.variable.setString('name', dog.name);
        cesdk.variable.setString('date', dog.date);
        cesdk.variable.setString('chain', dog.chain);
        cesdk.variable.setString('drool', dog.drool);
        cesdk.variable.setString('courage', dog.courage);

        // retrieving the background block
        const backgroundBlock = cesdk.block.findByType('page')[0];
        if (backgroundBlock) {
          // setting the background to a random color
          cesdk.block.setBackgroundColorRGBA(
            backgroundBlock,
            Math.random(), // r (from 0 to 1)
            Math.random(), // g (from 0 to 1)
            Math.random(), // b (from 0 to 1)
            1.0 // alpha (from 0 to 1)
          );
        }

        const blob = await cesdk.block.export(
          cesdk.block.findByType('scene')[0],
          'image/jpeg'
        );

        newTradingCardImages.push({
          isLoading: false,
          src: URL.createObjectURL(blob),
        });
      }

      // setting the new trading card images
      setTradingCardImages(newTradingCardImages);
    }
  }

  // generating the trading card images for the first time
  // when CE.SDK is initialized
  useEffect(() => {
    if (engineRef && initialized) {
      generateTradingCards();
    }
  }, [engineRef, initialized]);

  return (
    <div className="flex flex-col items-center">
      <div>
        <h3 className={`h4 ${styles.headline}`}>
          NFT Trading Card Collection Generator
        </h3>
      </div>
      <div>
        <button
          className={styles.button}
          onClick={() => {
            generateTradingCards();
          }}
        >
          Generate
        </button>
      </div>
      <div className={styles.contentWrapper}>
        <div className={styles.imageWrapper}>
          {tradingCardImages.map((image, i) => {
            return (
              <div
                style={{
                  width: 160,
                  height: 210,
                  position: 'relative',
                  backgroundColor: 'white',
                }}
                key={i}
              >
                {image?.src && (
                  <img
                    alt=""
                    src={image?.src}
                    style={{
                      opacity: image?.isLoading ? 0.5 : 1,
                      transition: 'opacity .5s',
                      transitionTimingFunction: 'ease-in-out',
                      objectFit: 'contain',
                    }}
                  />
                )}
                {image?.isLoading && <LoadingSpinner />}
              </div>
            );
          })}
        </div>
      </div>
    </div>
  );
}

This component uses CE.SDK to generate a trading card set you can transform into your NFT art collection. The first useEffect() hook function takes care of initializing a CE.SDK instance and loading the scene file exported above. To make everything work as expected, create an assets folder under public, and put your scene file in it.
Then, the generateTradingCards() function using CE.SDK is used to generate the trading card image set is defined. This is where the magic happens!

The trading card images are generated based on the DOGS variable imported from data by giving each scene’s variable a value and providing the background with a random color. This way, every time you press “Generate”, the resulting collection of trading cards will have the same data, but always a different color.
If you are wondering, this is what the data file looks like:

export const DOGS = [
  {
    name: 'Pancakes',
    date: '2022',
    chain: 'etherum',
    drool: '5/10',
    courage: '10/10',
  },
  {
    name: 'Clovis III',
    date: '2020',
    chain: 'flyx',
    drool: '9/10',
    courage: '1/10',
  },
  {
    name: 'Pickles',
    date: '2019',
    chain: 'cardano',
    drool: '1/10',
    courage: '10/10',
  },
  {
    name: 'Groucho Barks',
    date: '2018',
    chain: 'solana',
    drool: '10/10',
    courage: '5/10',
  },
  {
    name: 'Puppins',
    date: '2019',
    chain: 'ethereum',
    drool: '8/10',
    courage: '5/10',
  },
  {
    name: 'Mr. Chow',
    date: '2019',
    chain: 'flow',
    drool: '5/10',
    courage: '9/10',
  },
  {
    name: 'Dr. Droolz',
    date: '2020',
    chain: 'eos',
    drool: '10/10',
    courage: '10/10',
  },
  {
    name: 'Snuffles',
    date: '2018',
    chain: 'ethereum',
    drool: '4/10',
    courage: '5/10',
  },
];

As you can see, the DOGS array contains the data generating each card programmatically through CE.SDK’s Variable API. For simplicity, the DOGS array was hard-coded into the data file. In a real-world scenario, you can retrieve this data through an API call or however you prefer.
Finally, the trading card collection images are rendered and shown to the user as in the image below:

As you just learned, a CE.SDK template allows you to generate an infinite combination of different images. In this example, you learned how to use variables and change the background colors, but several other options are available. Learn more here.

Conclusion

The real challenge to entering the NFT world lies in producing a collection of appealing images. Converting them into NFTs takes only a few clicks. In this article, we used CE.SDK to generate a trading card collection of images that all share the same template. Specifically, CE.SDK gives you everything you need to create applications supporting template-based workflows – your users can adapt your templates or programmatically create variations of visually consistent, high-quality images.

We are only scratching the surface of what is possible with CE.SDK – you can implement any UI on top of its powerful Image Processing API and extend the core functionality with features such as content moderation and define validation rules to guide your users’ creation process.

Thanks for reading! We hope you found this article helpful. Feel free to reach out to us on Twitter with any questions, comments, or suggestions. For more, stay in the loop with our Newsletter.

CE.SDK v1.5.0 Release

Daniel — Thu, 14 Apr 2022 12:53:45 GMT

Another six weeks have passed since our v1.4.0 release that brought an updated asset library and headless APIs. Since the initial CE.SDK release, we’ve been listening to your feedback and continuously releasing updates to improve and expand CE.SDK to better serve your needs. Here’s a short overview of what’s happened in the last quarter.

CE.SDK v1.5.0 is adding:

Hierarchies (Groups)
Template Imports & Exports
More Performance, More Power
High-Performance Rendering
Improved Touch Support
PDF Export
Subscription/Events API

Hierarchies (Groups)

With this release, we finally extended our flat hierarchy model to the typical parent-child relationship model between items on the canvas. That allows you to build an even more complex design: Group items to form a single unit and handle those items as one or control their opacity.

Template Imports & Exports

Export your template scenes as .scene-files that pack all assets that were used. You can also share your template by URL - all dependencies will resolve accordingly.

More Performance

We have replaced our underlying rendering engine with a new high-performance implementation, resulting in the performance of distinct scenarios improving by 100%. Likewise, as a result of polishing up various logic, CE.SDK runs faster than ever.

Improved Touch & Mobile Support

After evaluation, CE.SDK v1.5.0 expands upon our mobile advancement efforts by improving our web UI for touch inputs on mobile devices.

PDF Export

CE.SDK now supports PDF export. Vector paths and images are embedded, and rasterization will only be applied when necessary. Scenes of multiple pages can be exported as a multipage document.

Build Your Custom UI

We vastly improved support for building custom UIs on top of CE.SDK engine:

The CreativeEngine offers a powerful set of APIs to inspect, modify and interact with a scene.
A new set of UI-specific APIs, including history and zoom management, was added as a preview.

Events API

An Event API allows subscribing to scene changes and updates custom UI components accordingly.

Other

For an even smoother UI experience, we have improved the visual quality of element state indicators, and they now respect the element order.

How To Apply Image Filters in WebGL

Antonello — Fri, 08 Apr 2022 13:06:16 GMT

In this article, you will learn how to apply custom filters to an image in WebGL. You can achieve this goal without using any external library. That is because the HTML5 canvas element natively supports WebGL and does not require the use of plug-ins.

As we have already covered in previous articles, <canvas> is a powerful tool that equips you with everything required to manipulate images. This is true regardless of the use of WebGL. So, you can filter images also without using WebGL.

Now, let’s see how to filter images in Vanilla JavaScript with WebGL in your browser. Follow this step-by-step tutorial and learn how to build the following demo:

Getting Started with WebGL

If you are not familiar with WebGL, you need to learn a few things before approaching it. To build an application in WebGL, start with the following three concepts.

1. Textures

Keep in mind that to draw an image in WebGL you have to use a texture. To use a texture, WebGL requires you to define the texture coordinates. These coordinates go from 0.0 to 1.0, regardless of the texture size.

2. Vertex shader

The vertex shader is a function you have to write in GLSL that is in charge of computing the vertex positions. Thanks to it, WebGL can rasterize the draw primitives, which include points, lines, and triangles. When rasterizing these primitives, WebGL calls another user-defined function called fragment shader. In other words, WebGL interpolates the values provided in the vertex shader function while it draws each pixel using the fragment shader function execution.

3. Fragment shader

The fragment shader is a function you have to write in GLSL whose goal is to generate a color for each pixel of the draw primitive currently being drawn. This function has little info per pixel, but you can provide it with everything required by using the varyings variables. These allow you to pass values from the vertex shader function to the fragment shader function.

Filtering Images with Kernels in WebGL with `<canvas>`

There are many ways to filter images, but the most common one involves the convolution operation. This is because when used on images, convolution applies a filter by taking the weighted sum of a square of pixels and assigning the resulting value to the current pixel. This logic is applied to every pixel the image consists of. Therefore, you can now imagine why convolution is one of the most relevant concepts when it comes to image processing.

The coefficients used to perform the weighted sum come from a matrix called kernel. The kernel represents the filter you want to apply through the convolution operation. So, by changing the kernel, the resulting image will change accordingly. Some kernels are more useful than others and can be used for blurring, sharpening, performing edge detection, and other operations. You can find a list of the most popular kernels on Wikipedia.

Now, let’s see how to implement kernel image filtering in WebGL.

Clone the GitHub repository that supports this article with the following command:

git clone https://github.com/Tonel/how-to-filter-an-image-in-webgl-imgly

Additionally, try the demo application by launching how-to-filter-an-image-in-webgl-imgly/index.html in your browser.

Otherwise, you can find the JavaScript function taking care of implementing the filter logic in WebGL below:

function filterImage(canvas, originalImage, kernel) {
  // assuming the kernel is a square matrix
  const kernelSize = Math.sqrt(kernel.length);
  const gl = canvas.getContext('webgl');

  // clearing the canvas
  gl.clearColor(1, 1, 1, 1);
  gl.clear(gl.COLOR_BUFFER_BIT);

  const vertexShaderSource = `
  attribute vec2 position;
  varying vec2 v_coordinate;
  void main() {
    gl_Position = vec4(position, 0, 1);
    v_coordinate = gl_Position.xy * 0.5 + 0.5;
  }
`;

  const fragmentShaderSource = `
  precision mediump float;
  // the varible defined in the vertex shader above
  varying vec2 v_coordinate;
  uniform vec2 imageSize;
  uniform sampler2D u_texture;
  
  void main() {
    vec2 position = vec2(v_coordinate.x, 1.0 - v_coordinate.y);
    vec2 onePixel = vec2(1, 1) / imageSize;
    vec4 color = vec4(0);
    mat3 kernel = mat3(
      ${kernel.join(',')}
    );
    // implementing the convolution operation
    for(int i = 0; i < ${kernelSize}; i++) {
      for(int j = 0; j < ${kernelSize}; j++) {
        // retrieving the sample position pixel
        vec2 samplePosition = position + vec2(i - 1 , j - 1) * onePixel;
        // retrieving the sample color
        vec4 sampleColor = texture2D(u_texture, samplePosition);
        sampleColor *= kernel\[i\][j];
        color += sampleColor;
      }
    }
    color.a = 1.0;
    gl_FragColor = color;
 }
`;

  const vertexShader = compileShader(gl, gl.VERTEX_SHADER, vertexShaderSource);
  const fragmentShader = compileShader(
    gl,
    gl.FRAGMENT_SHADER,
    fragmentShaderSource
  );

  // iniziailing the program
  const program = createProgram(gl, vertexShader, fragmentShader);

  const positionAttributeLocation = gl.getAttribLocation(program, 'position');

  const imageSizeLocation = gl.getUniformLocation(program, 'imageSize');

  // binding the position buffer to positionBuffer
  const positionBuffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);

  // using the program defined above
  gl.useProgram(program);
  // enabling the texcoord attribute
  gl.enableVertexAttribArray(positionAttributeLocation);
  // setting up the size of the image
  gl.uniform2f(imageSizeLocation, canvas.width, canvas.height);
  // telling positionAttributeLocation how to retrieve data out of positionBuffer
  gl.vertexAttribPointer(positionAttributeLocation, 2, gl.FLOAT, false, 0, 0);
  // provide the texture coordinates
  gl.bufferData(
    gl.ARRAY_BUFFER,
    new Float32Array([-1, -1, -1, 1, 1, -1, 1, 1, 1, -1, -1, 1]),
    gl.STATIC_DRAW
  );

  // loading the original image as a texture
  const texture = gl.createTexture();
  texture.image = new Image();

  // setting the anonymous mode
  // Learn more about it here:
  // https://developer.mozilla.org/en-US/docs/Web/API/HTMLImageElement/crossOrigin
  texture.image.crossOrigin = '';
  texture.image.src = originalImage.src;
  texture.image.onload = function () {
    gl.bindTexture(gl.TEXTURE_2D, texture);
    // setting the parameters to be able to render any image,
    // regardless of its size
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
    // loading the original image as a texture
    gl.texImage2D(
      gl.TEXTURE_2D,
      0,
      gl.RGBA,
      gl.RGBA,
      gl.UNSIGNED_BYTE,
      texture.image
    );
    gl.drawArrays(gl.TRIANGLES, 0, 6);
  };
}

function compileShader(gl, type, shaderSource) {
  const shader = gl.createShader(type);
  gl.shaderSource(shader, shaderSource);
  gl.compileShader(shader);

  const outcome = gl.getShaderParameter(shader, gl.COMPILE_STATUS);

  if (outcome === false) {
    // logging the error message on failure
    console.error(gl.getShaderInfoLog(shader));
    gl.deleteShader(shader);
  }

  return shader;
}

function createProgram(gl, vertexShader, fragmentShader) {
  const program = gl.createProgram();
  gl.attachShader(program, vertexShader);
  gl.attachShader(program, fragmentShader);
  gl.linkProgram(program);

  const outcome = gl.getProgramParameter(program, gl.LINK_STATUS);

  if (outcome === false) {
    // logging the error message on failure
    console.error(gl.getProgramInfoLog(program));
    gl.deleteProgram(program);
  }

  return program;
}

The filterImage() function is where the magic happens. It takes the following three parameters:

canvas: an Element object representing an HTML canvas element
originalImage: an Element object representing an HTML img element
kernel: an array containing the values of the kernel to use in the convolution operation

The first part of the function takes care of extracting the WebGLRenderingContext object representing a three-dimensional rendering context from the canvas element. Then, the vertex shader and fragment shader functions are defined as strings written in GLSL. Next, they are compiled and finally used to create a WebGL program.

The last part of the function creates a texture starting from the original image passed as a parameter and passes it to the WebGLRenderingContext to produce the final result. This represents the filtered image and is finally displayed by the browser in the canvas element.

The results produced by the filterImage() function depend on the kernel chosen, as you can verify by playing with the live demo you can find at the beginning of the article.

Et voilà! You just learned how to filter images in WebGL!

Final Considerations

As shown above, you can filter images with WebGL in Vanilla JavaScript with a hundred lines. At the same time, this cannot be considered an easy task to achieve. The reason is that getting into WebGL takes time and effort. Plus, you have to learn how to use GLSL to write the vertex shader and the fragment shader functions. So, things can get more complicated than expected.

Plus, you learned how to perform non-complex filters, but several filtering techniques are complex and do not involve the convolution operation. That also means that implementing them can be challenging and result in inefficient algorithms.

To avoid a headache, you should consider a complete and all-in-one solution like PhotoEditor SDK. Commercial solutions make things easier and shield you from all difficulties, offering features that would be complex, time-consuming, and challenging to implement. That is particularly true when it comes to using WebGL, which is PhotoEditor SDK’s main renderer.

You will be able to harness WebGL’s power without writing a single line of GLSL or knowing about its existence. WebGL will be used behind the scene for you! Keep also in mind that you would not be alone, since developers at IMG.LY are happy to provide support.

Filtering Images With PhotoEditor SDK

Read the article from the official documentation to learn how to get started with PhotoEditor SDK in HTML and Vanilla JavaScript. In detail, the Filters feature gives you more than 60 high-quality filters to play with. Achieve the following result with a few clicks:

Check out this feature on the PhotoEditor SDK demo page.

Conclusion

In this article, we looked at how to filter an image in WebGL with the HTML canvas element. Implementing a feature allowing users to filter images through a kernel-based approach in WebGL involves only a dozen of lines of code. However, understanding how WebGL works and coding in GLSL cannot be considered easy tasks.

As a result, you might want to avoid dealing with WebGL entirely. In this case, consider a commercial and easy-to-adopt solution using WebGL behind the scene – such as PhotoEditor SDK.

Thanks for reading! We hope that you found this article helpful. Feel free to reach out to us on Twitter with any questions, comments, or suggestions.‌

How To Resize and Compress an Image in JavaScript for Upload

Antonello — Fri, 04 Mar 2022 16:13:56 GMT

In this article, you will learn to compress an image in JavaScript and then upload it to Imgur. Similar to our previous guide on resizing images, and the one on drawing on images, you can accomplish everything with the HTML5 <canvas> element and we will involve any external libraries for this.

Smartphones cameras have become increasingly accurate and enhanced their photo quality for years. Consequently, their file size has grown as well. Since the speed of the average network has not improved at the same pace, it is essential to compress the images before uploading them.

Compressing is about downscaling an image. In other words, you want to reduce either its size or quality or both. By doing so, you can avoid uploading large images, saving the end-user time and money.

Compressing an Image With `<canvas>`

You can clone the GitHub repository that supports this article with the following commands:

git clone https://github.com/Tonel/how-to-compress-an-image-in-javascript-imgly

Then, you can try the demo application by opening the index.html file in your browser.

Otherwise, keep following this step-by-step tutorial and learn how to build the demo application.

1. Implementing the Compression Logic

You can compress an image by solely using the HTML <canvas> element. This is a powerful image manipulation tool that allows you to achieve many results, as we have already explored in our blog.
Now, let’s delve into how to compress an image with canvas:

function compressImage(imgToCompress, resizingFactor, quality) {
  // resizing the image
  const canvas = document.createElement('canvas');
  const context = canvas.getContext('2d');

  const originalWidth = imgToCompress.width;
  const originalHeight = imgToCompress.height;

  const canvasWidth = originalWidth * resizingFactor;
  const canvasHeight = originalHeight * resizingFactor;

  canvas.width = canvasWidth;
  canvas.height = canvasHeight;

  context.drawImage(
    imgToCompress,
    0,
    0,
    originalWidth * resizingFactor,
    originalHeight * resizingFactor
  );

  // reducing the quality of the image
  canvas.toBlob(
    (blob) => {
      if (blob) {
        // showing the compressed image
        resizedImage.src = URL.createObjectURL(resizedImageBlob);
      }
    },
    'image/jpeg',
    quality
  );
}

This function is an extension of the resizeImage() function defined in this article. So, follow the link to that tutorial to learn more about it.

What is new here are the last few lines, which take care of reducing the quality of the uploaded image based on the quality parameter. As you can see, the last part of the compressImage()is based on the toBlob() function. This transforms the image stored in the canvas into a Blob object and compresses it based on the last parameter passed to the function. This last parameter represents the quality of the target image file.

Just like resizingFactor, quality must contain a Number between 0 and 1.

Also, since the toBlob() function returns a Blob object, you can store it in a global variable. Then, you can use the blob object representing the compressed image file to upload it to your server. Let’s see how.

2. Uploading an Image to Imgur

First, you need an Imgur account. If you already have one, log in here. Otherwise, create a new account for free here.

Now, register an application here to have access to the Imgur API program. Fill out the form as follows:

Then, click on “Submit” and you should get access to this page.

Store your Client-ID in a safe place. You will need it later.
Now, you have everything required to start uploading images to your Imgur application.

Uploading an image to Imgur in JavaScript is easy and can be achieved with just a bunch of lines of code, as below:

// compressedImageBlob represents the compressed image Blob to upload
const formdata = new FormData();
formdata.append('image', compressedImageBlob);

fetch('https://api.imgur.com/3/image/', {
  method: 'POST',
  headers: {
    Accept: 'application/json',
    Authorization: 'Client-ID YOUR_CLIENT_ID',
  },
  body: formdata,
}).then((response) => {
  if (response?.status === 403) {
    console.error('Unvalid Client-ID!');
  } else if (response?.status === 200) {
    // retrieving the URL of the image
    // just uploaded to Imgur
    response.json().then((jsonResponse) => {
      console.log(`URL: ${jsonResponse.data?.link}`);
    });
  } else {
    console.error(response);
  }
});

Replace YOUR_CLIENT_ID with the Client-ID retrieved before, and you should now be able to use this snippet to upload your images to Imgur.

3. Putting It All Together

Now it is time to see the compressImage() function in action through a simple example.

<!DOCTYPE html>
<html>
  <body>
    <h1>Compress and Resize an Image</h1>
    <p>Upload an image and compress it or use the following demo image</p>
    <input id="upload" type="file" accept="image/*" />
    <div>
      <h2>Original Image</h2>
      <img
        style="margin-top: 5px;"
        id="originalImage"
        src="demo.jpg"
        crossorigin="anonymous"
      />
    </div>
    <div style="margin-top: 5px;">
      <span>Resizing: </span>
      <input type="range" min="1" max="100" value="80" id="resizingRange" />
    </div>
    <div style="margin-top: 5px; margin-left: 8px;">
      <span>Quality: </span>
      <input type="range" min="1" max="100" value="80" id="qualityRange" />
    </div>
    <h2>Compressed Image</h2>
    <div><b>Size:</b> <span id="size"></span></div>
    <img id="compressedImage" />
    <div>
      <button id="uploadButton">Upload to Imgur</button>
    </div>
    <script src="src/index.js"></script>
  </body>
</html>

const fileInput = document.querySelector('#upload');
const originalImage = document.querySelector('#originalImage');

const compressedImage = document.querySelector('#compressedImage');
const resizingElement = document.querySelector('#resizingRange');

const qualityElement = document.querySelector('#qualityRange');
const uploadButton = document.querySelector('#uploadButton');

let compressedImageBlob;

let resizingFactor = 0.8;
let quality = 0.8;

// initializing the compressed image
compressImage(originalImage, resizingFactor, quality);

fileInput.addEventListener('change', async (e) => {
  const [file] = fileInput.files;

  // storing the original image
  originalImage.src = await fileToDataUri(file);

  // compressing the uplodaded image
  originalImage.addEventListener('load', () => {
    compressImage(originalImage, resizingFactor, quality);
  });

  return false;
});

resizingElement.oninput = (e) => {
  resizingFactor = parseInt(e.target.value) / 100;
  compressImage(originalImage, resizingFactor, quality);
};

qualityElement.oninput = (e) => {
  quality = parseInt(e.target.value) / 100;
  compressImage(originalImage, resizingFactor, quality);
};

uploadButton.onclick = () => {
  // uploading the compressed image to
  // Imgur (if present)
  if (compressedImageBlob) {
    const formdata = new FormData();
    formdata.append('image', compressedImageBlob);

    fetch('https://api.imgur.com/3/image/', {
      method: 'POST',
      headers: {
        Accept: 'application/json',
        Authorization: 'Client-ID YOUR_CLIENT_ID',
      },
      body: formdata,
    }).then((response) => {
      if (response?.status === 403) {
        alert('Unvalid Client-ID!');
      } else if (response?.status === 200) {
        // retrieving the URL of the image
        // just uploaded to Imgur
        response.json().then((jsonResponse) => {
          alert(`URL: ${jsonResponse.data?.link}`);
        });
        alert('Upload completed succesfully!');
      } else {
        console.error(response);
      }
    });
  } else {
    alert('Rezind and compressed image missing!');
  }
};

function compressImage(imgToCompress, resizingFactor, quality) {
  // showing the compressed image
  const canvas = document.createElement('canvas');
  const context = canvas.getContext('2d');

  const originalWidth = imgToCompress.width;
  const originalHeight = imgToCompress.height;

  const canvasWidth = originalWidth * resizingFactor;
  const canvasHeight = originalHeight * resizingFactor;

  canvas.width = canvasWidth;
  canvas.height = canvasHeight;

  context.drawImage(
    imgToCompress,
    0,
    0,
    originalWidth * resizingFactor,
    originalHeight * resizingFactor
  );

  // reducing the quality of the image
  canvas.toBlob(
    (blob) => {
      if (blob) {
        compressedImageBlob = blob;
        compressedImage.src = URL.createObjectURL(compressedImageBlob);
        document.querySelector('#size').innerHTML = bytesToSize(blob.size);
      }
    },
    'image/jpeg',
    quality
  );
}

function fileToDataUri(field) {
  return new Promise((resolve) => {
    const reader = new FileReader();
    reader.addEventListener('load', () => {
      resolve(reader.result);
    });
    reader.readAsDataURL(field);
  });
}

// source: https://stackoverflow.com/a/18650828
function bytesToSize(bytes) {
  var sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];

  if (bytes === 0) {
    return '0 Byte';
  }

  const i = parseInt(Math.floor(Math.log(bytes) / Math.log(1024)));

  return Math.round(bytes / Math.pow(1024, i), 2) + ' ' + sizes[i];
}

The input element allows users to upload an image. This is then passed to the compressImage() function along with the resizingFactor and quality values retrieved from the respective range input HTML elements. This function takes care of compressing the image, displaying it, and storing its Blob representation to the global compressedImageBlob variable. Finally, compressedImageBlob is uploaded to Imgur when the “Upload to Imgur” button is clicked.
Notice that these two snippets are what allow you to implement the live example you can find at the beginning of the article.

Final Considerations

Compressing an image in Vanilla JavaScript is easy. You can achieve this with no extra libraries and in a dozen of lines of code. At the same time, the resizing part of the process relies on an image interpolation algorithm that changes according to the browser in use. This can lead to different results based on the end-user’s browser. Also, compressing while preserving quality is always tricky and can easily become a grueling goal to achieve.
If you want to avoid this stress, consider adopting a commercial and browser-consistent solution like PhotoEditorSDK. This library provides access to a wide set of tools that allow you to manipulate your images in many ways and with an advanced and easy-to-use UI.

Resizing an Image with PhotoEditor SDK

First, read this article from the official documentation to get started with PhotoEditorSDK in HTML and JavaScript. Then, you can use the transform tool to resize your image, as shown below:

Check out this feature on the PhotoEditorSDK demo page.

Conclusion

In this article, we learned how to downscale an image in JavaScript before uploading it to your server. In detail, we resized and reduced the quality of an uploaded image before uploading it to Imgur. Everything was achieved by using only the HTML5 <canvas> element. This is a powerful tool supported by most browsers that allows you to resize and change the quality of an image with just a few lines of code. On the other hand, this process may not be browser-consistent. That is why we also introduced a commercial and more reliable solution – such as PhotoEditorSDK.

Thanks for reading! We hope that you found this article helpful. Feel free to reach out to us on Twitter with any questions, comments, or suggestions. To stay in the loop with our latest articles and case studies, subscribe to our Newsletter.

CE.SDK v1.4.0 Release

Daniel — Tue, 22 Feb 2022 12:34:56 GMT

Six weeks have passed since our last release, where we shipped our new and improved default UI, Placeholders, and a preview for our new Dashboard.

This time, we focused heavily on making content discovery and provisioning better for you and your customers, as well as automation and API-driven design. Our release includes:

Custom Asset Libraries & Management
Internationalization
Headless Mode & APIs
Other
Outlook on upcoming releases

Asset Libraries & Management

With this release, we improve our library system and overall usability. CE.SDK 1.4.0 introduces a new toolbar to let your users quickly access their favorite images, shapes, stickers, or text assets.

Even more importantly, we added a General Asset Source API that allows you to add any external library sources you like. This way, you can provide custom assets to your users, such as images, stickers, or anything else.
Check out our Documentation on Custom Asset Source Integration.

Internationalization (i18n)

CE.SDK now supports the easy translation of the default UI into any language, including full i18n support. We ship with English and German locales, but have successfully tested it with other languages.
Learn more about adding translations to CE.SDK in our Documentation.

Headless Mode & APIs

We are finally extending CE.SDK with a full headless mode that is usable without a user interface in a browser. You can use it to load, modify, and export scene templates programmatically without the need for any user interaction. Use it to create previews of your scenes with custom user content without opening the editor, or export your designs in multiple resolutions on demand.

Included APIs

Scene API lets you create, load, save, or export scenes to a rasterized image format.
Block API lets you create, read, update and delete any item (so-called blocks) in the scene; modifying properties like position, size, or content is guaranteed easy.
Layouts API lets you use advanced layout capabilities: choose from modes like absolute positioning, relative positioning in percent relative to the parent, or even auto layout. An even auto-layout lets you auto-adapt the size of a block according to its parent size and siblings.

Other

Snapping now works with multiple selected items.
Slider input values now allow manual input to give you more control.
Easy asset library access in the UI.
Changing content in Preview Mode to preview your template settings does not alter your template anymore.

Upcoming

We are currently working hard to improve the overall CE.SDK Editor experience for you and your customers. That includes adding features like strokes, gradients, grouping, but also improving the rendering performance to create more complex designs. Additionally, we are improving asset discovery within the editor.

Excitingly, we are working hard to bring our VE.SDK video features to CE.SDK! In the near future, you can use videos and animated GIFs inside any template and export short videos from within CE.SDK for Web.

How To Scale, Crop, Flip, and Filter an Image in CSS

Antonello — Wed, 09 Feb 2022 19:41:38 GMT

In this article, you will learn how to perform popular image processing operations in CSS. That is possible thanks to the several features introduced by CSS3, which allow you to change how the images look to end-users. To learn instead about image manipulation operations for other frameworks, you can head on here: React or vanilla JS.

First, let’s delve into how image manipulation works in CSS. Then, let’s finally see the most common CSS image processing operations in action.

How CSS Image Manipulation Work

CSS does not allow you to deal with files, data formats, or data transformation. In other words, CSS cannot change an image on a deep level, yet with CSS, you can determine how a browser displays elements.

As a result, as opposed to what would happen when using an HTML canvas element, no pixel manipulation will be executed behind-the-scene. Therefore, your CSS rules won’t affect the file that represents your image. What changes is how your image is displayed frontend-side.

You can flip, blur, and crop an image in CSS, but when downloading the image with the right-click option, the image saved will always be a copy of the original image file. Read this page from MDN Web Docs, if you want to learn more about this phenomenon.

Prerequisites

The CSS properties you will see are widely based on CSS3 and are not supported by Internet Explorer. To verify if your target browsers support a CSS property, visit MDN Web Docs and explore the Browser compatibility section.

Image Processing in CSS

Let’s look at how to perform scaling, cropping, flipping, and filtering an image in CSS.

Scaling an image with `object-fit: scale-down`

The object-fit CSS property allows you to define how the content of an HTML <img> element should be resized to fit its container. In particular, scale-down acts as if none or contain were set, depending on which would result in a smaller image.

When it acts like none, the image will keep its original intrinsic size and not be scaled down. On the contrary, when it acts like contain the image will be scaled down to fit the container while maintaining its aspect ratio.

Test this CSS rule in the live demo below:

In particular, this is the CSS rule where the magic happens:

.manipulated-image {
  object-fit: scale-down;
  height: 100px;
}

Cropping an image with `object-fit` and `object-position`

By employing object-fit in conjunction with object-position, you can show a part of your image, just if it has been cropped. In detail, the object-position CSS property allows you to specify the positioning of your <img> element within its container box. This happens by passing this CSS property a position parameter containing from one to four values. This parameter is what defines the position of the <img> element.

By using object-fit to set how your <img> element should fit its container and by positioning it with object-position, you can show only a part of your image as shown in the live example below:

This is the CSS rule to look attention to:

.manipulated-image {
  object-fit: cover;
  object-position: -150px -25px;
}

Flipping an image with `transform`

The transform CSS property is a powerful tool that gives you the ability to rotate, scale, skew, or translate an<img> element. Specifically, you can flip an image by using the scaleX() and scaleY() transformation functions, or by adopting the rotateX() or rotateY() functions.

scaleX() and scaleY() define a transformation that resizes an HTML element along the x-axis and y-axis, respectively. Both accept a number representing the scaling factor as a parameter. Learn how to flip an image vertically with scaleX() in the example below:

In detail, this is the CSS rule to flip an image vertically:

.manipulated-image {
  transform: scaleX(-1);
}

Similarly, rotateX() and rotateY() define a transformation that rotates an HTML element without deforming it around the horizontal axis and vertical axis, respectively. Both require an angle representing the angle of rotation as a parameter. Now, let’s see how to flip an image vertically with rotateY() in the example below:

Specifically, this is what the CSS rule to flip an image vertically looks like:

.manipulated-image {
  transform: rotateY(180deg);
}

Filtering an image with `filter`

The filter CSS property allows you to apply visual effects on an <img> element. This property provides access to several built-in effects, such as blur(), grayscale(), contrast(), and many others. If required, you can also define your custom image filtering functions.

CSS filters are a powerful tool that is typically used to adjust how an image is displayed by the browser and shown to the end-user. Using them is really easy, as shown in the live demo below:

These are the CSS rules where the filtering operations are defined:

.blurred-image {
  filter: blur(2px);
}
.grayscale-image {
  filter: grayscale(80%);
}
.high-contrast-image {
  filter: contrast(150%);
}

CSS-Only Image Editor vs PhotoEditor SDK

Image manipulation in CSS is an easy and straightforward way to manipulate how an image is displayed with no effort. On the other hand, you cannot save or export the manipulated images with CSS. This means that it is not possible to implement a file image editor based solely on CSS.

Therefore, you can rely on CSS only if you are interested in changing how your images are rendered by the browser. Conversely, if you are looking for a way to manipulate your images and then export them as files, then you need a much more advanced tool. In this case, a commercial solution like IMG.LY’s Photo Editor SDK will provide you with a complete, fast, and easy-to-use image editor.

Conclusion

Today we learned how to use CSS to control how browsers display images. In detail, we delved into how to scale, crop, flip, and filter an image in CSS. These represent the most common image manipulation operations and allow you to transform your image CSS-side. As we have learned, these operations only determine how your browser renders an image. They do not change an image as a data structure.
This means that it is not possible to create an image editor that allows users to export manipulated images while using only CSS. So, if this is what you were looking for, you should consider adopting a fast, advanced, and ready-to-use solution – such as PhotoEditorSDK. And you can get right into the PhotoEditorSDK or VideoEditorSDK and its myriad integrations by starting with a free trial over here.

Thanks for reading! We hope that you found this article helpful. Feel free to reach out to us on Twitter with any questions, comments, or suggestions.

How To Manipulate an Image With Jimp in React

Antonello — Wed, 26 Jan 2022 13:36:27 GMT

In this article, you will learn how to use jimp in React to perform the most common image processing operations. As you are about to learn, this popular JavaScript library allows you to effortlessly manipulate an image with a few lines of code.

What is Jimp?

Jimp stands for JavaScript Image Manipulation Program, and it is one of the most popular open-source image processing libraries, with more than one million weekly downloads. As stated on the official GitHub page, Jimp is a Promise-based library that relies entirely on Vanilla JavaScript. This means that it has no native or external dependencies, as you can verify here. This makes it more reliable and lightweight.

Jimp comes with several functions that allows you to transform your images in endless ways. Particularly, it includes the resize(), blur(), crop(), rotate() functions and many others. At the time of writing, these are image formats supported by Jimp:

Let’s now see how to use the Jimp transformation functions.

Prerequisites

This is the list of prerequisites you need to make the React applications using Jimp we are about to build works:

You can add jimp to your project’s dependencies by launching the following command:

npm install jimp

Now, you have everything required to start getting your hands dirty with Jimp in React.

Basic Image Processing in Jimp

All the Jimp usage examples presented below share the same logic. So, let’s address it immediately.

Specifically, this is what a basic JimpDemo component looks like:

import Jimp from 'jimp';
import { useEffect, useState } from 'react';

export function JimpDemo({ imageUrl }) {
  const [jimpImage, setJimpImage] = useState(undefined);
  const [image, setImage] = useState(undefined);
  const [transformedImage, setTransformedImage] = useState(undefined);

  // loading an image every time imageUrl changes
  useEffect(() => {
    const loadImage = async () => {
      // generating the Jimp data structure
      // loading an image from an URL
      const jimpImage = await Jimp.read(imageUrl);
      setJimpImage(jimpImage);

      // transforming jimpImage into its Base64 representation
      // and storing it
      const image = await jimpImage.getBase64Async(Jimp.MIME_JPEG);
      setImage(image);
    };

    loadImage();
  }, [imageUrl]);

  // generating the transformed image
  // as soon as the Jimp data structure is ready
  useEffect(() => {
    if (jimpImage) {
      const transformImage = async () => {
        // performing the Jimp image processing operation
        // on jimpImage...

        // e.g. jimpImage.crop(100, 100)

        // storing the transformed image
        // in Base64 format
        const transformedImage = await jimpImage.getBase64Async(Jimp.MIME_JPEG);
        setTransformedImage(transformedImage);
      };

      transformImage();
    }
  }, [jimpImage, height, width]);

  return image && jimpImage ? (
    <>
      <h1>Original Image</h1>
      <img className="originalImage" src={image} alt="Original" />
      <h1>Transformed Image</h1>
      <img
        className="transformedImage"
        src={transformedImage}
        alt="Transformed"
      />
    </>
  ) : (
    <>Loading...</>
  );
}

First, the imageUrl prop value storing the URL to the image you want to transform is used to generate a Jimp data structure. This happens in the first useEffect() function when launching Jimp.read(). This function takes the path to a file or a URL and returns a Promise with the Jimp data structure you call the manipulation functions on. Also, the Base64 representation of the original image is stored to be able to show it in the HTML <img> tag.

Keep in mind that the function passed to useEffect() should not be async. This is why an internal async function was defined and then called instead.

Then, the second useEffect() function is called as soon as jimpImage is initialized and takes care of performing the transformation function on the original image. Again, the Base64 representation of the transformed image is saved.

Finally, both the original and transformed images are rendered by the component. As you can imagine, what will change in the following examples will be the Jimp image transform function called, which might also require extra props.

Cropping an image

The Jimp crop() function crops an image at a given point to a given size.

jimpImage.crop(x, y, width, height)

x: the x coordinate to crop from
y: the y coordinate to crop from
width: the width of the crop region
height: the height of the crop region

See it in action in the live example below:

Resizing an image

The Jimp resize() function resizes an image to set width and height using a 2-pass bilinear algorithm.

jimpImage.resize(width, height);

width: the width to resize the image
height: the height to resize the image to

See it in action in the live example below:

Rotating an image

The Jimp rotate() function rotates an image clockwise by a number of degrees. Note that the width and height of the image will be resized accordingly by default.

jimpImage.rotate(degrees);

degrees: the number of degrees to rotate the image by

See it in action in the live example below:

Filtering an image

Although Jimp does not come with a single filtering function, it offers several functions to achieve the desired results. For example, you can implement an image filtering feature by harnessing the blur(), color(), dither(), normalize(), or threshold() functions.

Let’s see some of them in action in the live demo below:

Jimp vs PhotoEditor SDK

Undoubtedly, Jimp is a powerful tool. It’s an amazingly lightweight and simple library that offers so many image manipulations. On the other hand, achieving complex results requiring multiple manipulations to be performed on the end user’s needs could become challenging. In detail, this would require an advanced, robust, reliable, and easy-to-use UI that only a commercial solution such as IMG.LY’s PhotoEditor SDK can offer.

You could build such a UI yourself on top of Jimp, but this would take a lot of time and effort. This is especially true if you want to implement a general-purpose application, such as an image processing application. For limited and specific tasks without any user interaction, Jimp is the ideal solution.

Also, now you can take a look at the CreativeEditor SDK which is a more advanced API solution and obvious alternative to Jimp!

Conclusion

Today we learned about the jimp JavaScript library, its dependencies, how it works, and how to use it in React real-world scenarios. Specifically, this npm library comes with several image manipulation functions that can be used with a few lines of code. However, building a complex image editing application on top of it would require an entire UI, and you might want to avoid spending time on it. In this case, you should look into a more advanced, fully-featured, and ready-to-use solution – such as PhotoEditorSDK.

Thanks for reading! We hope that you found this article helpful. Feel free to reach out to us on Twitter with any questions, comments, or suggestions.

To stay in the loop with our latest articles and case studies, subscribe to our Newsletter.

CE.SDK v1.3.0 Release

Daniel — Mon, 17 Jan 2022 15:07:02 GMT

After another six weeks of work, we are happy to ship another update of our CreativeEditor SDK (CE.SDK). This release is again packed with great features to enhance the developer experience and user experience. It includes:

New improved Default UI
Lockable Design & Placeholders
SVG Assets
Multi Selection
Nudging (moving elements with arrow and shift key)
Create scenes from an Image.

New Improved Default UI

That is the first release to introduce a new UI type for CE.SDK, which we call Default UI. It comes with a slick interface and focuses on fast design adoption. The Default UI removes the need for inspectors for many editing cases, thus leaving your users undistracted and focused on adapting or creating designs.

Check our official documentation to learn about configuring the UI Elements.

Lockable Design & Placeholders

In the previous version, CE.SDK introduced the roles “creator” and “adopter” – differentiating between template creation and adoption. In creator mode, you have the liberty to create a design from scratch or change existing designs.

When creators are ready to share a design, they define which design elements can be changed when opening the editor in the “adopter” role. Typically, creators will allow adopters to exchange content, but also the options to style or arrange an item can be controlled.

When the editor is opened in the “adopter” role (Default UI), items that are allowed to change will be highlighted and images will be shown with an overlay indicating that the content shall be replaced before exporting or saving the design.

Find more information about placeholders and constraints in our official documentation.

Dashboard

Creating and adopting designs is one part, but organizing designs is another. With this release, we also introduced a dashboard to create and organize your designs for your users. Reach out to us if you want to see a demo of the dashboard in action and understand.

SVG Assets

With this version, we introduced SVG as an asset-type alternative to JPEG and PNG image formats. This has the benefit that standard SVG Stickers can just be used or uploaded in every design. This has the major advantage that stickers will be crisp at any resolution while coming with a very low download size. Reducing the server load and data transfer for you and your users. In this turn, we also replaced all our example stickers with their SVG equivalents.

Multi Selection

Positioning multiple items at once has been a pain before this release, but not anymore. With the new multi-selection interaction, users can select two or more items at once. Just hold down the “shift” modifier key and start selecting multiple items. Once selected, they can be moved around simultaneously.

Nudging

Always wanted to fine-tune your design and pixel-perfectly align your content?

You can now use the arrow keys (⬅️, ➡️, ⬆️ ⬇️) to nudge the selected items to their correct position pixel by pixel. You can also hold down shift to nudge even further.

Create Scene from Image

Before this release, the CE.SDK could either start from an empty or a predefined scene. From now on, there is also an option to start the editor with any image. It will auto-create a page with the size of the given image and insert the image into this one, allowing a more content-driven creation process. Learn how to create a scene from an image in our documentation.

Minor changes

Besides the major features, there are a ton of small improvements. These include:

UI interface to rename items and pages to a user-defined name
Ability to choose between two UI types, “default” and “advanced”
Searching in image libraries now uses i18n translations
Ability to change the UI Font.

Feel free to send us any questions, or let us know what you think on Twitter.

Thank you for your time!

To stay in the loop with our latest articles and case studies, subscribe to our Newsletter.

Web Development – IMG.LY Blog

CE.SDK v1.13 Release Notes

Enhance Video Composition: Duration and Time Offset Control

Elevate iOS Video Experience with More Codecs

Easily Select Blocks in Complex Designs

Simplify Group Interaction and Enhance Mobile Usability

Configure via API

Outlook: Easily Remove Backgrounds in Your Browser

IMG.LY Partners with Getty Images to Revolutionize Image Integration

Empower your Developers and Users

Get Started

How To Build a Video Editor With Wasm in React

What is WebAssembly?

What is ffmpeg.wasm?

Building a Video Editor in React with WASM

Prerequisites

Initializing a React Project

Installing the Project’s Dependencies

Uploading a Video to the Video Editor

Playing a Video in React

Cutting a Video in React

Converting a Video to GIF in React

Putting It All Together

Commercial Alternative

Conclusion

CE.SDK v1.9.0 Release

Video Support for Web

Video Timelines

Web Audio Support

Native iOS Support

Spot and Named Colors

More Showcases

How to Apply Filters and Effects to an Image in Flutter

Introduction

Flutter ImageFiltered Widget

ImageFilter.blur constructor

ImageFilter.matrix constructor

ColorFiltered Effects

Final Thoughts

How to Draw on Images in React Native

Canvas Integration for Images

Drawing with Canvas

Drawbacks of the Canvas Method

PhotoEditor SDK Integration for Drawing on Your Images

Conclusion

CE.SDK v1.8.0 Release

CE.SDK v1.8.0

Node.js Platform Support

Common Touch Gesture Support for iOS

Emoji Support

Image Straightening in Advanced & Default UI

Multi Selection Rotation

Outlook

How To Build a Video Player in JavaScript

Setting Up the Project

Customize the Video Player With JavaScript

Add Play and Pause

Add Progress Bar

Closing Notes

CE.SDK v1.7.0 Release and Roadmap

CE.SDK v1.7.0

Auto Close Option in Web UI Configuration for Asset Libraries

Custom URI Resolver

Improving Mobile Touch Support

Rotation of Groups and Hierarchies

Custom Asset Source UI

Moving from Roles to Abilities: Disconnecting UI Variants from Roles

Shadows in Advanced UI

Blur API

Effects API

Image Fills API

Image Straighten API

Improved Group Selection and API

Scopes API

Shadows API

Zoom API

Product Roadmap

Thanks for reading! Let us know what you think on Twitter! To stay in the loop, subscribe to our Newsletter.

How To Build a Canva Clone with CE.SDK

What is Canva?

What is `ffmpeg.wasm`?

Filtering Images with Kernels in WebGL with `<canvas>`