Integrate a Custom Asset Source

In this example, we will show you how to integrate your custom asset sources into CE.SDK.

Explore a full code sample on Stackblitz or view the code on Github.

With CE.SDK you can directly add external image providers like Unsplash or your own backend. A third option we will explore in this guide is using the engine's Asset API directly. Follow along with this example while we are going to add the Unsplash library.

Adding an asset source is done creating an asset source definition and adding it using asset.addSource(source: AssetSource). The asset source needs a unique identifier as part of an object implementing the interface of the source. All Asset API methods require the asset source's unique identifier.

The most important function to implement is findAssets(query: AssetQueryData): AssetQueryResult. With this function alone you can define the complete asset source. It receives the asset query as an argument and returns a promise with the results.

The argument is the queryData and describes the slice of data the engine wants to use. This includes a query string and pagination information.
The result of this query is a promise that, besides the actual asset data, returns information like the current page, the next page and the total number of assets available for this specific query.

Returning a promise gives us great flexibility since we are completely agnostic of how we want to get the assets. We can use fetch, XMLHttpRequest or import a library to return a result.

Let us implement an Unsplash asset source. Please note that this is just for demonstration purposes only and may not be ideal if you want to integrate Unsplash in your production environment.

We will use the official unsplash-js library to query the Unsplash API (Link to Github page). According to their documentation and guidelines, we have to create an access key and use a proxy to query the API, but this is out of scope for this example. Take a look at Unsplash's documentation for further details.

Unsplash has different API endpoints for different use cases. If we want to search we need to call a different endpoint as if we just want to display images without any search term.

Therefore we need to check if the query data contains a query string. If findAssets was called with a non-empty query we can call the asynchronous method search.getPhotos. As we can see in the example, we are passing the query arguments to this method.

queryData.query: The current search string from the search bar in the asset library.
queryData.page: For Unsplash specifically the requested page number starts with 1. We do not query all assets at once but by pages. As the user scrolls down more pages will be requested by calls to the findAssets method.
queryData.perPage: Determines how many assets we want to have included per page. This might change between calls. For instance, perPage can be called with a small number to display a small preview, but with a higher number e.g. if we want to show more assets in a grid view.

Once we receive the response and check for success we need to map Unsplash's result to what the asset source API needs as a result. The CE.SDK expects an object with the following properties:

assets: An array of assets for the current query. We will take a look at what these have to look like in the next paragraph.
total: The total number of assets available for the current query. If we search for "Cat" with perPage set to 30, we will get 30 assets, but total likely will be a much higher number.
currentPage: Return the current page that was requested.
nextPage: This is the next page that can be requested after the current one. Should be undefined if there is no other page (no more assets). In this case we stop querying for more even if the user has scrolled to the bottom.

Every image we get as a result of Unsplash needs to be translated into an object that is expected by the asset source API. We will describe every mandatory and optional property in the following paragraphs.

id: The id of the asset (mandatory). This has to be unique for this source configuration.

locale (optional): The language locale for this asset is used in label and tags.

label (optional): The label of this asset. It could be displayed in the tooltip as well as in the credits of the asset.

tags (optional): The tags of this asset. It could be displayed in the credits of the asset.

meta: The meta object stores asset properties that depend on the specific asset type.

uri: For an image asset this is the URL to the image file that will be used to add the image to the scene.

Note that we have to use the Unsplash API to obtain a usable URL at first.

thumbUri: The URI of the asset's thumbnail. It could be used in an asset library.

blockType: The type id of the design block that should be created when this asset is applied to the scene.

If omitted, CE.SDK will try to infer the block type from an optionally provided mimeType property (e.g. image/jpeg) or by loading the asset data behind uri and parsing the mime type from that. However, this will cause a delay before the asset can be added to the scene, which is why it is always recommended to specify the blockType upfront.

fillType: The type id of the fill that should be attached to the block when this asset is applied to the scene.

If omitted, CE.SDK will default to a solid color fill //ly.img.ubq/fill/color.

shapeType: The type id of the shape that should be attached to the block when this asset is applied to the scene.

If omitted, CE.SDK will default to a rect shape //ly.img.ubq/shape/rect.

kind: The kind that should be set to the block when this asset is applied to the scene.

If omitted, CE.SDK will default to an empty string.

width: The original width of the image.

height: The original height of the image.

context: Adds contextual information to the asset. Right now, this only includes the source id of the source configuration.

credits (optional): Some image providers require to display credits to the asset's artist. If set, it has to be an object with the artist's name and a url to the artist page.

utm (optional): Some image providers require to add UTM parameters to all links to the source or the artist. If set, it contains a string to the source (added as utm_source) and the medium (added as utm_medium)

After translating the asset to match the interface from the asset source API, the array of assets for the current page can be returned.

In case, we've encountered an error, we need to reject the promise. Since we use the async/await language feature this means throwing a new error.

An empty result can be returned instead of the result object if for some reason it does not make sense to throw an error but just show an empty result.

Going further with our Unsplash integration we need to handle the case when no query was provided. Unsplash requires us to call a different API endpoint (photos.list) with slightly different parameters but the basics are the same. We need to check for success, calculate total and nextPage and translate the assets.

We have already seen that an asset can define credits for the artist. Depending on the image provider you might need to add credits and the license for the source. In case of Unsplash, this includes a link as well as the license of all assets from this source.

Local Asset Sources#

In many cases, you already have various finite sets of assets that you want to make available via asset sources. In order to save you the effort of having to implement custom asset query callbacks for each of these asset sources, CE.SDK also allows you to create "local" asset sources, which are managed by the engine and provide search and pagination functionalities.

In order to add such a local asset source, simply call the addLocalSource API and choose a unique id with which you can later access the asset source.

The addAssetToSource(sourceId: string, asset: AssetDefinition): void API allows you to add new asset instances to your local asset source. The local asset source then keeps track of these assets and returns matching items as the result of asset queries. Asset queries return the assets in the same order in which they were inserted into the local asset source.

Note that the AssetDefinition type that we pass to the addAssetToSource API is slightly different than the AssetResult type which is returned by asset queries. The AssetDefinition for example contains all localizations of the labels and tags of the same asset whereas the AssetResult is specific to the locale property of the query.


import CreativeEngine from 'https://cdn.img.ly/packages/imgly/cesdk-engine/1.39.0/index.js';

import * as unsplash from './vendor/unsplash-js.esm.js';

const unsplashApi = unsplash.createApi({
  apiUrl: '...'
});

const EMPTY_RESULT = {
  assets: [],
  total: 0,
  currentPage: 0,
  nextPage: undefined
};

const findUnsplashAssets = async (queryData) => {
  /* Unsplash page indices are 1-based. */
  const unsplashPage = queryData.page + 1;

  if (queryData.query) {
    const response = await unsplashApi.search.getPhotos({
      query: queryData.query,
      page: unsplashPage,
      perPage: queryData.perPage
    });
    if (response.type === 'success') {
      const { results, total, total_pages } = response.response;

      return {
        assets: await Promise.all(results.map(translateToAssetResult)),

        total,
        currentPage: queryData.page,
        nextPage:
          queryData.page + 1 < total_pages ? queryData.page + 1 : undefined
      };
    } else if (response.type === 'error') {
      throw new Error(response.errors.join('. '));
    } else {
      return Promise.resolve(EMPTY_RESULT);
    }
  } else {
    const response = await unsplashApi.photos.list({
      orderBy: 'popular',
      page: unsplashPage,
      perPage: queryData.perPage
    });

    if (response.type === 'success') {
      const { results, total } = response.response;
      const totalFetched = queryData.page * queryData.perPage + results.length;

      const nextPage = totalFetched < total ? queryData.page + 1 : undefined;

      return {
        assets: await Promise.all(results.map(translateToAssetResult)),

        total,
        currentPage: queryData.page,
        nextPage
      };
    } else if (response.type === 'error') {
      throw new Error(response.errors.join('. '));
    } else {
      return Promise.resolve(EMPTY_RESULT);
    }
  }
};

const getUnsplashUrl = async (unsplashResult) => {
  const trackDownloadResponse = await unsplashApi.photos.trackDownload({
    downloadLocation: unsplashResult.links.download_location
  });

  if (trackDownloadResponse.type === 'error') {
    throw new Error(trackDownloadResponse.errors.join('. '));
  }
  return trackDownloadResponse.response?.url;
};

const config = {
  license: 'mtLT-_GJwMhE7LDnO8KKEma7qSuzWuDxiKuQcxHKmz3fjaXWY2lT3o3Z2VdL5twm',
  userId: 'guides-user',
  baseURL: 'https://cdn.img.ly/packages/imgly/cesdk-engine/1.39.0/assets'
};

CreativeEngine.init(config).then(async (engine) => {
  document.getElementById('cesdk_container').append(engine.element);

  const scene = engine.scene.create();
  const page = engine.block.create('page');
  engine.block.appendChild(scene, page);
  engine.scene.zoomToBlock(page);

  const customSource = {
    id: 'unsplash',
    findAssets: findUnsplashAssets,
    credits: {
      name: 'Unsplash',
      url: 'https://unsplash.com/'
    },
    license: {
      name: 'Unsplash license (free)',
      url: 'https://unsplash.com/license'
    }
  };

  engine.asset.addSource(customSource);

  const result = await engine.asset.findAssets(customSource.id, {
    page: 0,
    perPage: 3
  });
  const asset = result.assets[0];

  await engine.asset.apply(customSource.id, asset);

  const localSourceId = 'background-videos';
  engine.asset.addLocalSource(localSourceId);

  engine.asset.addAssetToSource(localSourceId, {
    id: 'ocean-waves-1',
    label: {
      en: 'relaxing ocean waves',
      es: 'olas del mar relajantes'
    },
    tags: {
      en: ['ocean', 'waves', 'soothing', 'slow'],
      es: ['mar', 'olas', 'calmante', 'lento']
    },
    meta: {
      uri: `https://example.com/ocean-waves-1.mp4`,
      thumbUri: `https://example.com/thumbnails/ocean-waves-1.jpg`,
      mimeType: 'video/mp4',
      width: 1920,
      height: 1080
    },
    credits: {
      name: 'John Doe',
      url: 'https://example.com/johndoe'
    }
  });
});

async function translateToAssetResult(image) {
  const artistName = image?.user?.name;
  const artistUrl = image?.user?.links?.html;

  return {
    id: image.id,
    locale: 'en',
    //
    label: image.description ?? image.alt_description ?? undefined,
    tags: image.tags ? image.tags.map((tag) => tag.title) : undefined,

    meta: {
      uri: await getUnsplashUrl(image),
      thumbUri: image.urls.thumb,
      blockType: '//ly.img.ubq/graphic',
      fillType: '//ly.img.ubq/fill/image',
      shapeType: '//ly.img.ubq/shape/rect',
      kind: 'image',
      width: image.width,
      height: image.height
    },

    context: {
      sourceId: 'unsplash'
    },

    credits: artistName
      ? {
          name: artistName,
          url: artistUrl
        }
      : undefined,

    utm: {
      source: 'CE.SDK Demo',
      medium: 'referral'
    }
  };
}