Visual Prompting - CE.SDK Glossary

Definition

Visual prompting guides an image model with a picture of what the user has in mind, alongside or instead of a written prompt. The user lays things out on a canvas, for example a rough sketch of the layout plus a photo with the note “keep this style”. The AI looks at the whole arrangement, notes included, and creates a new image based on it. This helps with changes that are quicker to show than to describe, such as where a product should go or which photo has the right colors, since a note can sit right next to the thing it refers to.

In CE.SDK, the user builds this arrangement on the canvas, and the AI creates the new image from everything on it.

How It Works in the Editor

Step 01

Lay out the idea on the canvas

The user places what the model should work from, such as photos, shapes, or a quick sketch, and writes instructions as text on the canvas itself, for example a note on a headline saying “bigger and bolder”.

Step 02

Pick a page action

With the page selected, Turn Page into Image follows the notes written on it. Remix Page… works from a typed instruction, such as “rearrange the layout so the headline sits at the bottom”.

Step 03

Get the result on a new page

The generated picture fills a new page of the same size, and the original page with all its separate elements stays as it was.

Step 04

Adjust and run it again

To change the outcome, the user moves elements or rewrites a note on the original page and runs the action again. A result worth keeping is an ordinary image the user can crop, place elsewhere in the design, or refine with Image to Image.

Notes That Stay Out of the Picture

Turn Page into Image tells the model to follow the instructions written on the page and to keep those instructions out of the finished image, so a user can write “make this sky stormy” on top of a photo without the words showing up in the result.

Choosing a Model

Visual prompting needs an image model that takes a picture as input and follows instructions written inside it. The page actions only appear for models that support them. The AI Editor capabilities name the image editing models available today, and IMG.LY’s article What is Visual Prompting? walks through the idea in more depth.

Use Cases

From Sketch to Layout

A rough arrangement of boxes and placeholder text goes to the model with a request for a poster or social graphic that follows that composition.

Matching a Reference Look

A product photo sits next to a reference image with the note “same lighting and colors”, and the model is asked to render them as one picture in that look.

Feedback Written on the Design

A reviewer’s notes and the design stay in one file, so after each round of comments the same page can be run through the model again.

Trying Layout Alternatives

Remix Page… with an instruction like “move the product to the left and add more empty space” produces a new variation while the original page stays available for comparison.

Availability

Teams can add visual prompting to CE.SDK in the browser, with plain JavaScript or React, Vue, Angular, Svelte, SvelteKit, Next.js, or Nuxt.js, and to Electron desktop apps.