---
title: "Benchmark: Structured JSON Scene"
description: "15 image models compared on the canonical \"Structured JSON Scene\" prompt (Structured Spec Adherence)."
url: "https://img.ly/ai-benchmarks/prompts/j01-json-scene/"
type: "benchmark-prompt"
category: "Structured Spec Adherence"
suite: "pilot-0"
---

> This is the markdown version of [Benchmark: Structured JSON Scene](https://img.ly/ai-benchmarks/prompts/j01-json-scene/). For all pages in one file, see [llms-full.txt](https://img.ly/llms-full.txt). For an index of all available pages, see [llms.txt](https://img.ly/llms.txt).

---

# Benchmark: Structured JSON Scene

Category: Structured Spec Adherence.
Use cases: Marketing & ad creative (U1), Personalization at scale (U6).
Suite: pilot-0. Every model runs the identical prompt with default
parameters and fixed seeds. No per-model tuning.

## Why this benchmark

Structured JSON prompts are how creative-automation pipelines, agents and CE.SDK integrations actually drive image models, and nobody benchmarks it. Each spec field becomes a checklist item plus an automated color check, the modality closest to IMG.LY's product reality.

## Canonical prompt

> Render the following scene exactly: {"subject":"a lighthouse keeper","gaze":"toward the sea","wardrobe":"yellow raincoat","lighting":"golden hour from the left","composition":"rule of thirds, subject on the right","colorPlate":[{"name":"coat","hex":"#F4C430","target_pct":15},{"name":"sky","hex":"#F6A15C","target_pct":40}]}

## Scored assertions

- subject is a lighthouse keeper
- gaze directed toward the sea
- wearing a yellow raincoat
- golden-hour light from the left
- subject placed on the right (rule of thirds)
- coat color near #F4C430
- sky color near #F6A15C

## Results by model

### FLUX.2

[Model profile](https://img.ly/ai-benchmarks/models/flux-2.md)

- seed 1111: 1024×768, 2.0s, $0.013, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2--s1111.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 2222: 1024×768, 2.0s, $0.013, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2--s2222.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 3333: 1024×768, 2.1s, $0.013, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2--s3333.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 2.0 / 5

### FLUX.2 [pro]

[Model profile](https://img.ly/ai-benchmarks/models/flux-2-pro.md)

- seed 1111: 1024×768, 23.4s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-pro--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 2.0 / 5
- seed 2222: 1024×768, 11.7s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-pro--s2222.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 3333: 1024×768, 15.8s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-pro--s3333.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5

### FLUX.2 [dev] Turbo

[Model profile](https://img.ly/ai-benchmarks/models/flux-2-turbo.md)

- seed 1111: 1024×768, 2.1s, $0.015, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-turbo--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 2222: 1024×768, 2.4s, $0.015, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-turbo--s2222.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 3333: 1024×768, 2.2s, $0.015, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--flux-2-turbo--s3333.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5

### Gemini 2.5 Flash Image

[Model profile](https://img.ly/ai-benchmarks/models/gemini-25-flash-image.md)

- sample 1: 1024×1024, 7.9s, $0.039, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gemini-25-flash-image--n0.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 2: 1024×1024, 7.7s, $0.039, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gemini-25-flash-image--n1.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 3.0 / 5
- sample 3: 1024×1024, 7.9s, $0.039, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gemini-25-flash-image--n2.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5

### GPT Image 1.5

[Model profile](https://img.ly/ai-benchmarks/models/gpt-image-1-5.md)

- sample 1: 1024×1024, 38.0s, $0.120, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gpt-image-1-5--n0.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 3.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 2: 1024×1024, 36.5s, $0.120, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gpt-image-1-5--n1.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 3.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 3: 1024×1024, 38.4s, $0.120, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--gpt-image-1-5--n2.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 3.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 3.0 / 5

### Ideogram 3.0

[Model profile](https://img.ly/ai-benchmarks/models/ideogram-v3.md)

- seed 1111: 1024×1024, 17.5s, $0.060, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--ideogram-v3--s1111.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 3.6 / 5
  - Resolution: 3.0 / 5
- seed 2222: 1024×1024, 17.9s, $0.060, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--ideogram-v3--s2222.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 3.0 / 5
- seed 3333: 1024×1024, 17.3s, $0.060, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--ideogram-v3--s3333.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5

### Luma Photon

[Model profile](https://img.ly/ai-benchmarks/models/luma-photon.md)

- seed 1111: 1536×1536, 17.5s, $0.019, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--luma-photon--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 4.0 / 5
- seed 2222: 1536×1536, 14.6s, $0.019, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--luma-photon--s2222.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 4.0 / 5
- seed 3333: 1536×1536, 20.7s, $0.019, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--luma-photon--s3333.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 3.6 / 5
  - Resolution: 4.0 / 5

### Nano Banana 2

[Model profile](https://img.ly/ai-benchmarks/models/nano-banana-2.md)

- sample 1: 1408×768, 11.6s, $0.100, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2--n0.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 2: 1408×768, 12.5s, $0.100, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2--n1.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 3: 1408×768, 14.8s, $0.100, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2--n2.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 3.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5

### Nano Banana 2 Lite

[Model profile](https://img.ly/ai-benchmarks/models/nano-banana-2-lite.md)

- sample 1: 1408×768, 3.7s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2-lite--n0.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 2: 1376×768, 3.9s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2-lite--n1.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 3: 1408×768, 4.2s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-2-lite--n2.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5

### Nano Banana Pro

[Model profile](https://img.ly/ai-benchmarks/models/nano-banana-pro.md)

- sample 1: 1024×1024, 22.6s, $0.360, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-pro--n0.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 1.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 3.0 / 5
- sample 2: 1024×1024, 27.0s, $0.360, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-pro--n1.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 1.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- sample 3: 1024×1024, 31.7s, $0.360, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--nano-banana-pro--n2.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 1.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5

### Qwen-Image

[Model profile](https://img.ly/ai-benchmarks/models/qwen-image.md)

- seed 1111: 1024×768, 8.6s, $0.030, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--qwen-image--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 2222: 1024×768, 5.5s, $0.030, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--qwen-image--s2222.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 2.0 / 5
- seed 3333: 1024×768, 8.6s, $0.030, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--qwen-image--s3333.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 2.0 / 5

### Recraft V3

[Model profile](https://img.ly/ai-benchmarks/models/recraft-v3.md)

- seed 1111: 1024×1024, 10.4s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--recraft-v3--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 3.6 / 5
  - Resolution: 3.0 / 5
- seed 2222: 1024×1024, 7.2s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--recraft-v3--s2222.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 3.0 / 5
- seed 3333: 1024×1024, 8.7s, $0.040, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--recraft-v3--s3333.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 4.0 / 5
  - Latency: 3.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 3.0 / 5

### Seedream 4.5

[Model profile](https://img.ly/ai-benchmarks/models/seedream-4-5.md)

- seed 1111: 2048×2048, 20.4s, $0.048, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-4-5--s1111.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 5.0 / 5
- seed 2222: 2048×2048, 23.9s, $0.048, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-4-5--s2222.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 5.0 / 5
- seed 3333: 2048×2048, 12.0s, $0.048, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-4-5--s3333.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 2.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 5.0 / 5

### Seedream 5.0 Lite

[Model profile](https://img.ly/ai-benchmarks/models/seedream-5-lite.md)

- seed 1111: 2848×1600, 51.5s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-5-lite--s1111.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 5.0 / 5
- seed 2222: 2048×2048, 37.3s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-5-lite--s2222.webp)
  - Color Accuracy: 4.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 5.0 / 5
- seed 3333: 2848×1600, 44.0s, $0.020, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--seedream-5-lite--s3333.webp)
  - Color Accuracy: 3.0 / 5
  - Cost: 4.0 / 5
  - Latency: 1.0 / 5
  - Prompt Adherence: 5.0 / 5
  - Resolution: 5.0 / 5

### Stable Diffusion 1.5

[Model profile](https://img.ly/ai-benchmarks/models/stable-diffusion-1-5.md)

- seed 1111: 512×512, 2.5s, $0.010, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--stable-diffusion-1-5--s1111.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 5.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 1.4 / 5
  - Resolution: 1.0 / 5
- seed 2222: 512×512, 2.1s, $0.010, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--stable-diffusion-1-5--s2222.webp)
  - Color Accuracy: 0.0 / 5
  - Cost: 5.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 0.0 / 5
  - Resolution: 1.0 / 5
- seed 3333: 512×512, 2.1s, $0.010, [image](https://storage.googleapis.com/imgly-ai-benchmarks/j01-json-scene--stable-diffusion-1-5--s3333.webp)
  - Color Accuracy: 2.0 / 5
  - Cost: 5.0 / 5
  - Latency: 4.0 / 5
  - Prompt Adherence: 4.3 / 5
  - Resolution: 1.0 / 5

Result images: [https://img.ly/ai-benchmarks/prompts/j01-json-scene/](https://img.ly/ai-benchmarks/prompts/j01-json-scene/) (interactive grid with lightbox).

## Frequently asked questions

**Why am I seeing three samples per benchmark?**

Image generation is stochastic: the same prompt produces different images on every run, so a single sample measures luck, not ability. Every model runs each benchmark three times with fixed seeds (1111, 2222, 3333), or three unseeded samples where the API accepts no seed. Scores average all three samples, and the consistency benchmarks measure the variation itself.

**Do all AI models get the identical prompt?**

Yes. Every model receives the same prompt text with default parameters and no per-model tuning, so differences in output reflect the model, not prompt engineering. The suite is versioned and prompts are append-only, which keeps historical scores comparable.

**Which AI image model is best at structured spec adherence?**

The results on this page are scored on this exact prompt, so the strongest model for it is easy to spot; this benchmark sits in the Structured Spec Adherence category. Scores are per model version and reflect this benchmark only. For a ranking across every benchmark see the model rankings, and to weight the numbers by a specific job see the use-case pages.

**How are the benchmark scores calculated?**

Each image is scored 0 to 5 per criterion. Measured criteria (resolution, latency, cost, transparency, color accuracy) are computed automatically; quality criteria (text accuracy, prompt adherence, composition) are judged by an automated Claude vision tier against each prompt’s checklist. Page scores are unweighted means over all of a model’s runs in that scope. Blind expert-panel review has not run yet; the dataset is pilot-0.

**Can I use AI-generated images like these in production?**

Model output is a starting point, not a finished asset. Production work usually needs background removal, exact brand colors or editable text, none of which generation guarantees on every run. The IMG.LY AI Editor gives your users those controls to refine any model’s output to production quality.

**How often is the benchmark updated?**

The suite re-runs on notable model releases so the rankings stay current. The results shown are the pilot-0 dataset, scored by measured criteria plus an automated Claude vision tier, with blind expert review planned. Prompts are versioned and append-only, so scores stay comparable across runs.


---

## More Resources

- **[IMG.LY Website](https://img.ly/index.md)** - Creative editing SDKs for photo, video, and design
- **[Documentation](https://img.ly/docs/cesdk/)** - CE.SDK developer documentation
- **[Contact Sales](https://img.ly/forms/contact-sales.md)** - Get a custom quote. A public JSON API accepts the request directly, no account or key needed. Ask your user for consent and their details first.