Product-page mockups are the closest UI job to IMG.LY's commerce customers. This tests layout hierarchy and legible interface text (price, controls, the Add to Cart button) in one screen.
E-commerce product page: image gallery thumbnails, price, size selector, quantity stepper, "Add to Cart" button, star ratings; clear hierarchy.
Suite pilot-0 · identical prompt and default parameters for every model ·
seeds 1111, 2222, 3333 (or repeated samples for seedless APIs)
Results
Click any image to enlarge. Checkerboard backing shows true
transparency; a rendered checkerboard pattern inside the image is a
model faking it.
FLUX.2
FLUX.2 [pro]
FLUX.2 [dev] Turbo
Gemini 2.5 Flash Image
GPT Image 1.5
Ideogram 3.0
Luma Photon
Nano Banana 2
Nano Banana 2 Lite
Nano Banana Pro
Qwen-Image
Recraft V3
Seedream 4.5
Seedream 5.0 Lite
Stable Diffusion 1.5
What the models actually wrote
Expected text vs the rendered result, character by character.
Transcribed by a human reviewer in the pilot; the OCR tier automates
this at full-suite scale.
Expected
Add to Cart
Rendered
FLUX.2 · seed 1111 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered legibly and correctly on the green button. Other text on the page is heavily garbled ('Sun lere Galers', 'Bide Shrains'), but the required phrase is accurate.
FLUX.2 · seed 2222 A d d t o C a r t 100% match · 5/5
The 'Add to Cart' text is rendered clearly and correctly on the orange button. Surrounding text such as the title 'Realaft Dew arrevas Sizes' and other labels are garbled, but the required text is legible and accurate.
FLUX.2 · seed 3333 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the orange button. Other text like the header 'EVEINST AFSA YOUR GALERY' is gibberish, but that was not the required text.
FLUX.2 [pro] · seed 1111 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the black button. It matches the required spelling with no errors.
FLUX.2 [pro] · seed 2222 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue button. No misspellings or broken line breaks are present.
FLUX.2 [pro] · seed 3333 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue button. The surrounding product title contains typos ('Prodece Tire Lowld Size Streeet Portfiert') but those were not required.
FLUX.2 [dev] Turbo · seed 1111 A d d t o C a r t 100% match · 5/5
The 'Add to Cart' text is rendered exactly and legibly on the orange button. Other surrounding text contains misspellings (e.g., 'Galery', 'Size Quatty', 'Faunt'), but the required phrase is correct.
FLUX.2 [dev] Turbo · seed 2222 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered clearly and legibly on the orange button, spelled correctly. Surrounding text is garbled (e.g., 'Qurantity', 'Sallerker', '$15,9,99') but the required phrase is accurate.
FLUX.2 [dev] Turbo · seed 3333 A d d t o C a r t 100% match · 5/5
The required text "Add to Cart" is rendered exactly and legibly on the orange button. Surrounding text is garbled (e.g. "Galurdl, Size", "Surpaday, stes"), but the required phrase is correct.
Gemini 2.5 Flash Image · sample 1 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered legibly and exactly as specified on the teal button. No misspellings or broken line breaks are present.
Gemini 2.5 Flash Image · sample 2 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is rendered legibly as 'ADD TO CART' in uppercase on the button, which matches the intent though in different casing. Other descriptive text on the page is garbled and nonsensical, but the required button text is clear.
Gemini 2.5 Flash Image · sample 3 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is rendered legibly in all caps as 'ADD TO CART' on the blue button. Aside from the casing difference, it matches the required text exactly and is fully legible.
GPT Image 1.5 · sample 1 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue button, matching the requirement with correct spelling and no broken line breaks.
GPT Image 1.5 · sample 2 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly in white on the blue button. Other supporting text like '$59.99', 'Select Size:', 'Quantity:', and '(120 Reviews)' is also clean and correctly spelled.
GPT Image 1.5 · sample 3 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue button. No spelling errors or broken line breaks.
Ideogram 3.0 · seed 1111 H O O M B A D W e d c k J h a d u m R c o i t y S O U r S U N D O R G e r o B e t i c e H a n t o $ 7 9 . 9 9 S m a l l M e d i u m L a r g e 1 O l y p a t e t o e Q u n t i y A d d t o c a r t T o u t i c h e s s o l e o e e n p t o p u s e o u h 100% match · 5/5
The required text 'Add to Cart' appears as 'Add to cart' which is legible and close, though the casing differs slightly. The rest of the page text is largely nonsensical gibberish.
Ideogram 3.0 · seed 2222 A d d t o c a r t 100% match · 5/5
The required text renders as 'Add to cart' rather than the exact 'Add to Cart' (lowercase 'c' in cart), so capitalization differs but it is clearly legible.
Ideogram 3.0 · seed 3333 A d d t o c a r t 100% match · 5/5
The button reads 'Add to cart' rather than the exact required 'Add to Cart' (lowercase 'c' in cart). Otherwise it is legible and correctly placed, but capitalization differs from the required text. Surrounding text like 'Feontinet jun taget' and 'Sovemherat er an ad' is garbled gibberish.
Luma Photon · seed 1111 A d d t o C a r t 100% match · 5/5
The phrase "Add to Cart" is rendered legibly and correctly in at least three places, including the main black button, the top-right header, and the footer. Surrounding UI text is largely garbled gibberish, but the required text itself is accurate.
Luma Photon · seed 2222 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered clearly and correctly on the black button. Surrounding labels like 'Slector Price', 'Quantity', and 'Rctork s Reviews' are garbled, but the required phrase itself is legible and accurate.
Luma Photon · seed 3333 A d d C a r t / A l a t o C a s t 73% match · 3/5
The required text 'Add to Cart' is not legibly rendered; the button and header show garbled approximations like 'Add Cart' and 'Ala to Cast'. The correct exact spelling and phrasing are not present.
Nano Banana 2 · sample 1 A d d t o C a r t 100% match · 5/5
The required text "Add to Cart" is rendered exactly and legibly on the dark blue primary button. No misspellings or broken line breaks are present.
Nano Banana 2 · sample 2 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is legibly rendered as 'ADD TO CART' in the primary button. It is fully legible and matches the required phrase apart from capitalization.
Nano Banana 2 · sample 3 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue primary button, matching the prompt requirement with no spelling errors or broken breaks.
Nano Banana 2 Lite · sample 1 A d d t o B a g 73% match · 3/5
The required text "Add to Cart" is not present; the main call-to-action button instead reads "Add to Bag." The text is legible but does not match the required string.
Nano Banana 2 Lite · sample 2 A D D T O C A R T 100% match · 5/5
The button text renders as 'ADD TO CART' in all caps rather than the exact 'Add to Cart' casing, but it is fully legible and correctly spelled. The word 'Add to Wishlist' also appears in proper casing nearby.
Nano Banana 2 Lite · sample 3 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is rendered legibly, though in all caps as 'ADD TO CART'. The content matches exactly aside from capitalization styling.
Nano Banana Pro · sample 1 A D D T O C A R T 100% match · 5/5
The button renders as 'ADD TO CART' in all caps, which matches the required text 'Add to Cart' in content and is fully legible. Casing differs but the wording is correct and clear.
Nano Banana Pro · sample 2 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the blue button, matching the requirement precisely with correct spelling and capitalization.
Nano Banana Pro · sample 3 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered legibly and exactly on the blue button, matching the requirement precisely with no misspellings.
Qwen-Image · seed 1111 A d d t o C a r t 100% match · 5/5
The required text "Add to Cart" is rendered clearly and legibly on the black button, matching exactly. Other UI text (e.g., "Quepper", "Commerce Gallery Commerce") is nonsensical but not part of the required text.
Qwen-Image · seed 2222 E o m m e r c e G a l l e r y G r 0 4 : t o t S i z e O u s t s f 9 e x ↓ A d d t o C a r t S t o p e r 100% match · 5/5
The required 'Add to Cart' text renders correctly and legibly on the button. However, surrounding labels are heavily corrupted ('Eommerce', 'Gr04:tot', 'sf9e', 'Stoper'), degrading overall text quality.
Qwen-Image · seed 3333 A d d t o C a r t 100% match · 5/5
The "Add to Cart" button text is rendered clearly and correctly. Most other text in the interface (labels, headings, navigation) is garbled or nonsensical, but the required text matches exactly.
Recraft V3 · seed 1111 A D d t o C a r t 100% match · 5/5
The button text reads "ADd to Cart" with inconsistent capitalization (a stray capital D), so it does not exactly match the required "Add to Cart". It is legible but misspelled in its casing.
Recraft V3 · seed 2222 P r i c e S i z e S e l e c t o r D r o p d u n Q u n t l i y S l e p e r A D D 8 C A R T 82% match · 3/5
The required text 'Add to Cart' is not rendered correctly; the button reads 'ADD 8 CART'. Other labels are also garbled, such as 'Dropdun' and 'Quntliy Sleper'.
Recraft V3 · seed 3333 A d d t o C a r t 100% match · 5/5
The required text "Add to Cart" is rendered clearly and correctly. Other text on the page is garbled and illegible (e.g., 'Sce', 'Broo Down', 'Stegory', 'Price'), but the specific required phrase matches exactly.
Seedream 4.5 · seed 1111 A d d t o C a r t 100% match · 5/5
The required text "Add to Cart" is rendered exactly and legibly in white on the orange button. Spelling and formatting are correct.
Seedream 4.5 · seed 2222 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the orange button. Other text like the misspelled 'Steepter' exists but does not affect the required phrase.
Seedream 4.5 · seed 3333 A d d t o C a r t 100% match · 5/5
The required text 'Add to Cart' is rendered exactly and legibly on the orange button, matching the requirement precisely with correct spelling and no broken line breaks.
Seedream 5.0 Lite · seed 1111 A D D T O C A R T 100% match · 5/5
The required text is rendered in the button but in all caps as 'ADD TO CART' rather than 'Add to Cart'. It is fully legible with correct spelling, differing only in capitalization.
Seedream 5.0 Lite · seed 2222 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is rendered legibly in bold white uppercase letters on a black button. It matches the required text exactly aside from capitalization, which is a stylistic choice.
Seedream 5.0 Lite · seed 3333 A D D T O C A R T 100% match · 5/5
The required text 'Add to Cart' is rendered legibly in all caps as 'ADD TO CART', matching the phrase in content despite the capitalization styling.
Stable Diffusion 1.5 · seed 1111
no legible text
0% match · 0/5
The required text "Add to Cart" is nowhere legible in the image. All visible text is scrambled and unreadable, so no valid text was rendered.
Stable Diffusion 1.5 · seed 2222
no legible text
0% match · 0/5
The required text "Add to Cart" is not legibly rendered anywhere. All visible text is illegible, garbled placeholder-like characters.
Stable Diffusion 1.5 · seed 3333
no legible text
0% match · 0/5
The required text "Add to Cart" is not legible anywhere in the image; all text elements appear as scrambled, unreadable marks.
Run-to-run consistency
Same model, same prompt, different seed or sample. Click a card to
flip between the two generations: the less it moves, the safer the
model is for templated, variable-data production.
Scores
Mean per model across this prompt's runs. Auto-measured criteria plus an
interim Claude-VLM tier for prompt adherence and composition; any
criterion still awaiting review is labeled pending, never zeroed.
We re-run the identical 37-prompt suite on every model release. Get the
scores, and what changed in the rankings, in your inbox.
You're on the list.
We'll email you when new benchmark results ship.
Frequently asked questions
Image generation is stochastic: the same prompt produces different images on every run, so a single sample measures luck, not ability. Every model runs each benchmark three times with fixed seeds (1111, 2222, 3333), or three unseeded samples where the API accepts no seed. Scores average all three samples, and the consistency benchmarks measure the variation itself.
Yes. Every model receives the same prompt text with default parameters and no per-model tuning, so differences in output reflect the model, not prompt engineering. The suite is versioned and prompts are append-only, which keeps historical scores comparable.
The results on this page are scored on this exact prompt, so the strongest model for it is easy to spot; this benchmark sits in the UI & Design Mockups category. Scores are per model version and reflect this benchmark only. For a ranking across every benchmark see the model rankings, and to weight the numbers by a specific job see the use-case pages.
Each image is scored 0 to 5 per criterion. Measured criteria (resolution, latency, cost, transparency, color accuracy) are computed automatically; quality criteria (text accuracy, prompt adherence, composition) are judged by an automated Claude vision tier against each prompt’s checklist. Page scores are unweighted means over all of a model’s runs in that scope. Blind expert-panel review has not run yet; the dataset is pilot-0.
Model output is a starting point, not a finished asset. Production work usually needs background removal, exact brand colors or editable text, none of which generation guarantees on every run. The IMG.LY AI Editor gives your users those controls to refine any model’s output to production quality.
The suite re-runs on notable model releases so the rankings stay current. The results shown are the pilot-0 dataset, scored by measured criteria plus an automated Claude vision tier, with blind expert review planned. Prompts are versioned and append-only, so scores stay comparable across runs.
