A comic strip is a compound stress test in one asset: exact speech-bubble text, character consistency across panels, and multi-panel layout. Few models hold all three together.
Three-panel comic strip (3x1); flat colors with clean black outlines; panel 2 has a speech bubble reading "I forgot my keys." exactly; bubble tail points to the speaking character; consistent character design across panels; white gutters; legible lettering.
Suite pilot-0 · identical prompt and default parameters for every model ·
seeds 1111, 2222, 3333 (or repeated samples for seedless APIs)
Results
Click any image to enlarge. Checkerboard backing shows true
transparency; a rendered checkerboard pattern inside the image is a
model faking it.
FLUX.2
FLUX.2 [pro]
FLUX.2 [dev] Turbo
Gemini 2.5 Flash Image
GPT Image 1.5
Ideogram 3.0
Luma Photon
Nano Banana 2
Nano Banana 2 Lite
Nano Banana Pro
Qwen-Image
Recraft V3
Seedream 4.5
Seedream 5.0 Lite
Stable Diffusion 1.5
What the models actually wrote
Expected text vs the rendered result, character by character.
Transcribed by a human reviewer in the pilot; the OCR tier automates
this at full-suite scale.
Expected
I forgot my keys.
Rendered
FLUX.2 · seed 1111 I F O R G O T M Y K E Y S . 100% match · 5/5
The rendered text matches the required 'I forgot my keys.' exactly, including the period, though it is broken across two lines. It is legible and correctly placed in panel 2.
FLUX.2 · seed 2222 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble text matches the required 'I forgot my keys.' exactly, including punctuation. The lettering is legible and correctly placed in panel 2.
FLUX.2 · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble in panel 2 reads exactly 'I forgot my keys.' matching the required text with correct spelling, punctuation, and clear legibility.
FLUX.2 [pro] · seed 1111 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble text reads 'I forgot my keys.' exactly as required, matching spelling and punctuation. It is broken across two lines but is fully legible.
FLUX.2 [pro] · seed 2222 I f o r g o t m y k e y s . 100% match · 5/5
The text reads exactly "I forgot my keys." as required, broken across two lines within the bubble. It is clearly legible and correctly punctuated.
FLUX.2 [pro] · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The text is rendered exactly as required, "I forgot my keys.", including the period. Lettering is clean and legible.
FLUX.2 [dev] Turbo · seed 1111 I F O R G O T M Y K E Y S . 100% match · 5/5
The rendered text reads 'I FORGOT MY KEYS.' in uppercase across two lines, which matches the required text exactly aside from case. The lettering is clear and legible.
FLUX.2 [dev] Turbo · seed 2222 I f o r g o t m y k e y s . 100% match · 5/5
The text is rendered exactly as required, "I forgot my keys.", and is clearly legible in the panel 2 speech bubble with a tail pointing toward the character.
FLUX.2 [dev] Turbo · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The rendered text matches the required 'I forgot my keys.' exactly, including punctuation. The lettering is legible and cleanly placed within the speech bubble.
Gemini 2.5 Flash Image · sample 1 I F O R G O T M Y K E Y S . 100% match · 5/5
The required text 'I forgot my keys.' is rendered legibly in panel 2 as 'I FORGOT MY KEYS.' (all caps, split across two lines), matching exactly aside from capitalization. Additional bonus text 'GUESS I'M WAITING.' appears in panel 3.
Gemini 2.5 Flash Image · sample 2 I F O R G O T M Y K E Y S . 100% match · 5/5
The bubble displays "I FORGOT MY KEYS." in all caps across two lines. This matches the required text in wording and includes the period, though it is uppercase rather than the sentence-case "I forgot my keys." and one letter appears slightly malformed.
Gemini 2.5 Flash Image · sample 3 I F O R G O T M Y K E Y S . 100% match · 5/5
The text is legible but rendered in all uppercase across two lines as "I FORGOT MY KEYS." rather than the exact requested casing "I forgot my keys." The wording is correct but the capitalization does not match exactly.
GPT Image 1.5 · sample 1 I f o r g o t m y k e y s . 100% match · 5/5
The text is rendered exactly as required, "I forgot my keys.", with correct spelling and punctuation. It is clearly legible inside the speech bubble in panel 2.
GPT Image 1.5 · sample 2 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble text in panel 2 reads "I forgot my keys." exactly as required, broken across two lines. The lettering is clear and legible. The first panel also has partial sound effect text ("LICK! CLICK!") which is cut off but not part of the required text.
GPT Image 1.5 · sample 3 I f o r g o t m y k e y s . 100% match · 5/5
The text renders exactly as required, 'I forgot my keys.', with correct spelling and punctuation. It is clearly legible in the panel 2 speech bubble.
Ideogram 3.0 · seed 1111 I F O R G O T M Y K E Y S 94% match · 4/5
The visible text reads 'I FORGOT MY KEYS' in uppercase with a line break, but it is missing the required period at the end. It also appears in the bottom panel rather than panel 2 as specified.
Ideogram 3.0 · seed 2222 I F O R G O T M Y K E Y S 94% match · 4/5
The rendered text reads "I FORGOT MY KEYS" across two lines, missing the required period at the end. It is legible and correctly spelled otherwise, but does not exactly match "I forgot my keys."
Ideogram 3.0 · seed 3333 I F O R G O T M Y K E Y S . 100% match · 5/5
The speech bubble legibly renders "I FORGOT MY KEYS." which matches the required text exactly aside from case. Lettering is clear and readable.
Luma Photon · seed 1111 I f o r g o t m y k e y s . 100% match · 5/5
The top-right panel legibly renders the exact required text "I forgot my keys." Other panels contain garbled or misspelled variants (e.g. "I forgot my keys. Pard.", "I'll reeicierd hove forgot my keys.", "I forgot lu mvu s"), but at least one instance is correct.
Luma Photon · seed 2222 I M Y K E Y S / I F O R G O T M Y K E Y S / F O R O U T Y 94% match · 4/5
The required exact text "I forgot my keys." appears in one panel as "I FORGOT MY KEYS" but lacks the period and is all caps. Other panels show broken/garbled text like "I MY KEYS" and "FOROUTY".
Luma Photon · seed 3333 f o r g o t m y k e y s . I f o r g o t m y K e y s . t h e w a l l . i A x R N O c s u 100% match · 5/5
Panel 2 correctly renders "I forgot my keys." exactly and legibly. The other panels contain garbled or incorrect text like "forgot my keys.", "the wall.", and nonsensical characters, but the required exact string does appear in panel 2.
Nano Banana 2 · sample 1 I F O R G O T M Y K E Y S . 100% match · 5/5
The rendered text matches the required "I forgot my keys." exactly, including the period. It appears in uppercase lettering which is standard comic style and is fully legible.
Nano Banana 2 · sample 2 I f o r g o t m y k e y s . 100% match · 5/5
The text renders exactly as required, "I forgot my keys.", broken across two lines within the speech bubble. Spelling and punctuation are correct and the lettering is clearly legible.
Nano Banana 2 · sample 3 I f o r g o t m y k e y s . 100% match · 5/5
The text matches the required 'I forgot my keys.' exactly, split across two lines within the bubble. It is legible and correctly punctuated.
Nano Banana 2 Lite · sample 1 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble in panel 2 renders the required text exactly as "I forgot my keys." with correct spelling and punctuation. The lettering is clear and legible.
Nano Banana 2 Lite · sample 2 I f o r g o t m y k e y s . 100% match · 5/5
The required text 'I forgot my keys.' is rendered exactly and legibly in panel 2's speech bubble. Other bubbles contain additional dialogue but the required phrase matches perfectly.
Nano Banana 2 Lite · sample 3 O k a y , r e a d y t o g o f o r o u r h i k e ! W a i t a m i n u t e . . . I f o r g o t m y k e y s . U g h , s e r i o u s l y ? ! T h e y a r e n ' t h e r e ! T y p i c a l ! N e e d m e t o l o c k u p ? 100% match · 5/5
The required text "I forgot my keys." is rendered exactly and legibly in panel 2's second speech bubble. All other lettering across the panels is also clear and readable.
Nano Banana Pro · sample 1 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble legibly reads 'I forgot my keys.' with a line break, matching the required text exactly including punctuation. The lettering is clean and readable.
Nano Banana Pro · sample 2 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble in panel 2 reads exactly 'I forgot my keys.' with correct spelling and punctuation, broken across two lines. It is clearly legible.
Nano Banana Pro · sample 3 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble in panel 2 reads "I forgot my keys." exactly as required, split across two lines. The lettering is clean and legible with correct spelling and punctuation.
Qwen-Image · seed 1111 I f o r g o t m y k e y s . 100% match · 5/5
The rendered text reads exactly "I forgot my keys." and is clearly legible, matching the required text including the period. The only issue is placement in the first panel rather than the second.
Qwen-Image · seed 2222 I f o r g o t m y k e y s . ' 100% match · 5/5
The text is legible but includes a stray apostrophe/quote mark after the period, rendering it as "my keys.'" rather than the exact "I forgot my keys." required. The line break splits the phrase across two lines.
Qwen-Image · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The rendered text reads "I forgot my keys." broken across two lines, matching the required text exactly including punctuation. Lettering is clean and legible.
Recraft V3 · seed 1111 I F O R G O T M Y K E Y S . 100% match · 5/5
The speech bubble legibly and correctly displays "I FORGOT MY KEYS." in bold uppercase lettering, matching the required text exactly including the period.
Recraft V3 · seed 2222 I F o r g o t m y k e y s . 100% match · 5/5
The visible text reads "I Forgot my keys." with a capital F, whereas the required text is "I forgot my keys." with lowercase f. It is legible but not an exact match due to capitalization.
Recraft V3 · seed 3333 I f o r g o t m y k e y s 94% match · 4/5
The bubble reads "I forgot my keys" across two lines, which matches the required words but omits the terminal period specified as "I forgot my keys.". Otherwise the text is legible and correctly spelled.
Seedream 4.5 · seed 1111 I f o r g o t m y k e y s . 100% match · 5/5
The rendered text reads 'I forgot my keys.' broken across two lines, matching the required text exactly including punctuation. Lettering is clear and legible.
Seedream 4.5 · seed 2222 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble legibly reads 'I forgot my keys.' exactly as required, with a line break between 'forgot' and 'my' that does not affect correctness. Spelling and punctuation match the prompt.
Seedream 4.5 · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble text matches the required 'I forgot my keys.' exactly, including the period. It is broken across two lines ('I forgot' / 'my keys.') but remains fully legible and correctly spelled.
Seedream 5.0 Lite · seed 1111 I f o r g o t m y k e y s . 100% match · 5/5
The text matches the required string exactly, including the period. It is clearly legible in the speech bubble in panel 2.
Seedream 5.0 Lite · seed 2222 I f o r g o t m y k e y s . 100% match · 5/5
The text matches the required string exactly, including punctuation. It is clearly legible and well-lettered inside the speech bubble.
Seedream 5.0 Lite · seed 3333 I f o r g o t m y k e y s . 100% match · 5/5
The speech bubble in panel 2 renders the required text exactly as "I forgot my keys." with correct spelling and punctuation. The lettering is clean and legible.
Stable Diffusion 1.5 · seed 1111
no legible text
0% match · 0/5
No legible text matching "I forgot my keys." appears anywhere; only a few illegible scribbles are visible, so the required text is entirely missing.
Stable Diffusion 1.5 · seed 2222
no legible text
0% match · 0/5
The speech bubbles contain scribbled, illegible marks that do not spell any readable words. The required text "I forgot my keys." is not present.
Stable Diffusion 1.5 · seed 3333
no legible text
0% match · 0/5
None of the speech bubbles contain the required phrase "I forgot my keys." The lettering is illegible scribbles and nonsense characters, so no readable text matches the requirement.
Run-to-run consistency
Same model, same prompt, different seed or sample. Click a card to
flip between the two generations: the less it moves, the safer the
model is for templated, variable-data production.
Scores
Mean per model across this prompt's runs. Auto-measured criteria plus an
interim Claude-VLM tier for prompt adherence and composition; any
criterion still awaiting review is labeled pending, never zeroed.
We re-run the identical 37-prompt suite on every model release. Get the
scores, and what changed in the rankings, in your inbox.
You're on the list.
We'll email you when new benchmark results ship.
Frequently asked questions
Image generation is stochastic: the same prompt produces different images on every run, so a single sample measures luck, not ability. Every model runs each benchmark three times with fixed seeds (1111, 2222, 3333), or three unseeded samples where the API accepts no seed. Scores average all three samples, and the consistency benchmarks measure the variation itself.
Yes. Every model receives the same prompt text with default parameters and no per-model tuning, so differences in output reflect the model, not prompt engineering. The suite is versioned and prompts are append-only, which keeps historical scores comparable.
The results on this page are scored on this exact prompt, so the strongest model for it is easy to spot; this benchmark sits in the Print-Production Graphics category. Scores are per model version and reflect this benchmark only. For a ranking across every benchmark see the model rankings, and to weight the numbers by a specific job see the use-case pages.
Each image is scored 0 to 5 per criterion. Measured criteria (resolution, latency, cost, transparency, color accuracy) are computed automatically; quality criteria (text accuracy, prompt adherence, composition) are judged by an automated Claude vision tier against each prompt’s checklist. Page scores are unweighted means over all of a model’s runs in that scope. Blind expert-panel review has not run yet; the dataset is pilot-0.
Model output is a starting point, not a finished asset. Production work usually needs background removal, exact brand colors or editable text, none of which generation guarantees on every run. The IMG.LY AI Editor gives your users those controls to refine any model’s output to production quality.
The suite re-runs on notable model releases so the rankings stay current. The results shown are the pilot-0 dataset, scored by measured criteria plus an automated Claude vision tier, with blind expert review planned. Prompts are versioned and append-only, so scores stay comparable across runs.
