Non-Latin scripts are where most models fall apart, and IMG.LY has international print and social customers. One CJK sign is enough to separate models that handle multi-byte glyphs from those that mangle them (full multilingual coverage is a planned deep-dive).
Suite pilot-0 · identical prompt and default parameters for every model ·
seeds 1111, 2222, 3333 (or repeated samples for seedless APIs)
Results
Click any image to enlarge. Checkerboard backing shows true
transparency; a rendered checkerboard pattern inside the image is a
model faking it.
FLUX.2
FLUX.2 [pro]
FLUX.2 [dev] Turbo
Gemini 2.5 Flash Image
GPT Image 1.5
Ideogram 3.0
Luma Photon
Nano Banana 2
Nano Banana 2 Lite
Nano Banana Pro
Qwen-Image
Recraft V3
Seedream 4.5
Seedream 5.0 Lite
Stable Diffusion 1.5
What the models actually wrote
Expected text vs the rendered result, character by character.
Transcribed by a human reviewer in the pilot; the OCR tier automates
this at full-suite scale.
Expected
東京ラーメン
Rendered
FLUX.2 · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required string 東京ラーメン exactly, with all six characters correct and clearly legible. There are no misspellings or broken glyphs.
FLUX.2 · seed 2222 東 京 ラ ー メ ン 100% match · 5/5
The rendered text exactly matches the required '東京ラーメン' with correct, well-formed characters and no invented glyphs. The neon is bright and highly legible against the dark background.
FLUX.2 · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The text renders exactly as required with all six characters (東京ラーメン) correct and clearly legible in glowing red neon. No spelling errors or malformed glyphs are present.
FLUX.2 [pro] · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The rendered neon text matches the required string 東京ラーメン exactly, with correct kanji and katakana. The characters are sharp, well-formed, and highly legible against the dark storefront.
FLUX.2 [pro] · seed 2222 東 京 ラ ー メ ン 100% match · 5/5
The neon sign renders exactly '東京ラーメン' with correct, legible glyphs. A secondary noren curtain also reads 'ラーメン', consistent with the required text.
FLUX.2 [pro] · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The main neon sign renders exactly 東京ラーメン with accurate, legible glyphs. Background signage shows other partial characters but the required text is correct and clearly readable.
FLUX.2 [dev] Turbo · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required '東京ラーメン' exactly, with all characters correct, legible, and glowing in red neon. No misspellings or broken glyphs are present.
FLUX.2 [dev] Turbo · seed 2222 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly, with all six characters correctly formed and clearly legible in the neon sign. There are no spelling errors or broken glyphs.
FLUX.2 [dev] Turbo · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The rendered text exactly matches the required 東京ラーメン with correct characters and no invented glyphs. The neon lettering is clear and legible against the dark background.
Gemini 2.5 Flash Image · sample 1 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly. The characters are crisp, correctly formed, and legible in the neon sign. No spelling errors or broken glyphs are present.
Gemini 2.5 Flash Image · sample 2 東 京 ラ ー メ ン 100% match · 5/5
The rendered text exactly matches the required 東京ラーメン in glowing blue neon. The kanji and katakana are correct and legible with no spelling errors or broken characters.
Gemini 2.5 Flash Image · sample 3 東 京 ラ ー メ ン 100% match · 5/5
The sign renders exactly the required text 東京ラーメン with the kanji in blue and the katakana ラーメン in red neon. All characters are correct, legible, and properly formed.
GPT Image 1.5 · sample 1 東 京 ラ ー メ ン 100% match · 5/5
The main neon sign renders exactly 東京ラーメン with correct, legible characters. The noren curtain also shows ラーメン, consistent with the theme. No misspellings or invented glyphs in the primary sign.
GPT Image 1.5 · sample 2 東 京 ラ ー メ ン 100% match · 5/5
The neon sign renders the required text 東京ラーメン exactly and legibly. The lantern also reads ラーメン and the noren curtain shows ラーメン, all correct. No misspellings or broken glyphs.
GPT Image 1.5 · sample 3 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly, with correct kanji and katakana. The characters are sharp, well-lit, and fully legible in neon form.
Ideogram 3.0 · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly, with correct characters and good legibility. The neon glow does not distort the glyphs into invented forms.
Ideogram 3.0 · seed 2222 東 京 ラ ー メ ン A t a s t e o f T o k y o 100% match · 5/5
The main neon text correctly renders 東京ラーメン with accurate, legible kanji and katakana. An additional tagline 'A taste of Tokyo' appears at the bottom, which is legible and correctly spelled.
Ideogram 3.0 · seed 3333 東 京 ラ ー メ ン O P E N 83% match · 3/5
The main neon sign renders 東京ラーメン exactly and legibly across two lines, matching the required text. An additional 'OPEN' sign appears below, which was not required but does not conflict. Smaller poster text on the door is illegible gibberish but is separate from the required text.
Luma Photon · seed 1111 ヘ カ 木 杰 丈 0% match · 0/5
The required text 東京ラーメン is not rendered; the sign instead shows garbled, invented characters that only loosely resemble strokes of real glyphs. None of the intended kanji or katakana are correct or legible.
Luma Photon · seed 2222 の 展 T E H I O J U A L I A N 0% match · 0/5
The required text was 東京ラーメン, but the sign displays a jumble of distorted, fabricated characters plus meaningless Latin text like 'TEHIOJUALIAN'. Only fragments resembling kanji/kana appear, and none spell the intended phrase.
Luma Photon · seed 3333 ち ち ち 0% match · 0/5
The signs show distorted, pseudo-Japanese shapes that do not spell 東京ラーメン; the top red sign reads something like repeated 'ちちち' style glyphs, and the rest are illegible invented marks. The required text is not legibly rendered.
Nano Banana 2 · sample 1 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders 東京ラーメン exactly and legibly with correct glyphs. Surrounding signage includes additional Japanese text (e.g. 営業中, 東京ラーメン repeated on side panels) but the required text is accurate.
Nano Banana 2 · sample 2 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders 東京ラーメン exactly as required, with clean, correct characters. The text is repeated legibly elsewhere in the scene (noren, signboard), all matching the target.
Nano Banana 2 · sample 3 東 京 ラ ー メ ン 100% match · 5/5
The neon sign and the noren both render 東京ラーメン exactly and legibly, matching the required text with correct kanji and katakana. Surrounding signage shows other plausible Japanese text, but the target phrase is accurate.
Nano Banana 2 Lite · sample 1 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders 東京ラーメン exactly and legibly, matching the required text. Additional supporting text like 'RAMEN', 'OPEN', and 博多豚骨 also appears elsewhere but the core required string is correct.
Nano Banana 2 Lite · sample 2 東 京 ラ ー メ ン R A M E N T O K Y O 100% match · 5/5
The primary neon sign renders 東京ラーメン exactly and legibly, matching the required text precisely. An additional 'RAMEN TOKYO' subtitle appears below but does not detract from the correct rendering. Other background signage contains additional Japanese characters unrelated to the requirement.
Nano Banana 2 Lite · sample 3 東 京 ラ ー メ ン R A M E N I Z A K A Y A 100% match · 5/5
The main neon sign renders 東京ラーメン exactly and legibly with correct characters. Additional accurate text (RAMEN IZAKAYA, ラーメン on the noren, 拉麺 on lanterns) also appears cleanly.
Nano Banana Pro · sample 1 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders exactly 東京ラーメン with correct, legible characters and no misspellings. A secondary vertical sign also shows 東京ラーメン consistently.
Nano Banana Pro · sample 2 東 京 ラ ー メ ン 100% match · 5/5
The large neon storefront sign reads 東京ラーメン as required, matching the target text. The characters are mostly legible though the katakana portion is slightly warped, and secondary signs elsewhere show broken or nonsensical glyphs.
Nano Banana Pro · sample 3 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly, with all six characters correct and legible. There is also a smaller vertical 東京ラーメン sign visible on the side pillar, consistent and correct.
Qwen-Image · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders exactly 東京ラーメン with correct spelling and clean, legible glyphs. Smaller secondary signs in the storefront contain garbled or invented characters, but the required text is fully accurate.
Qwen-Image · seed 2222 東 京 ラ ー メ ン 100% match · 5/5
The primary neon sign renders the required text exactly as 東京ラーメン with correct, legible kanji and katakana. A smaller superscript ラーメーン appears above but does not detract from the main correct rendering.
Qwen-Image · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The main neon sign renders 東京ラーメン exactly as required, with the characters legible and correctly formed. Secondary signage in the windows is illegible but not required.
Recraft V3 · seed 1111 T O K Y O R A M E N 0% match · 0/5
The image shows the English romanization 'TOKYO' and 'RAMEN' in red neon rather than the required Japanese '東京ラーメン'. The intended Japanese characters are entirely absent.
Recraft V3 · seed 2222 T O K Y O R A M E N 0% match · 0/5
The main sign renders 'TOKYO RAMEN' in Roman characters instead of the required '東京ラーメン'. A faint vertical sign contains some illegible pseudo-glyphs that do not spell the correct Japanese text.
Recraft V3 · seed 3333 T O K Y O R A M E N 0% match · 0/5
The visible text is 'TOKYO RAMEN' in Latin script, which is the romanized equivalent but not the required Japanese characters 東京ラーメン. The prompt explicitly demanded the Japanese glyphs, which are entirely absent.
Seedream 4.5 · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The text renders exactly as required, '東京ラーメン', and appears correctly twice in the image. The characters are legible and accurately formed in neon script.
Seedream 4.5 · seed 2222 東 京 ラ ト メ ン 83% match · 3/5
The required text was 東京ラーメン, but the rendered sign shows 東京 correctly followed by distorted katakana approximating ラトメン, missing the prolonged sound mark (ー) and using an incorrect character. The kanji are legible but the katakana is inaccurate.
Seedream 4.5 · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The rendered text matches the required 東京ラーメン exactly, with the characters clear and correctly formed in neon. There are no misspellings or broken glyphs; the kanji and katakana are all accurate and legible.
Seedream 5.0 Lite · seed 1111 東 京 ラ ー メ ン 100% match · 5/5
The sign renders exactly 東京ラーメン with all six characters correct and legible. The neon glow is crisp and there are no misspellings or broken glyphs.
Seedream 5.0 Lite · seed 2222 東 京 ラ ー メ ン 100% match · 5/5
The text renders exactly the required '東京ラーメン' with correct, legible characters. There are no misspellings or invented glyphs, and it is clearly displayed in the neon sign.
Seedream 5.0 Lite · seed 3333 東 京 ラ ー メ ン 100% match · 5/5
The main neon sign renders 東京ラーメン exactly and correctly with all characters legible. The reflection in the puddle shows a mirrored, distorted version of the same text, which is expected. No spelling errors are present.
Stable Diffusion 1.5 · seed 1111 L C O N 0% match · 0/5
The visible neon renders Latin letters resembling 'LCON' plus several distorted, non-standard glyphs, rather than the required 東京ラーメン. The correct Japanese characters do not appear at all.
Stable Diffusion 1.5 · seed 2222 N O P R E O D E 0% match · 0/5
The required text was 東京ラーメン, but the sign instead shows distorted Latin-like characters reading approximately 'NOP REODE' along with illegible vertical glyphs on the right. None of the correct Japanese characters are legibly present.
Stable Diffusion 1.5 · seed 3333
no legible text
0% match · 0/5
No legible or correct text is present; the red glowing marks are distorted, invented glyphs that do not spell 東京ラーメン or any real characters.
Inspect the letterforms
FLUX.2
FLUX.2 [pro]
FLUX.2 [dev] Turbo
Gemini 2.5 Flash Image
GPT Image 1.5
Ideogram 3.0
Luma Photon
Nano Banana 2
Nano Banana 2 Lite
Nano Banana Pro
Qwen-Image
Recraft V3
Seedream 4.5
Seedream 5.0 Lite
Stable Diffusion 1.5
Move the cursor over any image: all panels magnify the same spot at 3×
so letterforms and fine detail compare directly.
Run-to-run consistency
Same model, same prompt, different seed or sample. Click a card to
flip between the two generations: the less it moves, the safer the
model is for templated, variable-data production.
Scores
Mean per model across this prompt's runs. Auto-measured criteria plus an
interim Claude-VLM tier for prompt adherence and composition; any
criterion still awaiting review is labeled pending, never zeroed.
We re-run the identical 37-prompt suite on every model release. Get the
scores, and what changed in the rankings, in your inbox.
You're on the list.
We'll email you when new benchmark results ship.
Frequently asked questions
Image generation is stochastic: the same prompt produces different images on every run, so a single sample measures luck, not ability. Every model runs each benchmark three times with fixed seeds (1111, 2222, 3333), or three unseeded samples where the API accepts no seed. Scores average all three samples, and the consistency benchmarks measure the variation itself.
Yes. Every model receives the same prompt text with default parameters and no per-model tuning, so differences in output reflect the model, not prompt engineering. The suite is versioned and prompts are append-only, which keeps historical scores comparable.
The results on this page are scored on this exact prompt, so the strongest model for it is easy to spot; this benchmark sits in the Typography & Text Rendering category. Scores are per model version and reflect this benchmark only. For a ranking across every benchmark see the model rankings, and to weight the numbers by a specific job see the use-case pages.
Each image is scored 0 to 5 per criterion. Measured criteria (resolution, latency, cost, transparency, color accuracy) are computed automatically; quality criteria (text accuracy, prompt adherence, composition) are judged by an automated Claude vision tier against each prompt’s checklist. Page scores are unweighted means over all of a model’s runs in that scope. Blind expert-panel review has not run yet; the dataset is pilot-0.
Model output is a starting point, not a finished asset. Production work usually needs background removal, exact brand colors or editable text, none of which generation guarantees on every run. The IMG.LY AI Editor gives your users those controls to refine any model’s output to production quality.
The suite re-runs on notable model releases so the rankings stay current. The results shown are the pilot-0 dataset, scored by measured criteria plus an automated Claude vision tier, with blind expert review planned. Prompts are versioned and append-only, so scores stay comparable across runs.
![FLUX.2 [pro]: CJK Neon Sign, magnified detail](https://storage.googleapis.com/imgly-ai-benchmarks/t05-cjk-sign--flux-2-pro--s1111.webp)
![FLUX.2 [dev] Turbo: CJK Neon Sign, magnified detail](https://storage.googleapis.com/imgly-ai-benchmarks/t05-cjk-sign--flux-2-turbo--s1111.webp)
