GenAI Benchmarks

What the data says

Findings from benchmarking 26 models on 37 design-production prompts: the places where every model falls short of production requirements, measured rather than asserted. Each finding names the gap, shows the evidence, and spells out what the editing step has to cover. No single model closes every gap, which is why the editing step matters: IMG.LY CoDesign is a free MCP server that returns fully editable designs rather than flat images, so those gaps get fixed instead of regenerated.

The benchmark avocado mascot as a detective, examining an evidence board of pinned images through a magnifying glass
Transparency

23 of 26 models fail at transparency

We asked every model for transparent PNGs and measured the alpha channel of what came back. Almost none of it survives contact with a real sticker, merch or cut-out pipeline.

Brand color

No model hits your exact hex

Measured with CIEDE2000 against required brand colors, the best model scores 4.11 of 5 and 16 of 26 land below 3. Close-enough color is not the color in your brand book.

Text

Rendered text: close is not shippable

Eleven models render every required string exactly and misspellings have become rare, while the model famous for text lands twenty-third of 26. Baked-in words still mean regenerating the whole image for a one-word change, unless they are editable layers.

Consistency

No model holds a character across scenes

The same described character drifts between scenes on every model we tested. Series work needs identity as a reusable asset, not a regeneration lottery.

AI Editor

The pattern behind every finding

Generation is probabilistic; production requirements are deterministic. The IMG.LY AI Editor closes the gap with manual controls for background removal, exact colors and editable text, giving your users the control they need to be productive with generative AI.

IMG.LY AI Editor: generative image tools next to manual editing controls on a canvas