Full search page →

Choosing image-generation models: capabilities, evidence and a fair trial

Which image-generation system fits my deliverable, and how do I test it before committing?

13 min readUpdated Sources checked 繁體中文

Choose an image system by the image you need to deliver. A convincing concept sketch, a product photo with an unchanged label, and a diagram with exact text need different checks. The most attractive sample in a gallery may not answer your question.

The main text explains capabilities and evaluation in terms that stay true as models change. Specific products, model IDs, open-weight releases and benchmark links are in Current options. No generation experiment or ranking is reported here.

Key points

  • Name the operation first: text-to-image, instruction editing, masked inpainting/outpainting, reference-guided generation, or structure-controlled generation. Systems differ more by operation than by overall “quality”.
  • Most current systems iterate from noise in a compressed latent space; a few generate discrete image tokens. Those mechanisms explain common failures such as drift in “unchanged” regions during edits.
  • Judge a system on capability axes that match the deliverable: prompt adherence, text rendering, edit preservation, identity consistency, resolution, control, speed and cost, and usage rights.
  • Automatic metrics and leaderboards measure narrow things: FID compares distributions, CLIP-based scores measure text–image agreement, compositional benchmarks count objects and relations, preference arenas measure taste. None certifies your use.
  • Run a small acceptance sheet with your own briefs and assets, keep the failures, and compare cost per accepted asset.
  • Check the license of the exact weights and the hosted terms; consider provenance metadata for published images.

First decide which operation you need

OperationStarts fromTypical useWhat to inspect
Text-to-imageA written descriptionConcepts, illustrationsComposition, object count, spatial relations
Instruction editingAn image and an instruction“Make the jacket red”Requested change and everything that should not change
Inpainting / outpaintingAn image plus a mask or a larger canvasRemove an object; extend a backgroundSeams, lighting continuity, mask boundaries
Reference-guidedOne or more reference imagesA consistent character or product across a setIdentity, proportions, small details such as logos
Structure-controlledA sketch, edge map, depth map or poseLayout-exact compositionsFaithfulness to the control signal
UpscalingA low-resolution imageFinal delivery sizeInvented detail, texture artifacts

A feature called “reference” can mean style guidance, identity preservation or a boundary frame depending on the product; read its definition. Structure control became widespread through methods such as ControlNet, which adds a trainable conditioning branch to a frozen text-to-image model so that an edge map, depth map or pose can steer layout. [1]

How image generators work, briefly

Plain version. Most modern generators start from random noise and remove it step by step, guided by your text, until an image remains. To save computation they work on a compressed version of the image and decode it at the end.

Precise version. Latent diffusion trains an autoencoder that maps images to a smaller latent grid, then trains a denoising network in that latent space, conditioned on text through cross-attention; generation runs the denoiser for a number of steps and decodes the result. [2] Many recent systems replace the convolutional denoiser with a transformer and train with rectified flow, a formulation that connects data and noise along a straight line. [3] A different family generates discrete image tokens, either one at a time or by predicting masked tokens in parallel, as in Google Research’s Muse. [4] Some current products generate and edit images inside a multimodal language model’s conversation, which makes multi-turn editing natural. The site’s ML 08 and ML 09 explain these mechanisms in depth.

What the mechanism implies for selection:

  • Edits re-generate. A common editing approach adds noise to the input image and denoises it toward the new instruction; the amount of noise trades faithfulness to the original against freedom to change. [5] That is why “unchanged” areas can drift, and why mask-based inpainting or dedicated edit models preserve more.
  • Text rendering varies by system and script. Letters are small, exact details, and some releases name text rendering as an explicit training focus (see Qwen-Image under Current options). Test your actual script and language, including Traditional Chinese, at the final display size.
  • Seeds are not portable. A seed fixes the starting noise for one model, sampler and resolution. The same seed in another system, or another version, is a different experiment.
  • Steps and guidance are part of the result. Fewer sampling steps are faster; the guidance scale trades prompt adherence against diversity and naturalness (classifier-free guidance in ML 08). Record these settings with every output.

Capability axes and how to test each

AxisWhat it meansA quick test
Prompt adherenceCorrect objects, counts, attributes and spatial relations“Three red cups left of a blue bowl”: count and position
Text renderingExact characters, spelling, punctuationYour real headline, in your language, at final size
Edit preservationOnly the requested region changesEdit one object; diff the rest of the image
Identity consistencySame subject across images and editsFive scenes of one character or product
Style range and controlCan it match a brand style or a reference?One brief in three specified styles
Resolution and aspectNative sizes; quality of upscalingGenerate at your delivery size and aspect ratio
Speed and costTime and spend per image at your settingsMeasure across the whole trial, including failures
Rights and provenanceWeight license, hosted terms, content credentialsRead the license file and terms for your intended use

What benchmarks and metrics can tell you

Plain version. Each metric checks one narrow property. Use them to shortlist, never to decide.

  • FID (Fréchet Inception Distance) compares the statistics of many generated images with many real images in a pretrained network’s feature space. It measures distribution-level realism and diversity, needs thousands of samples, and says nothing about whether a particular image matches its prompt. [6]
  • CLIP-based scores such as CLIPScore measure agreement between an image and a text using a pretrained image–text model; they are cheap, but like FID they give a holistic score that is unsuited to checking individual objects, counts or positions. [7, 8]
  • Compositional benchmarks (object, count, attribute and relation checks scored by detectors or question-answering models) target prompt adherence directly; their scores depend on the benchmark version and the judge model. [8, 9, 10]
  • Human preference leaderboards rank systems by pairwise votes. They capture overall appeal, not exact text, edit preservation or rights. [11, 12]

ML 08’s section on evaluating generative models covers FID and its successors. Whatever the source, record the benchmark version, model ID, quality setting, resolution, sample count and evaluator, and keep provider-reported claims separate from independent comparisons.

Run a small acceptance sheet

Use the same brief, input assets, delivery size and budget for every candidate.

TrialWhat to inspect
A simple scene with specified objects and positionsMissing objects, spatial relations, composition
A poster with your exact textSpelling, Traditional Chinese characters, punctuation, legibility
An edit confined to one objectThe requested change and unintended changes elsewhere
A reference-led set of related assetsSubject identity, style consistency, useful variation
An export in the final formatDimensions, transparency if required, visible artifacts

Keep failed outputs, not only the best sample. Record cost, retries, editing time and pass/fail per trial. Cost per accepted asset divides all spending, including failures, by the number of assets that passed; report human cleanup separately if you do not price it. A slower system that needs fewer repairs can be cheaper overall. The model-evaluation guide explains why a handful of trials gives only a rough estimate.

Running models locally

A downloadable model runs inside a pipeline: text encoders, the denoiser, the image decoder, and often extra components for control or upscaling. Node-based workflow tools let you wire these together and save the whole graph, which is the best record of how an image was made. The tool is a workbench, not the model: changing the model changes what the same graph can produce, and some tools also offer paid cloud nodes, so a local interface does not prove every step ran locally. [13]

Estimate memory for the whole pipeline, not just the main weight file: text encoders, decoder, intermediate tensors, output resolution and batch size all count. Reduced precision and moving components between devices trade memory for speed. Start with a documented configuration, measure peak memory and time, and add custom nodes only when they solve a clear need, since they bring executable code and dependencies.

Rights and provenance

Image models carry the same layered terms as language models: code, weights and hosted service can each have different licenses, and one family can mix permissive and non-commercial terms (licensing layers). Check the exact checkpoint’s license and any extra components it needs. If you publish images, consider attaching provenance metadata: the C2PA standard defines Content Credentials that record where a piece of content came from and how it was edited. [14]

A practical starting route

For occasional illustrations, begin with a hosted interface and a clear brief. For repeatable asset production, test an API or a saved workflow. For local operation or customization, evaluate an open-weight release against your hardware and license needs. Keep exact text and precise layout in a conventional design tool when that gives more reliable control. When still images pass, use them as deliberate inputs to motion work: continue with choosing video-generation models.

Current options, checked 6 November 2026

Representative examples, not a complete list or a ranking. Record the exact model ID or checkpoint revision with every output: an alias, a dated snapshot and a product-interface name are different records.

Hosted image systems

FamilyCurrent IDs or versions found in official documentationFirst trial to run
OpenAI GPT ImageCatalog lists gpt-image-2.6-sunburst (detailed work) and gpt-image-2.6-flare (fast iteration), plus 2.5 models with dated snapshots such as gpt-image-2.5-sunburst-2026-09-08; gpt-image-1 was shut down on 23 October 2026 [15, 16, 17]Revise one object while preserving the scene
Google Gemini image modelsgemini-nano-banana-2.1 (Nano Banana 2.1, an update to Nano Banana 2) and the stable gemini-3.1-flash-image (Nano Banana 2) and gemini-3-pro-image [18, 19]A sequence of conversational edits; check identity drift
MidjourneyExplicit version selection; check the version page for the current default [20]Concepts for one brief; effort to refine one
Adobe FireflyAdobe and partner models in one interface [21]Finish an asset inside the design workflow you will use
Meta Muse Imagemuse-image-1.0 for generation and multi-turn editing [22]A reference-led edit; inspect identity and unchanged regions
Others with official model pagesMicrosoft MAI-Image-2.6, xAI Grok Imagine Image 2.0, ByteDance Seedream 4.0 [23, 24, 25]Same acceptance sheet

Firefly and some API platforms host partner models; a model available inside a product is not necessarily trained by that product’s company. Runway’s API catalog, for example, lists image models from several companies, including Seedream 5 variants. [26] Subscription and API access can carry different entitlements.

Downloadable weights

ReleaseWhat the primary source establishesSelection implication
Qwen-Image and Qwen-Image-Edit-2509Apache-2.0 generation model with a text-rendering focus; a separate editing checkpoint [27, 28]Generation and editing checkpoints are not interchangeable
FLUX.2 [klein] 4B / 9B4B under Apache-2.0; 9B open weights under non-commercial terms [29, 30]Do not apply one size’s license to the family
Stable Diffusion 3.5 LargeStability AI Community License [31]Read its conditions; it is not Apache-2.0
SDXL base 1.0Separate model card and license [32]Match the workflow’s base/refiner requirements
PixArt-Σ XL-2-1024-MSDiffusion-transformer release with an inference pipeline [33]Measure the whole pipeline on your hardware
HunyuanImage-3.0Tencent community license; generation and Instruct variants [34]Keep the variants distinct
Janus-Pro-7BUnified understanding/generation research release [35]Test generation separately from understanding

Workflow tool. ComfyUI is a widely used node-based workbench and backend; its repository is GPL-3.0, and it offers optional paid API nodes. [13]

Two different “Muse” names. Meta Muse Image is the hosted line above. Google Research’s Muse (2023) is a research method for masked generative image transformers, read alongside the official MaskGIT implementation; it is not evidence of a callable Google Muse API or a downloadable checkpoint. [4, 36]

Benchmark entry points

ResourceUseful evidenceCheck before comparing
GenEval / GenEval 2 [8, 9]Objects, counting, attributes and relationsBenchmark version, judge, per-skill results
T2I-CompBench / ++ [10]Compositional attributes and relationshipsEdition, categories, evaluation model
DPG-Bench [37]Dense-prompt alignment through questionsPrompt set, sampling, evaluator
Arena text-to-image leaderboard [11]Pairwise preference rankingsModel variant, date, vote count and uncertainty
Artificial Analysis image models and methodology [12]Preference quality, price, generation timeGeneration versus editing, provider, settings

Further reading

References

Accessed 6 November 2026 unless noted:

  1. Lvmin Zhang, Anyi Rao and Maneesh Agrawala, “Adding Conditional Control to Text-to-Image Diffusion Models,” ICCV, 2023. Paper.
  2. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer, “High-Resolution Image Synthesis With Latent Diffusion Models,” CVPR, 2022. Paper.
  3. Patrick Esser et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML (PMLR 235), 2024. Paper.
  4. Huiwen Chang et al., “Muse: Text-To-Image Generation via Masked Generative Transformers,” arXiv, 2023. Project, paper.
  5. Chenlin Meng et al., “SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations,” arXiv:2108.01073, 2021. Paper.
  6. Martin Heusel et al., “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017 (introduces FID). Paper.
  7. Jack Hessel et al., “CLIPScore: A Reference-free Evaluation Metric for Image Captioning,” EMNLP, 2021. Paper.
  8. Dhruba Ghosh, Hanna Hajishirzi and Ludwig Schmidt, “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment,” 2023. Paper, evaluation code.
  9. A. Kamath et al., “GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation,” 2025. Official data and evaluation.
  10. K. Huang et al., “T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation,” NeurIPS, 2023. Benchmark and ++ release.
  11. Arena, “Text-to-Image Leaderboard,” live resource. Leaderboard.
  12. Artificial Analysis, “Image Model Comparisons” and “Image Generation Benchmarking Methodology.” Comparisons, methodology.
  13. ComfyUI contributors, “ComfyUI.” Repository and license.
  14. Coalition for Content Provenance and Authenticity (C2PA), “C2PA Specifications.” Overview, specification 2.2.
  15. OpenAI, “Image generation.” API guide.
  16. OpenAI, “Models” and “Create image” API reference. Catalog, GPT-Image-2.6 Sunburst, supported model IDs and snapshots.
  17. OpenAI, “Deprecations.” Shutdown schedule.
  18. Google, “Gemini Nano Banana 2.1,” developer documentation. Model page.
  19. Google, “Gemini API models” and “Gemini 3.1 Flash Image.” Catalog, Gemini 3.1 Flash Image, deprecations.
  20. Midjourney, “Version.” Version documentation.
  21. Adobe, “Adobe Firefly.” Product and partner models.
  22. Meta, “Muse Image,” 2026. Model IDs, image-generation guide, announcement.
  23. Microsoft AI, “MAI-Image-2.6.” Model page.
  24. xAI, “Grok Imagine Image 2.0,” API documentation. Model ID and capabilities.
  25. ByteDance Seed, “Seedream 4.0,” official model page. Release and access.
  26. Runway, “Available AI Models,” API documentation. Model catalog.
  27. Qwen team, “Qwen-Image,” 2025. Model card.
  28. Qwen team, “Qwen-Image-Edit-2509,” 2025. Model card.
  29. Black Forest Labs, “How does FLUX licensing work?” License overview.
  30. Black Forest Labs, “FLUX.2 [klein],” 2026. 4B model card, 9B model card.
  31. Stability AI, “Stable Diffusion 3.5 Large,” 2024. Model card and license.
  32. Stability AI, “Stable Diffusion XL base 1.0,” 2023. Model card.
  33. PixArt team, “PixArt-Sigma-XL-2-1024-MS,” 2024. Model card.
  34. Tencent Hunyuan, “HunyuanImage-3.0,” 2025. Model card.
  35. DeepSeek-AI, “Janus-Pro-7B,” 2025. Model card.
  36. Google Research, “MaskGIT,” official implementation, 2022. Code and paper links.
  37. TencentQQGYLab, “ELLA — DPG-Bench,” 2024. Prompts and evaluation instructions.
  • #image-generation
  • #models
  • #tools
  • #evaluation