DALL-E 3: Advancements and Challenges

• DALL-E 3 is a text-to-image system known for its strong prompt-to-image alignment, conversational interface, and iterative prompt refinement.
• It excels in generating emotionally aligned and design-oriented images with superior benchmark performance, yet struggles with precise 3D controllability and accurate non-English script rendering.
• Studies show that its natural language mediation elevates descriptive inputs and design utility, while highlighting persistent issues in cultural representation and stereotype propagation.

DALL-E 3 is a text-to-image system in the DALL-E family that is studied in the recent literature primarily through its observable behavior rather than through public architectural disclosure. Research consistently treats it as a ChatGPT-mediated generator with strong prompt-to-image alignment, unusually broad design utility, and high performance on several human-evaluated tasks, while also documenting persistent weaknesses in exact 3D controllability, multilingual script rendering, and stereotype-sensitive domains (Jahani et al., 2024, Lin et al., 2023, Gao et al., 2024, Sobhan et al., 27 Feb 2025).

1. Position within the DALL-E lineage

The most useful historical contrast comes from work on earlier DALL-E-family systems. A multilingual storytelling wrapper built around the original DALL-E API documented poor non-English prompting performance, lack of native multi-frame storytelling support, difficulty preserving recurring objects and settings across frames, spurious text insertion, and style or color drift; its solution relied on translation into English, iterative edit calls, masking, OCR-based text removal, and saturation matching (Mudrik et al., 2022). In that lineage, DALL-E 3 appears not as an openly specified new architecture in the literature here, but as a later system that is generally stronger at prompt following and easier to use through natural-language interaction.

Work explicitly inspired by DALL-E 3 inside ChatGPT describes the public system less as a standalone image model than as an interaction pattern: conversational image generation, natural-language understanding, prompt rewriting or refinement, and iterative editing across turns. “Mini DALL·E 3” is explicitly framed as an attempt to reproduce that pattern with prompted LLMs and off-the-shelf text-to-image backends, rather than to reproduce DALL-E 3’s proprietary image model itself (Lai et al., 2023).

A related benchmark paper, “Generate Any Scene,” likewise treats DALL-E 3 as a strong proprietary comparator rather than an internally explained model. Its emphasis is on how DALL-E 3 behaves under systematic benchmark stress, especially on compositional prompts and long-tail concepts, not on the model’s undisclosed training stack or internal mechanics (Gao et al., 2024).

2. Conversational prompting and prompt mediation

A defining feature of DALL-E 3 in the current literature is prompt mediation. The public ChatGPT-integrated usage pattern is associated with natural-language interaction, hidden or semi-hidden prompt elaboration, and iterative refinement. In the “Mini DALL·E 3” reconstruction, an LLM interprets ordinary-language requests, decides whether a turn should create a new image or edit a previous one, emits internal image descriptions, and then performs an additional prompt-refinement stage before sending the request to the backend generator; this is presented as the core reason DALL-E 3 felt different from earlier prompt-engineering-heavy workflows (Lai et al., 2023).

The strongest direct evidence that this interaction model changes user behavior comes from an online experiment with 1,891 participants and 18,152 prompts. In a repeated image-recreation task, participants assigned to DALL-E 3 produced images that were on average $z = 0.20$ standard deviations closer to their target images than those produced by participants assigned to DALL-E 2, and the gain decomposed roughly equally into improved model capability and endogenous changes in prompting. Even though participants were blind to assignment, those using DALL-E 3 wrote longer prompts, wrote prompts that were more semantically similar to one another across attempts, and used a greater number of descriptive words. The same study found that automatic prompt revision reduced the benefits of using DALL-E 3 by $58\%$, and that the “verbatim” arm still had $59\%$ of prompts modified by GPT (Jahani et al., 2024).

The implication is not that prompting disappears as models improve. Rather, the literature suggests that stronger systems such as DALL-E 3 induce a different prompting regime: more descriptive, more incremental, and more exploitative of model capacity.

3. Measured capability profile

Several recent studies place DALL-E 3 at or near the top of comparative evaluations, but they do so on different axes.

In a within-subjects $3 \times 2 \times 10$ study of emotional expression, DALL-E 3 achieved a mean emotional-alignment score of $M = 7.04,\ SD = 2.64,\ N = 1920$, compared with $M = 5.79,\ SD = 2.91,\ N = 1920$ for DALL-E 2 and $M = 2.13,\ SD = 2.71,\ N = 1920$ for Stable Diffusion v1. Model choice produced the strongest effect in the study, $F(2, 46) = 242.05,\ p < .001,\ \eta^2_G = 0.62$. DALL-E 3 was the best-performing model overall, but the gains were not uniform: “shock” and “positive surprise” were among the most aligned emotions, while “resentment” and “affection” were among the least aligned, and person depictions were rated higher than robot depictions (Lomas et al., 2024).

On compositional prompt faithfulness, “Generate Any Scene” reports especially strong DALL-E 3 results on a 10K-prompt benchmark: CLIP Score $0.2518$, Pick Score $0.2006$, VQA Score $58\%$0, TIFA Score $58\%$1, and ImageReward Score $58\%$2. The paper states that DALL-E 3’s TIFA score is about $58\%$3 higher than other models, and that it maintains consistent performance across increasing scene-graph complexity, even showing a slight improvement on VQA Score as complexity rises (Gao et al., 2024).

On design-oriented evaluation, “DEsignBench” reports that human annotators preferred DALL-E 3 over both Midjourney and SDXL in all eight evaluated categories: text rendering, composition and layout, color harmony, 3D and cinematography, medium and style, overall image-text alignment, overall aesthetics, and overall design. The paper’s clearest hard numbers concern OCR-based text rendering on 40 samples: DALL-E 3 achieved $58\%$4 word-level accuracy on short words, $58\%$5 on challenging words, $58\%$6 on sentences, and $58\%$7 overall, substantially above Midjourney, SDXL, and IF (Lin et al., 2023).

Taken together, these studies present DALL-E 3 as unusually strong at prompt faithfulness, emotionally legible image generation, and design-relevant multimodal synthesis. They also show that the gains are task-specific rather than uniform across all prompt types.

4. Design, style, and workflow applications

A substantial portion of the literature treats DALL-E 3 as a design tool rather than only as a generic generator. “DEsignBench” is explicitly built around authentic visual design contexts and divides evaluation into “design technical capability” and “design application scenario,” covering text rendering and typography, layout and composition, color harmony, medium and style, 3D and cinematography, infographics, animation or gaming design, product design, and visual art design. Within that framework, DALL-E 3 is treated as the current top-tier model among those tested, and the ChatGPT prompt-expansion workflow is presented as a practical advantage for harder design prompts such as posters, books, and advertisements (Lin et al., 2023).

A separate case study places DALL-E 3 inside a hybrid style-transfer pipeline. There, DALL-E 3 generates the style image from a textual description, and Magenta Arbitrary Image Stylization performs the actual content-style fusion. In that setup, the DALL-E-based condition reached SSIM $58\%$8 and PSNR $58\%$9, compared with SSIM $59\%$0 and PSNR $59\%$1 without DALL-E. The reported mean runtime figures were: with DALL-E, style-image generation $59\%$2, style transfer $59\%$3, and total processing time $59\%$4; without DALL-E, style-image upload $59\%$5, style transfer $59\%$6, and total processing time $59\%$7 (Ike, 2024).

Older workflow work on 3DALL-E predates DALL-E 3, but it remains relevant as a family-level design precedent. Embedded inside Fusion 360, it showed how DALL-E-family generation can be integrated into CAD as a source of reference images, ideation breadth, and anti-fixation support. Designers in that study used AI-first, AI-last, and AI-throughout strategies, and prompt complexity clustered around roughly 2 to 6 concepts (Liu et al., 2022). This suggests that DALL-E 3’s later design relevance is not only a matter of raw image quality, but also of workflow fit.

5. Limits of control and representation

The strongest documented weaknesses of DALL-E 3 concern exact control, especially when prompts demand precise geometry, exact scripts, or culturally specific representational fidelity.

On 3D-controllable image generation, DALL-E 3 is explicitly outperformed by MUSES. In T2I-CompBench it remains competitive on attribute binding, with color $59\%$8, shape $59\%$9, texture $3 \times 2 \times 10$0, and 2D-spatial $3 \times 2 \times 10$1. But on the paper’s new T2I-3DisBench, which targets object count, object orientation, 3D spatial relationships, and camera view, DALL-E 3 scored only $3 \times 2 \times 10$2 on average, with object count $3 \times 2 \times 10$3, orientation $3 \times 2 \times 10$4, 3D spatial relationship $3 \times 2 \times 10$5, and camera view $3 \times 2 \times 10$6. The authors’ interpretation is that strong end-to-end T2I systems such as DALL-E 3 still lack explicit 3D layout planning, 3D object acquisition and calibration, and simulation-grounded rendering (Ding et al., 2024).

A sharper representational failure appears in “Broken Letters, Broken Narratives,” a focused study of Arabic script in DALL-E 3. Using 351 unique prompts, each generated four times under normal prompting and four times with prompt revision disabled, the study analyzes 2,808 images and concludes that correctness of Arabic letter-form representation is effectively zero. The model generates decorative pseudo-writing rather than authentic letters, often produces Latin letters for Arabic or Farsi prompts, and fails to render named calligraphic styles such as Naskh, Thuluth, Nasta'liq, Kufic, Diwani, and Ruq'ah faithfully. The study also reports that terms such as “traditional,” “classical,” “historic,” “ancient,” “old,” “antique,” “antiquity,” and “heritage” appeared in $3 \times 2 \times 10$7 of revised prompts, with an average of 164 instances per batch, while terms like “modern” and “contemporary” were absent (Sobhan et al., 27 Feb 2025).

The broader implication is that multilingual prompt acceptance does not imply multilingual representational competence. Earlier DALL-E-family systems already required external translation and prompt engineering for non-English use (Mudrik et al., 2022); the Arabic-script case study shows that even a later model can remain non-functional on culturally central script-rendering tasks.

6. Bias, stereotype propagation, and evaluation culture

DALL-E 3 is also the subject of domain-specific bias audits. In a study of autism-related imagery, the evaluated OpenAI model is explicitly DALL-E 3. Across 53 prompts and 249 analyzed images, its average ratio of stereotypical themes to number of images was $3 \times 2 \times 10$8, nearly identical to SDXL’s $3 \times 2 \times 10$9, below Midjourney’s $M = 7.04,\ SD = 2.64,\ N = 1920$0, and below Stable Diffusion’s $M = 7.04,\ SD = 2.64,\ N = 1920$1. The study reports strong homogeneity in skin colour, gender, and age, with autistic people often depicted as white, male, and young, and identifies blue colour theme, isolation, and nerd or artist portrayals as especially characteristic. It also highlights a notable dissociation: DALL-E sometimes generated accompanying text claiming to move beyond the puzzle-piece stereotype while still visually reproducing it (Wodziński et al., 2024).

This line of work sits within a broader evaluation culture that predates DALL-E 3. “DALL-Eval” argued that high-fidelity generation and coarse image-text alignment are insufficient evaluation targets, and proposed explicit testing of object recognition, counting, spatial relations, and social bias for DALL-E-family and diffusion-based systems. Its central warning is that visually impressive models can still show large gaps to upper bounds in counting and spatial reasoning, and can still reproduce gender and skin-tone biases learned from web image-text pairs (Cho et al., 2022). Later DALL-E 3 papers, especially those on scene-graph evaluation, design, stereotypes, and Arabic script, effectively instantiate that warning at higher capability levels.

A final contextual strand concerns governance rather than capability. In a family-level comparison that predates DALL-E 3, OpenAI’s DALL-E line is described as incrementally released, centrally governed, trust-and-safety supervised, and access-controlled, in contrast to more open diffusion ecosystems (Brusseau, 2022). That framing does not explain DALL-E 3’s internals, but it does help explain why the literature often encounters the model as a mediated product environment—ChatGPT integration, prompt rewriting, and safety layers included—rather than as a reproducible research artifact.

DALL-E 3 therefore occupies a distinctive place in contemporary text-to-image research. It is simultaneously a leading benchmark baseline, a design-oriented generative tool, a prompt-mediated conversational interface, and a case study in the uneven distribution of generative capability: exceptionally strong on prompt fidelity and many design tasks, but still limited where exact geometry, culturally specific scripts, or stereotype-sensitive representation require more than fluent image synthesis.