Visual Allusions in Projection and AI
- Visual Allusions are structured discrepancies between an object’s actual features and its apparent visual cues that enable dynamic reinterpretation without material change.
- They merge historical art practices with modern projection mapping by aligning physical surfaces and superposed imagery to create perceptually fused experiences.
- In computational contexts, visual allusions serve as diagnostic test cases for multimodal models, highlighting differences between human perceptual judgment and AI interpretation.
Searching arXiv for the cited papers and closely related work on visual illusions/allusions in projection mapping and multimodal models. arXiv search: "Augmenting reality shared history perceptual illusion video projection mapping (Pastor, 2020)" arXiv search: "The Illusion-Illusion Vision LLMs See Illusions Where There are None (Ullman, 2024)" arXiv search: "Do Large Vision-LLMs Distinguish between the Actual and Apparent Features of Illusions (Shinozaki et al., 6 Jun 2025)"; "Grounding Visual Illusions in Language (Zhang et al., 2023)"; "IllusionBench+ (Zhang et al., 1 Jan 2025)"; "SemVink (Li et al., 3 Jun 2025)"; "Beneath the Surface Visual Allusions Dixit (Ahuja et al., 7 Apr 2026)"; "The Art of Deception Color Visual Illusions and Diffusion Models (Gomez-Villa et al., 2024)". Visual allusions are configurations in which an image, object, scene, or clue causes one thing to be seen, inferred, or communicated as another without requiring a material change in the support itself. In the most historically expansive account, their operative basis is not merely resemblance but the “spatial correspondence between objects and displayed images,” through which physical surfaces and superposed imagery become perceptually fused (Pastor, 2020). In more recent computational work, related phenomena are formalized through the distinction between actual and apparent features, through “illusion-illusions” that resemble canonical illusion templates while lacking the decisive illusory property, through hidden content that emerges only under scale change, and through multimodal clueing tasks in which image reference must remain indirect rather than literal (Shinozaki et al., 6 Jun 2025, Ullman, 2024, Li et al., 3 Jun 2025, Ahuja et al., 7 Apr 2026).
1. Conceptual definition and scope
Visual allusion is best understood as a structured discrepancy or tension between literal support and perceived, implied, or assigned identity. Pastor does not provide a formal dictionary-style definition, but his account makes clear that the phenomenon is produced when an object’s visible identity is altered “without materially changing the object itself,” and when image and support are brought into sufficiently coherent alignment that the object appears altered, animated, replaced, or re-signified (Pastor, 2020).
This places visual allusion adjacent to, but not identical with, representation, simulation, trompe-l’oeil, illusionism, augmented reality, and spatial augmented reality. Representation can occur on a picture plane without any necessary relation to a physical target. Simulation aims at broader substitution or modeling. Trompe-l’oeil and quadratura are narrower historical techniques. Pastor’s account is broader because it includes site-specific painting, panoramic environments, lantern projection, theatre, expanded cinema, spatial augmented reality, and dynamic projector-camera systems. He explicitly distinguishes projector-based augmented reality from “mere spatialized motion image” by the “strong interactive component of the experience,” while also showing that visual allusion can occur without interactivity and can therefore predate AR as a technical category (Pastor, 2020).
A second conceptual axis is the difference between what an image objectively contains and what it appears to contain. The FILM framework names these as “actual features” and “apparent features.” A genuine illusion is one in which the two diverge; a fake illusion preserves the familiar geometry of an illusion stimulus while making actual and apparent features coincide; a control image removes the relevant inducer (Shinozaki et al., 6 Jun 2025). This distinction is directly useful for visual allusions, because many allusive images depend on the fact that apparent interpretation reaches beyond literal structure.
A third axis is pragmatic rather than optical. In the Visual Allusions game, an image clue is successful only when it is understood by some but not all players. Here the allusive property lies not in the image alone, but in socially calibrated, selective reference. Literal captioning fails because the clue becomes too obvious; complete opacity fails because nobody can recover the intended image (Ahuja et al., 7 Apr 2026). This suggests that visual allusion can be perceptual, semantic, and communicative at once.
2. Historical lineage of illusionistic practice
Pastor places contemporary projection-based practices inside a genealogy that begins well before digital media. He starts with pre-Renaissance painting, especially Giotto and Duccio, whose use of shadow and color contrast in chiaroscuro produced early effects of depth and volume. The decisive turn, however, is the fifteenth-century development of linear perspective. Brunelleschi’s experiments and Alberti’s De Pictura establish a “standardized model of human visual perception” from which mathematical rules could be derived for pictorial composition. Perspective thereby becomes both a representational technique and a theory of perception, governing vanishing points, horizon lines, and diminution with distance. In Piero della Francesca’s Brera Madonna, perspective also organizes symbolic and compositional meaning rather than realism alone (Pastor, 2020).
The transition from pictorial illusion to site-bound allusion becomes clearer in quadratura and trompe-l’oeil. Masaccio’s Holy Trinity and Mantegna’s Camera degli Sposi align painted architecture with actual architecture, while Andrea Pozzo’s Apotheosis of St Ignatius extends this principle into large-scale ceiling illusion. In these cases, the illusion does not remain enclosed within a frame; it collaborates with the site and is experienced by a moving body. Pastor therefore treats them as early precedents for projection mapping, because depicted elements and built form are made to merge coherently (Pastor, 2020).
Eighteenth- and nineteenth-century immersive image systems intensify this environmental logic. Robert Barker’s Panorama transforms perspective into a 360-degree form in which the spectator stands inside a circular world reinforced by stairs, railings, flooring, and fountains matching the painted environment. The Magic Lantern introduces projection proper through painted glass slides, lenses, and a light source. Phantasmagoria radicalizes this with frontal and rear projections on screens and solid objects. Daguerre’s Diorama adds enormous painted scenes, translucent layers, controlled illumination, and moving scenic elements. Across these cases, fixed pictorial illusion migrates toward live, temporal, and environmental illusion (Pastor, 2020).
Twentieth-century developments complicate the older regime of a single privileged observer. Pastor includes Cubism and Futurism because they destabilize Renaissance fixed-point viewing. Cubist illusionism, in his formulation, “subverts this kind of realistic representation by calling attention to its own artifice.” Expanded cinema and theatre then extend projection into complex object-image environments: Josef Svoboda’s multi-projection stage works, Stan VanDerBeek’s Movie Drome, Disney’s Haunted Mansion, Tony Oursler’s projections on dolls and sculptural bodies, Michael Naimark’s Displacements, Jeffrey Shaw’s dome systems, Robert Lepage’s stage environments, and Fura dels Baus’s Le Grande Macabre all exemplify the persistent drive to animate objects and dissolve the boundary between support and image (Pastor, 2020).
3. Operative mechanics: correspondence, registration, and identity transference
Pastor’s most useful analytic term is “spatial correspondence.” A physical object and a projected or depicted image become perceptually fused when they correspond in location, contour, scale, and orientation. In this model, the image does not simply depict the object; it overlays and modifies the object’s appearance. Registration is therefore not an auxiliary concern but a constitutive one. The projected image must be geometrically adjusted so that its forms coincide with the target surface, and the scale must match closely enough to seem native to the object (Pastor, 2020).
Viewpoint is equally decisive. Renaissance quadratura often depends on a fixed privileged point from which the illusion coheres. Modern mapping systems can preserve correspondence dynamically through calibration and tracking, recomputing the image as the observer or target moves. Pastor’s account of spectatorship therefore runs from controlled and predetermined viewing positions to adaptive, full-body perceptual illusions “no longer tied to the spectator’s spatial positions and viewpoints predetermined beforehand” (Pastor, 2020).
He also distinguishes two nearly simultaneous levels of result. The first is sensory. Projection mapping modifies the visible “shape, color and texture properties” of the physical surface: the surface may contribute its material qualities to the superimposed image, or those qualities may seem to disappear beneath the image. The second is semantic. Here the interaction between imaginary content and material support can generate “identity transference,” so that objects may contrast with projected images or “completely assume its identity and its network of semantic associations.” A bust can become a singing ghost; a ceiling can become an opening to heaven; a face can become a skull or android without ceasing materially to be what it is (Pastor, 2020).
This two-level account makes visual allusion broader than total immersive deception. Allusion can involve partial correspondence, friction, or overt artifice. The cue need not erase the support completely; it can instead redirect interpretation. A plausible implication is that visual allusion often operates at the threshold where support remains visible enough to be recognized, yet is reorganized strongly enough to sustain a second identity.
4. Projection mapping, spatial augmented reality, and synthetic reality
Pastor defines video projection mapping as “a set of imaging techniques in service of the spatially coherent projection of bi dimensional images onto physical three dimensional objects.” What distinguishes contemporary systems from many earlier illusion devices is real-time adaptation. Earlier forms often depended on static alignment, controlled lighting, and predetermined spectator positions. Modern projector-camera systems add computer vision, calibration, and tracking so that images can be recomputed under changing real-world conditions (Pastor, 2020).
Several systems mark this transition. The CAVE provided surround projection and motion tracking for embodied interaction. “Office of the Future” projected onto walls, furniture, objects, and people using “real time computer vision techniques to dynamically extract per pixel depth and reflectance information for the visible surfaces in the office including walls, furniture, objects, and people, and then to either project images on the surfaces, render images of the surfaces, or interpret changes in the surfaces.” Omote used infrared markers on a model’s face to estimate “face position and orientation,” render a CG face model with animated texture, and send the registered image to the projector in real time. Work from the Ichikawa Senoo Laboratory is described as overcoming “the limited static or quasi static conditions required for video projection mapping” through a system “capable of adjusting images in real time onto deforming non rigid surfaces” (Pastor, 2020).
Two technologies are singled out in that context. The “Deformable Dot Cluster Marker” is presented as an infrared acquisition method robust to deformation and occlusion. “DynaFlash” is described as a high-speed projector displaying 8-bit images at up to $1000$ fps with latency as low as $3$ milliseconds. Pastor interprets this as a “breakpoint” in spatial illusion because it produces the effect “as if the projected images were printed onto the target surface.” That phrase expresses an ideal of perfect registration in which projection no longer appears as external illumination but as an inherent property of the object (Pastor, 2020).
Pastor locates projection mapping within the “reality-virtuality continuum” as “an intermediate stance,” and uses the term “synthetic reality” for environments in which virtual appearances increasingly supplant the visible authority of the material world. The distinguishing feature of projector-based illusion is that it is “untethered” and “available to the plain senses”: it is shared in ordinary space rather than mediated by head-mounted devices. The culmination of the historical trajectory is captured in his claim that “every surface may act as a screen and the relation to everyday objects is open to perceptual alterations” (Pastor, 2020).
5. Visual allusions as computational test cases
In multimodal AI, visual allusions have become diagnostic stimuli for separating image-grounded perception from template matching, language priors, and hallucination. Ullman’s “illusion-illusion” paradigm inverts ordinary illusion testing by presenting “neighbors of common illusions” that should not produce a discrepancy between appearance and reality for human viewers. The study uses 10 representative visual illusions, paired illusion-illusion counterparts, and controls, and evaluates eight vision-LLMs: GPT-4o, Claude 3, Gemini Pro Vision / Gemini 1.5, miniGPT, Qwen-VL, InstructBLIP, BLIP-2, and LLaVA-1.5. On base prompts, GPT-4o, Claude 3, and Gemini report a majority of illusion-illusions as illusions; when prompts are prefixed with “In the following visual illusion,” performance on illusion-illusions and controls “crater[s],” and “all or nearly all illusion-illusions are reported as illusions” by Claude 3, GPT-4o, and Gemini Pro (Ullman, 2024).
The FILM dataset sharpens the same issue through actual/apparent dissociation. It constructs genuine illusions, fake illusions, and controls using abstract stimuli, then asks separate VQA questions about what a feature “is” and what it “appears” to be. After human validation with 52 participants and removal of a Ponzo illusion that fooled only of participants, the final dataset contains 112 images across 14 illusion types: 28 genuine, 28 fake, and 56 control. Humans achieve 100.0% “Both Correct” on fake illusions, whereas GPT-4o and Claude 3.5 overwhelmingly become “Only Apparent”: for GPT-4o, 78.6% to 85.0% depending on prompting; for Claude 3.5, 83.3% to 95.0% (Shinozaki et al., 6 Jun 2025). The implication is not merely error on illusions, but failure to revise answers when a familiar-looking configuration has been changed.
Earlier VLM evaluation work makes a related point through grounded language tasks. GVIL contains 100 images and 1600 instances across five illusion categories and four tasks: SameDiffQA, RefQA, AttrQA, and RefLoc. The headline result is that overall alignment with human illusion perception is low, but larger models are closer to human perception and more susceptible to illusions. Humanlike performance peaks at 14.0% on RefQA and 11.2% on AttrQA, while the best RefLoc score reaches 44.5%, achieved by Unified-IO XL (Zhang et al., 2023). IllusionBench scales this logic to 1,051 images, 5,548 question-answer pairs, and 1,051 golden descriptions spanning classical, real-scene, no-illusion, Ishihara, and trap categories. GPT-4o is the top model on true-or-false and multiple-choice tasks at 80.59% and 76.75%, yet trap illusions remain especially revealing: GPT-4o scores 0.5000 on true-or-false trap items, versus 1.0000 for humans, and its trap-description score is only 0.3333 (Zhang et al., 1 Jan 2025).
A broader synthesis argues that illusions reveal both overlap and divergence between human and AI perception. Some illusion-like effects emerge in models through targeted training or as by-products of pattern recognition, but AI also exhibits distinctive vulnerabilities such as hallucinations and pixel-level sensitivity. Even better recent models are summarized as misclassifying illusion-of-illusion cases “around 50%” of the time, indicating that superficial alignment with human illusion judgments does not guarantee shared mechanisms (Yang et al., 17 Aug 2025).
Generative models add a different perspective. In diffusion models, intermediate DDIM inversion states exhibit human-like brightness, hue, and saturation shifts. On BRI3L, DDIM 10-step inversion achieves Perception Accuracy Scores of 84.17, 98.85, and 100.00 at thresholds 0.8, 0.9, and 1.0; on color-oriented datasets, DDIM 10 steps reaches 90.40 on IllusionVQA, 96.70 on GVIL, and 100.00 on HallusionBench. The same work also generates new unseen brightness and color illusions in realistic images and validates them psychophysically: 15 observers judged targets as different in generated illusion images with mean 64%, versus mean 13% in controls (Gomez-Villa et al., 2024). In that setting, visual allusion becomes not only an evaluation problem but a design tool for context-dependent perceptual effects.
6. Hidden content, subtext, and selective image communication
Some visual allusions are scale-dependent rather than context-contrast-based. HC-Bench targets hidden content recognition in 112 synthetic images: 56 hidden text images and 56 hidden object images. Standard VLM prompting performs at near-zero accuracy, reported as 0–5.36%, even under direct questioning, hints, prompt engineering, and few-shot examples. SemVink instead applies low-resolution preprocessing, with best performance at 32–128 pixels, and reports 91.07–100% accuracy across models, summarized in the abstract as “>99% accuracy.” The paper attributes the effect to removal of redundant visual noise and reduction of embedding redundancy from about 1000 repeated tokens in high-resolution inputs to about 10 repeated tokens in low-resolution inputs (Li et al., 3 Jun 2025). Here visual allusion depends on global structure that becomes legible only when local texture is suppressed.
The explicitly named Visual Allusions environment turns the problem into a multimodal communication game inspired by Dixit. The image deck contains 156 synthetic images generated with Imagen 4. In 100 four-player games, frontier models show a strong bias toward overly literal clueing. Gemini-2.5-Pro is the best baseline storyteller, yet its clues are still “obvious” 59.28% of the time; GPT-5, GPT-5-mini, Claude-Sonnet-4.5, and Gemini-2.5-Flash all exceed 70% obvious clues. The paper therefore concludes that even the best performing models generate literal clues about 60% of the time in Visual Allusions. When explicit common ground is introduced through shared short stories, stronger models can achieve a 30%–50% reduction in overly literal clues, but they remain weak at inferring common ground when it is not explicitly stated (Ahuja et al., 7 Apr 2026).
A position paper on LLM interaction recasts this family of failures through a perceptual metaphor. It argues that LLM hallucinations should be understood less as mind-like deceit than as phenomena analogous to visual interpolation and illusion: systems build coherent surfaces from incomplete sampling. The Penrose Triangle is used as the key example of something locally consistent but globally inconsistent, and the paper proposes “saccade map[s] of abstract concept space,” saliency-like displays of what information the model is relying on, and artifact-grounded shared representations as alternatives to treating fluent output as evidence of stable understanding (Robinson et al., 14 Apr 2025). This suggests that visual allusion, in machine systems as well as in art and media history, is inseparable from selective attention, partial grounding, and the reconstruction of wholes from fragments.
Taken together, these lines of work place visual allusions at the intersection of art history, perceptual theory, imaging systems, multimodal evaluation, and pragmatic communication. In historical illusion practices, they arise from the merger of image and site; in projection mapping, from real-time registration between imagery and object; in psychophysical and computational settings, from the gap between actual and apparent features; in hidden-content images, from scale-sensitive global organization; and in multimodal games, from the need to refer indirectly and selectively. The recurring issue is not simply deception, but controlled reconfiguration of what an image, object, or clue can be taken to be.