- The paper introduces a zero-shot framework combining cross-space denoising with SDF blending to fuse dual semantic prompts into a unified 3D mesh.
- It employs CLIP-guided orientation search and view-conditioned texture synthesis to ensure geometric coherence and prompt-specific detail from designated viewpoints.
- Experimental results demonstrate improved semantic recognizability, smoother boundary transitions, and a fast runtime of 3–5 minutes per object.
JanusMesh: Zero-Shot, Fast 3D Dual-Semantic Visual Illusion Generation
Problem Statement and Motivation
Generating 3D objects that present distinct, recognizable semantic interpretations from multiple, specified viewpoints—effectively creating "dual-view" or multi-view 3D visual illusions—poses significant challenges in geometric coherence, semantic recognizability, and computational efficiency. Conventional pipeline-based or optimization-driven approaches, such as Score Distillation Sampling (SDS) methods, are inefficient (requiring tens of minutes per object) and prone to artifacts such as geometric seams, semantic leakage, and over-saturated texturing. Naive compositional strategies like direct mesh concatenation introduce boundary discontinuities and fail to conceal unintended semantics at non-targeted viewpoints. The fundamental challenge is generating a single, watertight 3D mesh that exhibits distinct, prompt-driven semantic content from dedicated viewpoints, while appearing as abstract or nondescript geometry from others—without excessive per-instance optimization or task-specific model fine-tuning.
Methodology
JanusMesh introduces a training-free, zero-shot 3D visual illusion generation framework, decoupled into two primary stages: (1) dual-branch cross-space geometric denoising and (2) view-conditioned texture synthesis.
Cross-Space Dual-Branch Denoising
Leveraging sparse latent representations from TRELLIS [73], JanusMesh introduces parallel denoising branches for each target prompt, both initiated from shared latent noise. At each denoising step, latents are decoded into voxel space, rotated for reference-frame alignment (using a CLIP-guided orientation search), and SDF-blended to form an averaged geometry. The element-wise average of the signed distance fields (SDFs) from both prompts, followed by binarization, reliably yields an intermediate volumetric shape that is geometrically consistent and naturally interpolates between objects. The result is re-encoded to latent space, enabling iterative refining of a single volumetric mesh fusing target semantics at target viewpoints, yet suppressing semantic leakage at others.
Noise guidance strategies—either by blending initial latents from pre-generated single-semantic objects ("Noise Blending") or interpolating between guidance latents and noise over early denoising steps ("Space Control")—are deployed to resolve silhouette conflicts and control the semantic ambiguity of the fusion, especially for objects with highly disparate canonical orientations or structures.
View-Conditioned Texture Synthesis
Given a fused mesh, JanusMesh performs surface texturing in an independent stage. Rather than texturing with a single prompt or merging textures naively, it uses depth-conditioned diffusion (via ControlNet-driven Stable Diffusion [63,80]) to generate clean images per prompt/viewpoint. These are projected and aggregated onto the mesh, selecting contributions per angular sector based on the current view, with cosine-weighted blending for seam minimization. Unlike prior methods (e.g., [4,11,51,78]), textures are prompt- and view-specific, maximizing both local realism and semantic fidelity at target viewpoints.
CLIP-Guided Orientation Search
Canonical poses for 3D objects can differ greatly depending on the prompt (e.g., birds in flight vs. upright fruit), so JanusMesh introduces an alignment routine leveraging CLIP [61]: For the first prompt, the best anchor view is automatically selected by maximizing CLIP text-image similarity across orthogonal renders. For the second, exhaustive search over discrete 3D rotations (sampled at 90° intervals on all axes) maximizes image-image similarity with the anchor view, aligning silhouettes for optimal SDF blending. This is critical for producing distinct silhouettes and avoiding collapsed or blended geometric artifacts.
Extension to Three-Object Illusions
The dual-branch methodology extends to three-object illusions by introducing a third denoising branch, fixing three target angles at 0°, 120°, and 240°. SDF blending becomes more challenging due to object conflicts; accordingly, Space Control guidance is intensified. This extension is handled without architectural modifications, demonstrating pipeline versatility.
Experimental Evaluation
Baselines and Setup
Comparisons are made against representative SDS-based multi-view illusion generation approaches ("Shape From Semantic" [40]), direct mesh concatenation, state-of-the-art feed-forward 3D generation models (TRELLIS [73]), and part-aware text-to-3D models (DreamBeast [42]). Object prompts are sampled from five categories over 60 distinct objects. Metrics encompass:
- CLIP Similarity (prompt-image alignment per viewpoint)
- GPT-4.1-mini accuracy (semantic recognizability via LLM-based classification)
- FID and KID (visual realism)
- Object Detection (average detected object count, multi-object detection rate)
- View-conditional CLIP contrast (leakage of unintended semantics)
- Boundary seam score (surface curvature continuity across fusion boundaries)
A user study further corroborates perceptual recognizability and method preference.
Quantitative and Qualitative Results
JanusMesh achieves highly competitive, often superior, numerical results:
| Method |
GPT Accuracy (%) |
FID |
Multi-Obj. Rate |
Boundary Impact Factor |
Runtime |
| JanusMesh (Ours) |
84 |
185.6 |
18% |
0.994 |
3–5 min |
| Direct Concatenation |
76 |
187.9 |
56% |
1.129 |
3–5 min |
| Shape From Semantic [40] |
70 |
194.1 |
20% |
0.973 |
40 min |
| TRELLIS [73] |
58 |
174.1 |
22% |
0.952 |
2–3 min |
| DreamBeast [42] |
76 |
185.0 |
19% |
0.974 |
3–4 hr |
JanusMesh presents the highest semantic clarity, lowest multi-object detection (indicating tight geometric fusion), and best boundary smoothness among rapid-generation techniques. It eliminates the unnatural seams and semantic leakage of direct concatenation and avoids the color/geometry artifacts and inefficiency of optimization-driven baselines. Qualitative assessments consistently show artifact-free, dual-semantic meshes where each target viewpoint yields a clearly recognizable target object, with suppressed semantics elsewhere. User studies confirm 78.5% recognition rates and 71% overall preference.
Ablation studies validate each architectural decision: SDF-based blending outperforms alternative volumetric fusion strategies; noise guidance is crucial for challenging object pairs; view-conditioned texturing is necessary for semantic distinction; and CLIP-guided orientation is essential for non-canonically aligned prompt pairs.
Implications and Future Directions
JanusMesh demonstrates that zero-shot, dual-viewpoint 3D illusions are tractable within minutes, without dataset-specific supervision or per-shape optimization. Its modular architecture (cross-space latent denoising, SDF fusion, CLIP-based alignment, view-conditioned texture aggregation) is broadly applicable and scalable, enabling compositional content generation with controllable semantic disclosure. The technique provides new opportunities for design, digital art, and cognitive studies of perception, where interpretational ambiguity is engineered into 3D form.
The framework’s extension to three-object illusions suggests broader potential for multi-conditional 3D generative synthesis, though challenges in automated multi-object orientation alignment (especially as N increases) remain. Integration of more granular, learned fusion controls, denser orientation search, and leveraging future advances in 3D-aware diffusion architectures (such as panoptic scene generation or mesh manifold learning) could further augment flexibility and fidelity.
Conclusion
JanusMesh establishes a new state of the art for fast, zero-shot, dual- (and tri-) semantic 3D visual illusion generation, offering a robust and scalable alternative to optimization-based pipelines. Its integration of cross-space denoising, volumetric SDF blending, CLIP-guided alignment, and view-conditional texture transfer achieves significant advances in geometric coherence, semantic recognizability, and runtime efficiency. The approach provides a practical and extensible tool for prompt-driven, multi-view 3D illusion creation and invites further exploration into compositional 3D generative models and controllable semantic fusion techniques (2606.20563).