Holo-World: Holistic Cross-Domain Models

• Holo-World is a holistic concept that integrates full-scale world representations—from 3D spatial memory to holographic reconstructions—across multiple disciplines.
• Systems under this notion employ diverse methodologies, including dual-stream Transformer encoders, diffractive computation, and scene graph optimizers to fuse multimodal data.
• The frameworks enable practical applications from total-body medical imaging and real-time telepresence to embodied AI navigation and theoretical quantum gravity models.

Searching arXiv for the main "Holo-World" references used in the article. Searching arXiv for "Holo-World" and closely related titles. Using arXiv search to verify relevant papers and ids. Holo-World is a cross-domain research notion for systems that seek a coherent representation of an entire world, body, scene, or spacetime region rather than a narrowly localized view or task. In current arXiv usage, the term spans multimodal total-body medical foundation models, holographic telepresence and diffractive display pipelines, editable 3D world generators, simulation-ready digital twins, embodied-agent frameworks with persistent 3D spatial memory, and generalized holographic reconstructions in gravity (Chen et al., 19 Jan 2026, Córdova-Esparza et al., 2018, Bian et al., 7 Aug 2025, Zhou et al., 22 Jun 2026, Bousso et al., 2023).

1. Scope and terminological range

The literature uses “Holo-World” in several technically distinct but structurally related ways. The common thread is holism: each system attempts to preserve a larger state space than a task-specific, local, or single-view model would ordinarily retain. In some cases the emphasis is perceptual and multimodal; in others it is optical, generative, embodied, or theoretical.

Domain	Representative formulation	Technical emphasis
Total-body medical AI	SDF-HOLO	Whole-body multimodal world model
Optical telepresence and display	Simulated holographic display, HoloTile RGB, diffractive video call	Volumetric capture, multi-view rendering, CGH
3D world generation	HOLODECK 2.0, HoloScene	Editable scenes, scene graphs, simulation-ready assets
Embodied AI	HoloAgent-0	3D spatial memory and closed-loop execution
Quantum gravity	Generalized entanglement wedges, small-world holography	Holographic encoding of bounded regions

A common misconception is to equate Holo-World exclusively with optical holography. The cited work is broader. Some systems are physically optical; some are world models in the machine-learning sense; some are object-centric digital twins; some are explicit 3D spatial memories; and some use “hologram” in the AdS/CFT sense of region reconstruction (Córdova-Esparza et al., 2018, Xia et al., 7 Oct 2025, Zhou et al., 22 Jun 2026, Bousso et al., 2023). A separate terminological caution appears in structure-based drug discovery, where “holo” denotes a ligand-bound protein state rather than a holographic world representation; AANet and FABFlex explicitly define holo structures as ligand-bound reference states and study blind apo/predicted or apo-to-holo settings (Zhu et al., 6 Jun 2025, Zhang et al., 20 Feb 2025).

2. Holistic internal world models in multimodal perception

A strong internal-model use of the term appears in SDF-HOLO, which is described as essentially a “holo-world” model for total-body PET/CT: a single generalist system that builds a coherent internal representation of nearly the entire human body, its anatomy in CT and its metabolism in PET, and uses that representation for segmentation, detection, report generation, and system-wide metabolic profiling (Chen et al., 19 Jan 2026). The model is pre-trained on 10,460 total-body PET/CT and report pairs from four centers, partitions the body into $R = 6$ anatomically meaningful axial regions, tokenizes each region into $16 \times 16 \times 16$ voxel patches, and uses synchronized 90% masking for CT and PET so that both modalities learn from corresponding visible tokens.

Its architecture separates and then reunifies modalities. CT and PET are processed by dual-stream EvaMAE-style 3D Transformer encoders; a Cross-Modal Interaction Module is inserted at selected depths, specifically at $1/3$ and $2/3$ of the encoder; and a Global Anatomical Aggregation Transformer operates over region-ordered tokens so that head/neck, thorax, abdomen, pelvis, and limbs can attend to one another. The semantic bridge to language is explicit rather than weakly aligned: masks from TotalSegmentator and CADS are fused into 180 non-overlapping anatomical classes, and voxel–mask–text alignment is imposed with an InfoNCE anchor loss. The total pre-training objective combines masked image modeling, language modeling, global anatomical classification, and anchor alignment, with $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$ (Chen et al., 19 Jan 2026).

Downstream results make clear that the model is intended as a shared world representation rather than a single-task network. On AutoPET tumor segmentation, SDF-HOLO reaches DSC $= 0.6328$, FNV $= 2.2745$, FPV $= 3.0544$ on Imu, and DSC $= 0.6933$, FNV $= 0.6447$, FPV $16 \times 16 \times 16$0 on UKT. For Chinese report generation, the same encoder backbone with a BERT-initialized LSTM plus attention attains BLEU-1 $16 \times 16 \times 16$1, BLEU-4 $16 \times 16 \times 16$2, METEOR $16 \times 16 \times 16$3, ROUGE-L $16 \times 16 \times 16$4, and CIDEr $16 \times 16 \times 16$5, all substantially above the cited baselines. The same representation is also used for system-wide metabolic profiling in a healthy cohort of 181 subjects, with 16-dimensional organ embeddings used to study age-associated covariance changes, inter-organ metabolic networks, and system-level declines such as a neural decrease of $16 \times 16 \times 16$6 from young to old (Chen et al., 19 Jan 2026).

This use of Holo-World is notable because it is not about external display. It is about constructing a latent “holo-map” of a patient in which organ embeddings, anatomical relations, metabolic saliency, and report semantics are jointly encoded. A plausible implication is that, in this strand of research, Holo-World denotes a foundation-model strategy for preserving global state and long-range dependencies that localized medical models systematically discard.

3. Optical Holo-World systems: capture, projection, and diffractive computation

In telepresence and display research, Holo-World refers more literally to a 3D optical world that can be captured, reconstructed, and viewed. One representative implementation uses 4 Kinect V2 sensors, each with RGB $16 \times 16 \times 16$7 and depth $16 \times 16 \times 16$8, arranged around a subject at roughly $16 \times 16 \times 16$9 intervals and about 2 m height. After one-off intrinsic and extrinsic calibration using a 60 cm 1D wand, background subtraction is performed with an encoder–decoder CNN, the four foreground depth streams are fused into a global colored point cloud, and four rendered views are projected into an inverted acrylic quadrangular pyramid, yielding a 360° simulated hologram without head-mounted displays. The system runs in MATLAB 2017b at about 10 FPS, with mean reprojection errors of about 0.13 px for depth cameras and 0.46 px for RGB cameras, and a cardboard-box reconstruction error of about 3 mm in width and 2 mm in height (Córdova-Esparza et al., 2018).

A different line emphasizes actual diffractive computation. HoloTile RGB implements sub-hologram tiling and PSF shaping on a phase-only HoloEye GAEA 2.1 SLM. Output pixel spacing is controlled by $1/3$0, with wavelength-specific sub-hologram sizes chosen so that RGB channels share the same output pixel size. Each wavelength is optimized by stochastic gradient descent, displayed sequentially in Color Field Sequential mode, and reconstructed in a single shot. The system reports full 8-bit phase modulation at 60 Hz for each wavelength, uses $1/3$1, $1/3$2, and $1/3$3, and claims more than $1/3$4 speed improvement over conventional full-frame CGH while producing high-fidelity, pseudo-digital multi-wavelength images without temporal averaging (Madsen et al., 2024).

A practical end-to-end diffractive telepresence pipeline pushes the same agenda toward live communication. A 3CPT system captures RGBZ on an iPhone, packs color and depth into a vertically stacked superframe, compresses it with H.264 via FFmpeg, tunnels it over TCP via ngrok, reconstructs RGB plus disparity-in-diopters on a receiver, and generates real-time phase holograms for a $1/3$5 phase-only SLM illuminated by RGB lasers. The capture resolution is $1/3$6, each raw superframe occupies $1/3$7 bytes, and the reported end-to-end delay is subsecond (Samanta et al., 2 Feb 2025).

Portable “holographic overlays” occupy an intermediate position between pure AR and free-space holography. On an iPhone 14 Pro, LiDAR and TrueDepth are fused, upscaled with SRCNN, projected into a world frame with IMU pose, and rendered through Metal shaders; the LiDAR-plus-SRCNN pipeline runs at about 50 FPS, in contrast to about 2 FPS for MiDaS and about 0.25 FPS for Marigold at 10 denoising steps on the same device (Agrawal, 2024). At the opposite end of the design spectrum, Holodeck-style flying-light-speck proposals replace screens and wavefront modulators with room-scale swarms of luminous micro-UAVs; at a 0.264 mm pitch estimate, a room-size sphere with $1/3$8 mm would require about 110,554,359 FLSs, underscoring the scale of the engineering problem (Ghandeharizadeh, 2021). Survey literature on holographic projection situates such systems within a broader ecology of telepresence, business, education, and healthcare applications, ranging from Pepper’s Ghost stage systems to holovision televisions and holographic cellphones (Elmorshidy, 2010).

4. Generated, editable, and simulation-ready 3D worlds

A second major strand treats Holo-World as a generated or reconstructed virtual environment. HOLODECK 2.0 addresses open-domain 3D world generation from language with editing. Its pipeline has four modules: Scene Analysis, Object Generation, Scene Generation, and Scene Editing. GPT-Image-1 produces a reference scene image, isolated object images, and a tileable ground texture; GPT-o3 parses a structured JSON object list and generates spatial constraints; Hunyuan3D 2.1 converts object images into GLB meshes with PBR materials; and a DFS-based layout solver enforces relations such as left of, right of, near, on, above, and face to. On human evaluation, the method improves indoor asset-level and scene-level scores from 4.28 and 3.67 to 8.00 and 7.71, and open-domain scores from 3.77 and 3.06 to 7.58 and 7.32. CLIP-based scores rise from 0.253 to 0.299 indoors and from 0.256 to 0.307 in open-domain scenes. Layout optimization is preferred to initial VLM layout in 67.93% of comparisons, and editing is preferred to BlenderAlchemy in 71.12% of comparisons (Bian et al., 7 Aug 2025).

HoloScene starts instead from a single monocular video of a static indoor scene and optimizes an interactive scene graph whose node state is $1/3$9: a neural SDF for geometry, Gaussians-on-Mesh for appearance, rigid-body physical properties, and object pose. Reconstruction minimizes a unified energy,

$2/3$0

where the terms encode observation fidelity, amodal completion from generative priors, non-penetration, and static stability under gravity. Optimization proceeds in three stages: gradient-based visible-region reconstruction, sampling-based completion with tree-structured search over support relations, and gradient-based appearance refinement. On Replica, HoloScene reports Stable(All) $2/3$1, compared with 39.4% for ObjectSDF++, 5.6% for PhyRecon, and 8.5% for DP-Recon; it also achieves OR $2/3$2, indicating full object reconstruction in the reported setting (Xia et al., 7 Oct 2025).

Dynamic video world models occupy a related but distinct position. Holo-World for video generation is a first-frame-anchored source-to-state model that starts from a single image $2/3$3, accepts world controls $2/3$4, uses $2/3$5 as a source appearance anchor, and generates an 81-frame video that either preserves the original world or transfers it to a target weather state. HoloStateData contains 15,066 training samples, split into 7571 Real, 3541 Simulation, and 3954 V2V records. The model adds a Unified Scene Adapter that separates world-preservation and weather-transfer residuals into disjoint parameter subspaces, and applies Scene-Weather Decomposed CFG,

$2/3$6

so that weather can be intensified without over-amplifying the entire condition. On the Real subset it achieves the best reported RotErr, TransErr, and ObjMC; on the Weather subset it reports Weather Alignment $2/3$7 and VLM Evaluation $2/3$8, outperforming the cited video-to-video weather-editing baselines despite using only the first frame as real visual evidence (Yin et al., 18 Jun 2026).

Taken together, these systems define a continuum: text-to-scene assembly, single-video digital-twin reconstruction, and first-frame-conditioned dynamic world simulation. This suggests that “Holo-World” increasingly denotes not a single representation class, but an object-centric and controllable world stack spanning assets, scene graphs, geometry buffers, and state-conditioned video generation.

5. Persistent Holo-Worlds for embodied agents

In embodied AI, Holo-World appears as an explicit, persistent 3D memory shared by planning and control. HoloAgent-0 organizes deployment into three coupled layers: Embodied AgentOS for closed-loop execution, 3D spatial memory for world grounding, and embodied skills for action. AgentOS converts natural-language instructions into executable skill graphs, schedules robot resources, monitors execution, and triggers clarification or re-planning from runtime feedback. The memory layer stores metric geometry, traversability, an open-vocabulary semantic map, and a hierarchical multimodal scene graph with floor, room, view, and object nodes. A fused semantic descriptor is built as

$2/3$9

combining SigLIP features of the full keyframe, the masked segment, and the bounding box around the segment. Persistent 3D instances are maintained by projecting prior instances into the current view and associating them to new SAM2 masks by IoU (Zhou et al., 22 Jun 2026).

The framework exposes navigation, manipulation, motion, perception, and interaction as typed skills with command schemas and status streams. Navigation uses HMSG retrieval and open-vocabulary semantic maps to localize candidate goal rooms and views, then verifies them with a VLM; manipulation uses HoloBrain as the skill layer for local actions; and HoloMotion handles whole-body motion generation and control. Because memory is shared, cross-robot coordination is possible within the same world model, and dynamic updates revise only the affected subgraph when geometry or semantics change. Quantitatively, HoloAgent-Nav reaches SR $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$0 and SPL $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$1 on HM3D-ObjNav, and real-robot navigation reports 97.7% success at Top-1 across 1.0 m, 2.0 m, and 3.0 m thresholds. On ScanNet semantic mapping, HoloAgent-Memory reports $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$2, $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$3, and $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$4 (Zhou et al., 22 Jun 2026).

This version of Holo-World is neither a display nor a generator. It is an operational world substrate: a queryable, continuously updated, robot-readable environment that supports long-horizon execution. A plausible implication is that, in embodied systems, “holo” primarily signals persistence and cross-module coherence rather than optical holography.

6. Holographic worlds in quantum gravity

A more abstract but historically central use of the concept appears in quantum gravity. “Holograms in Our World” generalizes the entanglement wedge beyond AdS boundaries to arbitrary gravitating regions $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$5. It defines a max-entanglement wedge $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$6 and a min-entanglement wedge $$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$7, with

$$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$8

The generalized entropy of a wedge is

$$(\lambda_1,\lambda_2,\lambda_3,\lambda_4) = (1.0, 1.0, 0.1, 0.5)$$9

and the proposal interprets $= 0.6328$0 as the largest region whose information can flow inward to $= 0.6328$1, while information outside $= 0.6328$2 can flow outward. The generalized entropies of appropriate wedge combinations satisfy strong subadditivity, and suitably independent wedges obey a no-cloning relation. This suggests that holography can be treated as a local or quasi-local property of arbitrary gravitating regions rather than only as a boundary duality in AdS/CFT (Bousso et al., 2023).

“Holography for a small world” addresses a different question: which CFT states are sufficient to describe physics inside a bounded bulk region in AdS. It shows that UV/IR, understood as a spatial lattice cutoff on the boundary CFT, fails for explicit states localized deep in the bulk yet encoded on short scales in the CFT. The paper then proposes that the relevant small-world hologram is a low-energy subspace of the CFT: states with energy below $= 0.6328$3, where $= 0.6328$4 is the mass of the AdS black hole whose horizon radius matches the chosen bulk sphere. In that formulation, the finite hologram is not a coarse spatial discretization but a finite-energy code subspace whose number of states is of order $= 0.6328$5 (Rosenhaus, 2013).

These theoretical works are conceptually distant from PET/CT, telepresence, or simulation-ready scene graphs, yet they sharpen the same core issue: what counts as the minimal, sufficient representation of a world bounded by finite resources. In that sense, the theoretical literature supplies a limiting case of Holo-World thinking: the world is not merely rendered or simulated, but encoded.