Cosmos 3: Extragalactic Survey & Physical AI

• Cosmos 3 is a dual-faceted resource combining a deep VLA-COSMOS 3 GHz survey with an advanced omnimodal Physical AI world model.
• The survey employs multi-frequency synthesis and robust cross-matching to yield precise radio catalogs and effective AGN classifications.
• The AI component uses a mixture-of-transformers decoder to integrate language, vision, audio, and action data for enhanced physical reasoning.

Cosmos 3 refers to both the VLA-COSMOS 3 GHz Large Project—an ultra-deep, large-area radio continuum survey of the COSMOS field—and a pivotal omnimodal world modeling family for Physical AI. The term thus encapsulates transformative resources in extragalactic survey science and advanced AI modeling. The sections below delineate the survey's methodology, scientific results, AI model architecture, and their respective impacts on the understanding of galaxy evolution, AGN, star formation, and embodied intelligence.

1. Survey and Observational Framework

The VLA-COSMOS 3 GHz Large Project deployed the Karl G. Jansky Very Large Array in A+C configurations to mosaic the central 2 deg² of the COSMOS field at S-band (centered at 3 GHz, bandwidth 2 GHz) with a uniform beam of 0.75″ FWHM and median sensitivity of 2.3 μJy beam⁻¹ (Smolcic et al., 2017). The data comprise 192 pointings, generating a catalog of 10 830 radio sources down to a 5σ threshold of ~11.5 μJy, with astrometric precision σ ≲0.01″ for high S/N sources.

Imaging leveraged multi-scale, multi-frequency synthesis (MS-MFS) to reconstruct both monochromatic intensity and in-band spectral index over the 2 GHz range. Source extraction utilized BLOBCAT with robust completeness corrections from extensive Monte Carlo injection tests.

The survey's multi-wavelength integration was accomplished by cross-matching radio positions with the COSMOS2015 optical/NIR catalog, yielding 7 729 secure counterparts over 1.77 deg² and >35% spectroscopic redshift coverage (Delvecchio et al., 2017). This dataset is the deepest, largest-area census of radio sources to date at sub-arcsecond resolution.

2. Galaxy and AGN Classification and Host Properties

A multi-faceted approach classified sources as star-forming, high-luminosity AGN (HLAGN, radiatively efficient), or low/moderate-luminosity AGN (MLAGN, mechanically dominated) (Delvecchio et al., 2017):

• HLAGN were identified by SED decomposition (magphys + sed3fit, including AGN tori), X-ray luminosity L_X > 10⁴² erg s⁻¹, and MIR color diagnostics. The SED-AGN union comprises 21% of the sample.
• MLAGN are radio-excess sources, identified via a >3σ offset in log(L_1.4 GHz/SFR_IR) relative to the SF locus, capturing systems with significant mechanical AGN output but weak radiative signatures.

Key trends:

• HLAGN predominantly occupy the star-forming main sequence (MS) at z<1.5; MLAGN hosts are more massive (median log M_★ ≈ 11.1 M_⊙) and exhibit lower SFRs, often 2–3× below the MS.
• At z>1.5, HLAGN are found in higher-mass systems (median log M_★ ≈ 11.1) than MLAGN, supporting "downsizing" scenarios: massive galaxies trigger high-efficiency AGN earlier, then transition to jet-dominated low-Eddington modes at lower redshift.
• Median SFRs and stellar masses in HLAGN/MLAGN show statistically significant (4–10σ) differences up to z~2, weakening at higher z due to sample size (Delvecchio et al., 2017).

These demographics reinforce the distinction between quasar-mode (HLAGN) and radio/jet-mode (MLAGN) AGN feedback, with implications for galaxy quenching and SMBH coevolution.

3. Star Formation History and Luminosity Functions

The 3 GHz continuum provides a dust-unbiased probe of star formation out to z ~ 5 (Novak et al., 2017). Using a composite SED-fitting approach and survival analysis to treat upper/lower limits, rest-frame L_1.4 GHz luminosities were computed assuming S_ν ∝ ν^α (median α=–0.7).

The radio luminosity function (LF) for star-forming galaxies is modeled with a modified Schechter function:

$$\Phi(L,z) = \Phi^*(L/L^*)^{1-\alpha} \exp\left[-\frac{1}{2\sigma^2}\log^2(1 + L/L^*)\right]$$

with evolution

$L_*(z) = L_0 (1+z)^{3.16 - 0.32z}$

Integrated SFRD shows a rapid rise from z=0 to z~2–3, peaking at SFRD ≃ 0.1 M_⊙ yr⁻¹ Mpc⁻³ and declining smoothly to z ~ 5. Ultra-luminous IR galaxies (ULIRGs, SFR=100–1000 M_⊙ yr⁻¹) contribute ~25% to SFRD at z~2.5; hyper-luminous systems (HyLIRGs) remain a minor (≲2%) fraction throughout. Comparison with rest-frame UV SFRDs reveals a 15–20% deficit at z > 4, suggesting significant SF in heavily dust-enshrouded galaxies (Novak et al., 2017).

The A³COSMOS ALMA program (Traina et al., 2023) complements this picture at sub-mm/mm, extending IR LFs and obscured SFRD constraints to z ~ 6. The IR LF knee L* rises from 10¹¹ L_⊙ at z~1 to >10¹² L_⊙ at z~5, while Φ* decreases. Integrated SFRD peaks at z ~ 1–3, confirming that ~80% of cosmic SFRD is dust-obscured at this epoch.

4. Active Galactic Nucleus (AGN) Spectral, Morphological, and Environmental Properties

The radio-excess AGN ("RxAGN") sample (Tisanić et al., 2020) spans 0<z<4 with log L_1.4GHz ∈ [24,26]. The average AGN radio spectral energy distribution (SED) deviates significantly from a simple power law:

• Fitted with a broken power law:
  • α₁ = 0.28 ± 0.03 (ν < 4.1 GHz)
  • α₂ = 1.16 ± 0.04 (ν > 4.1 GHz)
  • Δα = 0.88, exceeding the canonical value for synchrotron aging (0.5), indicating complex energy-loss and absorption processes, including synchrotron self-absorption.

Spectral indices increase with both source size and redshift, implying that spectral curvature must be accounted for in next-generation radio survey simulations (e.g., SKADS, T-RECS, SKA) to avoid systematic biases in luminosity and SFR inference (Tisanić et al., 2020).

Morphological analysis at 0.75″ (VLA 3 GHz) (1901.10168, Vardoulaki et al., 2020) identifies 67 multi-component sources: 58 AGN (head-tail, core-lobe, WAT, X/Z-shaped) and 9 SFGs. Multi-component AGN lie predominantly in massive, green-valley or quenched hosts (M_★ > 10¹⁰.⁵ M_⊙). At μJy levels and z~1, FRI/FRII/COM AGN distinctions blur in host mass, luminosity, and environment, challenging canonical jet-mode categorization.

Environmental studies using galaxy density/cluster catalogs show no strong spatial clustering difference among FR classes, though FRIIs as satellites in clusters may experience accelerated quenching, likely via enhanced jet-mode feedback (Vardoulaki et al., 2020).

5. Omnimodal World Model: Cosmos 3 for Physical AI

Cosmos 3 also denotes a family of unified, omnimodal world models for Physical AI, capable of ingesting and generating language, image, video, audio, and action sequences in a single mixture-of-transformers (MoT) decoder architecture (Aditi et al., 1 Jun 2026).

Architecture:

• Mixture-of-Transformers Decoder: Parallel Reasoner (autoregressive, causal mask) and Generator (bidirectional, diffusion) towers.
• Tokenization: Language (WordPiece/BPE), Vision (ViT, VAE), Audio (hop-based VAE), Actions (parameterized 6D rotation + 3D translation, grasp states).

Objectives and Training:

• Reasoner: next-token cross-entropy on discrete modalities.
• Generator: rectified-flow matching (velocity MSE w.r.t. denoising continuous latents).
• Multi-stage data curation: 767M images, 348M videos for pretraining; high-fidelity SDG-generated datasets for Physical AI scenarios.

Evaluation:

• Cosmos 3 sets open-source SoTA in text-to-image (Artificial Analysis), image-to-video, and robot policy (RoboArena A/B ranking).
• Supports physical reasoning, audio-visual grounding, and embodied action prediction with high closed-loop consistency (video-action PSNR, RTE, RRE metrics).
• Limitations persist for high-fidelity audio, long-horizon consistency, and fine-grained sim-to-real transfer.

Applications include synthetic data generation, unified perception/action/simulation backbones, and accelerated embodied agent research.

6. Data Products, Tools, and Community Access

VLA-COSMOS 3 GHz data products (Smolcic et al., 2017):

• Full source catalog (positions, fluxes, morphological flags) to 10 830 entries.
• 0.75″/2.3 μJy beam⁻¹ mosaic FITS images with uniform depth and robust astrometry.
• Public release at https://cosmos.astro.caltech.edu.

A³COSMOS (Traina et al., 2023):

• ALMA-imaged sub-mm/mm sources with uniform blind statistics, access to IR LFs, SFRD, and deep matched multi-wavelength catalogs.

Cosmos 3 world model resources (Aditi et al., 1 Jun 2026):

• Code and models under OpenMDW-1.1 License at GitHub and HuggingFace.

Survey methodologies, completeness corrections, AGN/SF separation procedures, and cross-matched catalogs ensure replicability and comparability for next-generation extragalactic, AGN, and physical AI research.

7. Impact, Limitations, and Future Directions

Cosmos 3 empirical results strongly shaped the understanding of galaxy evolution across cosmic time, establishing robust measurements of star formation, AGN feedback, and environmental trends to z ~ 6. Its modeling framework advanced multimodal generative and predictive tasks foundational for embodied AI.

Outstanding challenges:

• For surveys: sim-to-real astrophysical modeling, further cross-modal population studies, and enhanced environmental metrics.
• For AI models: high-fidelity audio synthesis, long-horizon video and action consistency, and physics extrapolation beyond training regimes.

Together, the Cosmos 3 suite provides a definitive reference for statistical cosmology, AGN/galaxy co-evolution, and a scalable AI backbone for complex, real-world and simulated environments (Smolcic et al., 2017, Novak et al., 2017, Delvecchio et al., 2017, Cooke et al., 2019, 1901.10168, Tisanić et al., 2020, Vardoulaki et al., 2020, Traina et al., 2023, Aditi et al., 1 Jun 2026).