Flamingo in Cosmology & Multimodal AI

• Flamingo is a dual-use research platform with two distinct branches: one for full-physics cosmological simulations and another for autoregressive multimodal learning.
• The cosmology branch utilizes high-resolution hydrodynamics and calibrated feedback techniques to model large-scale structure, weak lensing, and X-ray emissions.
• The multimodal branch employs frozen visual and language encoders with gated cross-attention, enabling seamless integration of images, videos, and text for in-context few-shot tasks.

FLAMINGO is a research name used in two technically unrelated but individually prominent lineages. In cosmology and astrophysics, FLAMINGO denotes “Full-hydro Large-scale structure simulations with All-sky Mapping for the INterpretation of next Generation Observations,” a suite of large-volume cosmological hydrodynamical simulations built for large-scale structure, cluster cosmology, and baryonic-systematics modeling (Braspenning et al., 2023). In multimodal machine learning, Flamingo denotes a family of autoregressive visual-LLMs designed for in-context few-shot learning from arbitrarily interleaved image, video, and text sequences (Alayrac et al., 2022). The shared name therefore spans two distinct research programs: one centered on baryons, feedback, and survey-scale mock universes, and the other on cross-attentive multimodal generation.

1. Scope and nomenclature

In the cosmological literature, FLAMINGO is a simulation program built around full-physics hydrodynamics, massive neutrinos, feedback calibration, and all-sky mapping products for weak lensing, Sunyaev-Zel’dovich observables, X-ray emission, and related probes (Kugel et al., 2023). Its flagship intermediate-resolution run has a $(2.8\,\mathrm{Gpc})^3$ comoving volume, with $m_{\rm gas}=1.07\times10^9\,M_\odot$ and $m_{\rm CDM}=5.65\times10^9\,M_\odot$; at $z=0$ in the fiducial cosmology it contains $\sim2.0\times10^6$ haloes with $M_{500\mathrm c}>10^{13}\,M_\odot$ and $\sim1.1\times10^5$ haloes with $M_{500\mathrm c}>10^{14}\,M_\odot$ (Braspenning et al., 2023). The suite was designed explicitly for large-scale structure cosmology, cluster gas physics, and forward modeling of next-generation survey observables.

In the multimodal-model literature, Flamingo is a visual-language architecture that accepts sequences of text tokens interleaved with images and videos, and models

$$p(y\mid x)=\prod_{\ell=1}^{L} p(y_\ell\mid y_{<\ell},x_{\le \ell}),$$

so that task behavior can be induced purely by prompting rather than parameter updates (Alayrac et al., 2022). It was introduced as a few-shot multimodal learner and later replicated in open-source form and extended to audio and music domains (Awadalla et al., 2023).

A plausible implication is that “FLAMINGO” now functions less as a single topic than as a stable label for two independent but influential infrastructures: one for simulation-based cosmology, one for cross-modal generative modeling.

2. FLAMINGO as a cosmological hydrodynamical simulation suite

The FLAMINGO simulation program is built with the SWIFT code, SPHENIX hydrodynamics, and massive neutrinos treated with the $\delta f$ method (Braspenning et al., 2023). Initial conditions are generated with MONOFONIC using higher-order baryon–CDM initial conditions and massive neutrinos, while halo identification and spherical-overdensity properties are derived with VELOCIraptor and SOAP (Braspenning et al., 2023). The fiducial cosmology adopts the DES Y3 “3×2pt + all external” parameter set: $m_{\rm gas}=1.07\times10^9\,M_\odot$0 with $m_{\rm gas}=1.07\times10^9\,M_\odot$1 eV (Braspenning et al., 2023).

Its subgrid model includes element-by-element radiative cooling and photo-heating, pressure-based star formation, time-delayed stellar mass loss and metal enrichment, stellar feedback, black-hole growth, and either thermally driven or kinetic jet AGN feedback (Braspenning et al., 2023). A central design feature is calibration by Gaussian-process emulators trained on Latin hypercubes of 32 smaller-volume simulations, allowing feedback parameters to be tuned against the $m_{\rm gas}=1.07\times10^9\,M_\odot$2 galaxy stellar mass function and group/cluster gas fractions rather than by manual adjustment (Kugel et al., 2023). This calibration framework also defines controlled model families such as fgas±Nσ, M*−σ, and jet-mode variants, each corresponding to shifted target observables rather than arbitrary parameter changes (Kugel et al., 2023).

The suite was built to enable both physical realism and survey relevance. Large-volume hydrodynamics gives access to rare massive haloes and two-halo-scale correlations, while the calibration strategy links nuisance freedom to directly interpretable observables such as $m_{\rm gas}=1.07\times10^9\,M_\odot$3 and the stellar mass function (Kugel et al., 2023). This makes FLAMINGO particularly suitable for analyses in which baryonic uncertainty must be propagated into power spectra, lensing, SZ, or X-ray observables.

3. Cluster gas physics, scaling relations, and baryonic response

A major early application was the comparison of FLAMINGO clusters to X-ray data. Clusters and groups are selected above $m_{\rm gas}=1.07\times10^9\,M_\odot$4, with thermodynamic profiles measured in 3D shells and analyzed using volume-, mass-, and X-ray-weighted estimators (Braspenning et al., 2023). The fiducial FLAMINGO model reproduces observed temperature, density, pressure, and entropy profiles well, and matches X-ray and SZ scaling relations once hydrostatic mass bias is accounted for; the principal discrepancy is that core iron metallicities are too high by about $m_{\rm gas}=1.07\times10^9\,M_\odot$5 dex (Braspenning et al., 2023). The simulations show very little evolution away from self-similar expectations in $m_{\rm gas}=1.07\times10^9\,M_\odot$6, $m_{\rm gas}=1.07\times10^9\,M_\odot$7, $m_{\rm gas}=1.07\times10^9\,M_\odot$8, and common scaling relations, whereas metallicity decreases with redshift (Braspenning et al., 2023).

Feedback mode strongly affects cluster cores. Relative to thermal AGN feedback calibrated to the same gas fraction at $m_{\rm gas}=1.07\times10^9\,M_\odot$9, kinetic jet feedback produces hotter, higher-entropy, lower-density cores, lower central metallicities, and a cool-core fraction lower by more than a factor of two (Braspenning et al., 2023). Lower global gas fractions in thermal models likewise reduce core and intermediate-radius densities, raise temperatures and entropies at $m_{\rm CDM}=5.65\times10^9\,M_\odot$0, lower pressure, and modestly alter metallicity profiles (Braspenning et al., 2023).

The same feedback variations leave a clear imprint on non-linear clustering. FLAMINGO’s baryon-response analysis defines

$m_{\rm CDM}=5.65\times10^9\,M_\odot$1

and shows that the response is well converged only for simulation volumes in excess of $m_{\rm CDM}=5.65\times10^9\,M_\odot$2, while a Gaussian-process emulator reproduces the FLAMINGO responses to better than one per cent up to $m_{\rm CDM}=5.65\times10^9\,M_\odot$3 and $m_{\rm CDM}=5.65\times10^9\,M_\odot$4 across the feedback grid (Schaller et al., 2024). Lower gas fractions produce stronger suppression of the total matter power spectrum, the use of collimated jets enhances the effect, and the tested dependence on background cosmology is small, at the $m_{\rm CDM}=5.65\times10^9\,M_\odot$5 level for $m_{\rm CDM}=5.65\times10^9\,M_\odot$6 (Schaller et al., 2024). In weak-lensing scattering-transform analyses, the same baryonic physics suppresses scattering coefficients by up to $m_{\rm CDM}=5.65\times10^9\,M_\odot$7 on scales corresponding to $m_{\rm CDM}=5.65\times10^9\,M_\odot$8, although shape noise and $m_{\rm CDM}=5.65\times10^9\,M_\odot$9 arcmin smoothing reduce the effective suppression to $z=0$0 under Euclid-like conditions (Marinichenko et al., 10 Oct 2025).

These results do not imply that baryons alone solve the late-time clustering problem. A dedicated FLAMINGO study of the $z=0$1 tension finds that baryonic effects are not sufficiently large to reconcile low-redshift cosmic shear, tSZ, and related cross-spectra with the CMB-specified standard model, even though those baryonic effects materially alter non-linear observables (McCarthy et al., 2023).

4. Extended observational programs and astrophysical applications

Beyond cluster thermodynamics, FLAMINGO has been used as a forward model for a wide range of baryon-sensitive observables, from kSZ stacking and diffuse X-ray cross-correlations to quasar statistics and high-redshift galaxy abundances.

Application	Observable or sample	Representative result
Thermal history from tSZ–LSS	$z=0$2, $z=0$3	$z=0$4 scales roughly as $z=0$5 with $z=0$6 over $z=0$7, and current data prefer low $z=0$8 plus strong feedback (Salcido et al., 11 May 2026)
kSZ plus galaxy–galaxy lensing	BOSS galaxies with Planck+ACT kSZ stacks	kSZ measurements prefer stronger feedback than models calibrated to low-$z=0$9 X-ray gas fractions (McCarthy et al., 2024)
X-ray–cosmic-shear cross-correlation	DES shear × ROSAT diffuse X-ray maps	The signal is most sensitive to haloes with $\sim2.0\times10^6$0; unresolved AGN treatment changes whether fiducial or stronger-feedback models are preferred (McDonald et al., 2 Feb 2026)
Secondary CMB and foreground maps	$\sim2.0\times10^6$1, tSZ, kSZ, CIB, radio, screening	FLAMINGO produces self-consistent full-sky mock maps reproducing a wide range of observational constraints (Yang et al., 10 Dec 2025)
High-$\sim2.0\times10^6$2 quiescent galaxies	JADES spectroscopy vs FLAMINGO	Number densities at $\sim2.0\times10^6$3 are about ten times higher in the data than in FLAMINGO, ruling out cosmic variance at the $\sim2.0\times10^6$4 level (Baker et al., 2024)
Bright quasars	QLF and clustering	FLAMINGO matches low-$\sim2.0\times10^6$5 and faint-end QLFs, underpredicts bright quasars at $\sim2.0\times10^6$6, but reproduces observed clustering over $\sim2.0\times10^6$7 (Ding et al., 28 Oct 2025)

The tSZ–large-scale-structure analysis is especially notable because it treats the bias-weighted mean electron pressure as a simulation-measured quantity rather than a purely halo-model construct. FLAMINGO finds that stronger feedback increases $\sim2.0\times10^6$8 on the large scales relevant to tSZ–LSS cross-correlations, even though the same feedback lowers gas fractions and one-halo $\sim2.0\times10^6$9 within $M_{500\mathrm c}>10^{13}\,M_\odot$0 (Salcido et al., 11 May 2026). This is the opposite of the usual feedback response in small-scale weak lensing and is one reason the thermal history inferred from tSZ tomography is a complementary baryon probe.

The kSZ-plus-lensing analysis arrives at a similarly feedback-sensitive conclusion. Once galaxy–galaxy lensing is used to match the halo masses of BOSS-like samples, the Planck+ACT kSZ stacks favor stronger feedback than the fiducial FLAMINGO model calibrated to X-ray gas fractions; that stronger feedback can reduce the degree of suppression required to reconcile small-scale shear with the CMB, but only marginally alleviates tensions in tSZ power and tSZ–lensing, which are more sensitive to higher halo masses (McCarthy et al., 2024).

At the same time, not all discrepancies favor stronger feedback. In high-redshift galaxy formation, FLAMINGO underpredicts the abundance of massive quiescent galaxies at $M_{500\mathrm c}>10^{13}\,M_\odot$1, and JADES spectroscopy indicates that the mismatch cannot be explained by survey-volume fluctuations (Baker et al., 2024). In quasar demographics, the simulation reproduces low-redshift and faint-luminosity behavior but significantly underpredicts the bright end of the quasar luminosity function at $M_{500\mathrm c}>10^{13}\,M_\odot$2, even though the large volume permits robust clustering measurements (Ding et al., 28 Oct 2025). These results suggest that the same calibrated feedback model that succeeds for several low-redshift baryonic observables does not close all high-redshift or AGN-related gaps.

5. Flamingo as a visual-LLM

In machine learning, Flamingo is a family of visual-LLMs that combine a frozen vision encoder, a Perceiver Resampler, and a frozen decoder-only LLM with interleaved gated cross-attention layers (Alayrac et al., 2022). The model is autoregressive over text, but conditions on images or videos that precede each token in an interleaved multimodal sequence. Its purpose is in-context few-shot learning: a prompt containing a handful of multimodal demonstrations specifies a new task without gradient updates (Alayrac et al., 2022).

The original architecture uses a frozen NFNet-F6 visual backbone, a Perceiver-style resampler that compresses variable-size visual feature maps into a fixed set of latent tokens, and a frozen Chinchilla LLM augmented by learned gated cross-attention-dense blocks (Alayrac et al., 2022). A distinctive masking rule lets each text token cross-attend only to the most recent preceding image or video, while ordinary self-attention in the LLM carries earlier contextual information forward (Alayrac et al., 2022). This enables sequences of arbitrarily interleaved visual and textual inputs and supports both image and video conditioning.

Training mixes interleaved webpage-style multimodal data and paired image-caption or video-caption corpora under a next-token objective. The published recipe combines M3W, ALIGN, LTIP, and VTP in a multi-objective setup, with the interleaved-data component being especially important for few-shot behavior (Alayrac et al., 2022). Empirically, Flamingo established strong few-shot performance across captioning, open-ended and multiple-choice VQA, visual dialogue, and video tasks; on numerous benchmarks it outperformed models fine-tuned on vastly larger amounts of task-specific supervision (Alayrac et al., 2022).

A recurrent technical theme is preserving pretrained unimodal competence while adding multimodal conditioning. The LLM and vision encoder remain frozen during Flamingo training, and the gated cross-attention blocks are initialized so that their contribution is initially zero, allowing the pretrained LLM behavior to be retained at the start of optimization (Alayrac et al., 2022).

6. Open implementations and domain-specific descendants

OpenFlamingo is an open-source replication effort that reproduces the Flamingo design with public components and datasets, yielding models from roughly 3B to 9B parameters (Awadalla et al., 2023). It uses a CLIP ViT-L/14 vision encoder, a Perceiver Resampler, and open autoregressive language-model backbones, training on LAION-2B and MMC4 as stand-ins for the proprietary data used in the original work (Awadalla et al., 2023). Across seven vision-language datasets, OpenFlamingo models average between 80 and 89 per cent of the corresponding Flamingo performance, while making code, checkpoints, hyperparameters, and evaluation pipelines public (Awadalla et al., 2023).

The Flamingo design has also been generalized beyond vision. Music Flamingo is a large audio-LLM for music understanding, built by fine-tuning an enhanced Audio Flamingo 3 backbone on MF-Skills, a large music-caption and QA corpus, and then strengthening reasoning with MF-Think chain-of-thought data and GRPO-based reinforcement learning (Ghosh et al., 13 Nov 2025). It targets tasks such as harmonic analysis, structure, lyrics, timbre, and cultural context, and reports state-of-the-art results across more than ten music-understanding benchmarks (Ghosh et al., 13 Nov 2025). The model extends the original multimodal-fusion paradigm with long-context support and Rotary Time Embeddings, reflecting the special temporal structure of music (Ghosh et al., 13 Nov 2025).

Taken together, OpenFlamingo and Music Flamingo show that Flamingo has become a reusable architectural template rather than a single fixed model. In the open-source replication, the template is applied to public vision-language training; in the music system, it is specialized to audio-language reasoning and post-trained with domain-specific supervision and reinforcement learning (Awadalla et al., 2023). This suggests that the Flamingo lineage now occupies a broader place in multimodal modeling: not only as an initial few-shot VLM, but as a general cross-attentive framework for conditioning large autoregressive LLMs on non-text modalities (Ghosh et al., 13 Nov 2025).