---
title: Flamingo in Cosmology & Multimodal AI
url: https://www.emergentmind.com/topics/flamingo
type: topic
---

# Flamingo in Cosmology & Multimodal AI

FLAMINGO is a research name used in two technically unrelated but individually prominent lineages. In cosmology and astrophysics, FLAMINGO denotes “Full-hydro Large-scale structure simulations with All-sky Mapping for the INterpretation of next Generation Observations,” a suite of large-volume cosmological hydrodynamical simulations built for large-scale structure, cluster cosmology, and baryonic-systematics modeling [2312.08277]. In multimodal machine learning, Flamingo denotes a family of autoregressive visual-language models designed for in-context few-shot learning from arbitrarily interleaved image, video, and text sequences [2204.14198]. The shared name therefore spans two distinct research programs: one centered on baryons, feedback, and survey-scale mock universes, and the other on cross-attentive multimodal generation.

## 1. Scope and nomenclature

In the cosmological literature, FLAMINGO is a simulation program built around full-physics hydrodynamics, massive neutrinos, feedback calibration, and all-sky mapping products for weak lensing, Sunyaev-Zel’dovich observables, X-ray emission, and related probes [2306.05492]. Its flagship intermediate-resolution run has a \((2.8\,\mathrm{Gpc})^3\) comoving volume, with \(m_{\rm gas}=1.07\times10^9\,M_\odot\) and \(m_{\rm CDM}=5.65\times10^9\,M_\odot\); at \(z=0\) in the fiducial cosmology it contains \(\sim2.0\times10^6\) haloes with \(M_{500\mathrm c}>10^{13}\,M_\odot\) and \(\sim1.1\times10^5\) haloes with \(M_{500\mathrm c}>10^{14}\,M_\odot\) [2312.08277]. The suite was designed explicitly for large-scale structure cosmology, cluster gas physics, and forward modeling of next-generation survey observables.

In the multimodal-model literature, Flamingo is a visual-language architecture that accepts sequences of text tokens interleaved with images and videos, and models
\[
p(y\mid x)=\prod_{\ell=1}^{L} p(y_\ell\mid y_{<\ell},x_{\le \ell}),
\]
so that task behavior can be induced purely by prompting rather than parameter updates [2204.14198]. It was introduced as a few-shot multimodal learner and later replicated in open-source form and extended to audio and music domains [2308.01390].

A plausible implication is that “FLAMINGO” now functions less as a single topic than as a stable label for two independent but influential infrastructures: one for simulation-based cosmology, one for cross-modal generative modeling.

## 2. FLAMINGO as a cosmological hydrodynamical simulation suite

The FLAMINGO simulation program is built with the SWIFT code, SPHENIX hydrodynamics, and massive neutrinos treated with the \(\delta f\) method [2312.08277]. Initial conditions are generated with MONOFONIC using higher-order baryon–CDM initial conditions and massive neutrinos, while halo identification and spherical-overdensity properties are derived with VELOCIraptor and SOAP [2312.08277]. The fiducial cosmology adopts the DES Y3 “3×2pt + all external” parameter set:
\[
\Omega_{\rm m}=0.306,\quad \Omega_{\rm b}=0.0486,\quad \sigma_8=0.807,\quad H_0=68.1\,{\rm km\,s^{-1}Mpc^{-1}},\quad n_s=0.967,
\]
with \(\sum m_\nu=0.06\) eV [2312.08277].

Its subgrid model includes element-by-element radiative cooling and photo-heating, pressure-based star formation, time-delayed stellar mass loss and metal enrichment, stellar feedback, black-hole growth, and either thermally driven or kinetic jet AGN feedback [2312.08277]. A central design feature is calibration by Gaussian-process emulators trained on Latin hypercubes of 32 smaller-volume simulations, allowing feedback parameters to be tuned against the \(z\simeq0\) galaxy stellar mass function and group/cluster gas fractions rather than by manual adjustment [2306.05492]. This calibration framework also defines controlled model families such as `fgas±Nσ`, `M*−σ`, and jet-mode variants, each corresponding to shifted target observables rather than arbitrary parameter changes [2306.05492].

The suite was built to enable both physical realism and survey relevance. Large-volume hydrodynamics gives access to rare massive haloes and two-halo-scale correlations, while the calibration strategy links nuisance freedom to directly interpretable observables such as \(f_{\rm gas}(R_{500\mathrm c})\) and the stellar mass function [2306.05492]. This makes FLAMINGO particularly suitable for analyses in which baryonic uncertainty must be propagated into power spectra, lensing, SZ, or X-ray observables.

## 3. Cluster gas physics, scaling relations, and baryonic response

A major early application was the comparison of FLAMINGO clusters to X-ray data. Clusters and groups are selected above \(M_{500\mathrm c}>10^{13}\,M_\odot\), with thermodynamic profiles measured in 3D shells and analyzed using volume-, mass-, and X-ray-weighted estimators [2312.08277]. The fiducial FLAMINGO model reproduces observed temperature, density, pressure, and entropy profiles well, and matches X-ray and SZ scaling relations once hydrostatic mass bias is accounted for; the principal discrepancy is that core iron metallicities are too high by about \(0.3\) dex [2312.08277]. The simulations show very little evolution away from self-similar expectations in \(T\), \(P\), \(K\), and common scaling relations, whereas metallicity decreases with redshift [2312.08277].

Feedback mode strongly affects cluster cores. Relative to thermal AGN feedback calibrated to the same gas fraction at \(R_{500\mathrm c}\), kinetic jet feedback produces hotter, higher-entropy, lower-density cores, lower central metallicities, and a cool-core fraction lower by more than a factor of two [2312.08277]. Lower global gas fractions in thermal models likewise reduce core and intermediate-radius densities, raise temperatures and entropies at \(0.1\!-\!1\,R_{500\mathrm c}\), lower pressure, and modestly alter metallicity profiles [2312.08277].

The same feedback variations leave a clear imprint on non-linear clustering. FLAMINGO’s baryon-response analysis defines
\[
R(k,z)=\frac{P_{\rm hydro}(k,z)}{P_{\rm DMO}(k,z)},
\]
and shows that the response is well converged only for simulation volumes in excess of \(200^3\,\mathrm{Mpc}^3\), while a Gaussian-process emulator reproduces the FLAMINGO responses to better than one per cent up to \(k=10~h\,\mathrm{Mpc}^{-1}\) and \(z\le3\) across the feedback grid [2410.17109]. Lower gas fractions produce stronger suppression of the total matter power spectrum, the use of collimated jets enhances the effect, and the tested dependence on background cosmology is small, at the \(\sim1\%\) level for \(k<10~h\,\mathrm{Mpc}^{-1}\) [2410.17109]. In weak-lensing scattering-transform analyses, the same baryonic physics suppresses scattering coefficients by up to \(10\%\) on scales corresponding to \(k\gtrsim0.1\,h\,\mathrm{Mpc}^{-1}\), although shape noise and \(1.5\) arcmin smoothing reduce the effective suppression to \(\sim1\%\) under Euclid-like conditions [2510.09761].

These results do not imply that baryons alone solve the late-time clustering problem. A dedicated FLAMINGO study of the \(S_8\) tension finds that baryonic effects are not sufficiently large to reconcile low-redshift cosmic shear, tSZ, and related cross-spectra with the CMB-specified standard model, even though those baryonic effects materially alter non-linear observables [2309.07959].

## 4. Extended observational programs and astrophysical applications

Beyond cluster thermodynamics, FLAMINGO has been used as a forward model for a wide range of baryon-sensitive observables, from kSZ stacking and diffuse X-ray cross-correlations to quasar statistics and high-redshift galaxy abundances.

| Application | Observable or sample | Representative result |
|---|---|---|
| Thermal history from tSZ–LSS | \(\langle b_{\rm h}P_{\rm e}\rangle\), \(\mathrm{d}y/\mathrm{d}z\) | \(\langle b_{\rm h}P_{\rm e}\rangle\) scales roughly as \(S_8^{\epsilon(z)}\) with \(\epsilon(z)\approx3\) over \(0.1\le z\le1\), and current data prefer low \(S_8\) plus strong feedback [2605.11083] |
| kSZ plus galaxy–galaxy lensing | BOSS galaxies with Planck+ACT kSZ stacks | kSZ measurements prefer stronger feedback than models calibrated to low-\(z\) X-ray gas fractions [2410.19905] |
| X-ray–cosmic-shear cross-correlation | DES shear × ROSAT diffuse X-ray maps | The signal is most sensitive to haloes with \(10^{14}\le M_{200\mathrm c}/M_\odot\le10^{15}\); unresolved AGN treatment changes whether fiducial or stronger-feedback models are preferred [2602.02484] |
| Secondary CMB and foreground maps | \(\kappa_{\rm CMB}\), tSZ, kSZ, CIB, radio, screening | FLAMINGO produces self-consistent full-sky mock maps reproducing a wide range of observational constraints [2512.09891] |
| High-\(z\) quiescent galaxies | JADES spectroscopy vs FLAMINGO | Number densities at \(z=3\!-\!4.5\) are about ten times higher in the data than in FLAMINGO, ruling out cosmic variance at the \(3\sigma\) level [2410.14773] |
| Bright quasars | QLF and clustering | FLAMINGO matches low-\(z\) and faint-end QLFs, underpredicts bright quasars at \(z\approx1\!-\!3\), but reproduces observed clustering over \(0\lesssim z\lesssim3\) [2510.24283] |

The tSZ–large-scale-structure analysis is especially notable because it treats the bias-weighted mean electron pressure as a simulation-measured quantity rather than a purely halo-model construct. FLAMINGO finds that stronger feedback increases \(\langle b_{\rm h}P_{\rm e}\rangle\) on the large scales relevant to tSZ–LSS cross-correlations, even though the same feedback lowers gas fractions and one-halo \(Y\) within \(r_{500}\) [2605.11083]. This is the opposite of the usual feedback response in small-scale weak lensing and is one reason the thermal history inferred from tSZ tomography is a complementary baryon probe.

The kSZ-plus-lensing analysis arrives at a similarly feedback-sensitive conclusion. Once galaxy–galaxy lensing is used to match the halo masses of BOSS-like samples, the Planck+ACT kSZ stacks favor stronger feedback than the fiducial FLAMINGO model calibrated to X-ray gas fractions; that stronger feedback can reduce the degree of suppression required to reconcile small-scale shear with the CMB, but only marginally alleviates tensions in tSZ power and tSZ–lensing, which are more sensitive to higher halo masses [2410.19905].

At the same time, not all discrepancies favor stronger feedback. In high-redshift galaxy formation, FLAMINGO underpredicts the abundance of massive quiescent galaxies at \(z>3\), and JADES spectroscopy indicates that the mismatch cannot be explained by survey-volume fluctuations [2410.14773]. In quasar demographics, the simulation reproduces low-redshift and faint-luminosity behavior but significantly underpredicts the bright end of the quasar luminosity function at \(z\approx1\!-\!3\), even though the large volume permits robust clustering measurements [2510.24283]. These results suggest that the same calibrated feedback model that succeeds for several low-redshift baryonic observables does not close all high-redshift or AGN-related gaps.

## 5. Flamingo as a visual-language model

In machine learning, Flamingo is a family of visual-language models that combine a frozen vision encoder, a Perceiver Resampler, and a frozen decoder-only language model with interleaved gated cross-attention layers [2204.14198]. The model is autoregressive over text, but conditions on images or videos that precede each token in an interleaved multimodal sequence. Its purpose is in-context few-shot learning: a prompt containing a handful of multimodal demonstrations specifies a new task without gradient updates [2204.14198].

The original architecture uses a frozen NFNet-F6 visual backbone, a Perceiver-style resampler that compresses variable-size visual feature maps into a fixed set of latent tokens, and a frozen Chinchilla language model augmented by learned gated cross-attention-dense blocks [2204.14198]. A distinctive masking rule lets each text token cross-attend only to the most recent preceding image or video, while ordinary self-attention in the language model carries earlier contextual information forward [2204.14198]. This enables sequences of arbitrarily interleaved visual and textual inputs and supports both image and video conditioning.

Training mixes interleaved webpage-style multimodal data and paired image-caption or video-caption corpora under a next-token objective. The published recipe combines M3W, ALIGN, LTIP, and VTP in a multi-objective setup, with the interleaved-data component being especially important for few-shot behavior [2204.14198]. Empirically, Flamingo established strong few-shot performance across captioning, open-ended and multiple-choice VQA, visual dialogue, and video tasks; on numerous benchmarks it outperformed models fine-tuned on vastly larger amounts of task-specific supervision [2204.14198].

A recurrent technical theme is preserving pretrained unimodal competence while adding multimodal conditioning. The language model and vision encoder remain frozen during Flamingo training, and the gated cross-attention blocks are initialized so that their contribution is initially zero, allowing the pretrained language model behavior to be retained at the start of optimization [2204.14198].

## 6. Open implementations and domain-specific descendants

OpenFlamingo is an open-source replication effort that reproduces the Flamingo design with public components and datasets, yielding models from roughly 3B to 9B parameters [2308.01390]. It uses a CLIP ViT-L/14 vision encoder, a Perceiver Resampler, and open autoregressive language-model backbones, training on LAION-2B and MMC4 as stand-ins for the proprietary data used in the original work [2308.01390]. Across seven vision-language datasets, OpenFlamingo models average between 80 and 89 per cent of the corresponding Flamingo performance, while making code, checkpoints, hyperparameters, and evaluation pipelines public [2308.01390].

The Flamingo design has also been generalized beyond vision. Music Flamingo is a large audio-language model for music understanding, built by fine-tuning an enhanced Audio Flamingo 3 backbone on MF-Skills, a large music-caption and QA corpus, and then strengthening reasoning with MF-Think chain-of-thought data and GRPO-based reinforcement learning [2511.10289]. It targets tasks such as harmonic analysis, structure, lyrics, timbre, and cultural context, and reports state-of-the-art results across more than ten music-understanding benchmarks [2511.10289]. The model extends the original multimodal-fusion paradigm with long-context support and Rotary Time Embeddings, reflecting the special temporal structure of music [2511.10289].

Taken together, OpenFlamingo and Music Flamingo show that Flamingo has become a reusable architectural template rather than a single fixed model. In the open-source replication, the template is applied to public vision-language training; in the music system, it is specialized to audio-language reasoning and post-trained with domain-specific supervision and reinforcement learning [2308.01390]. This suggests that the Flamingo lineage now occupies a broader place in multimodal modeling: not only as an initial few-shot VLM, but as a general cross-attentive framework for conditioning large autoregressive language models on non-text modalities [2511.10289].

Source: https://www.emergentmind.com/topics/flamingo