Flinch: Differentiable Field-Level Inference

• Flinch is a fully differentiable framework for field-level cosmological inference, directly mapping masked CMB data to parameter estimates via gradient propagation.
• It integrates advanced gradient-based sampling methods like MCLMC, achieving significant efficiency gains over traditional HMC and pseudo-Cℓ pipelines.
• Its Julia implementation with tools such as Zygote.jl and Capse.jl yields 20–40% tighter parameter constraints, demonstrating both computational scalability and statistical accuracy.

Flinch refers to a fully differentiable, high-performance framework developed for field-level inference (FLI) on angular/curved-sky cosmological data, as implemented in Flinch.jl. Its principal aim is to propagate gradients from individual map pixels through the forward inference pipeline to cosmological parameters, enabling direct map-to-parameter Bayesian inference without intermediate summary statistic likelihoods. The framework targets masked, noisy spherical data (e.g., cosmic microwave background [CMB] temperature fields), delivering statistically efficient parameter constraints and computational scalability beyond the limitations of standard pseudo-$C_\ell$ pipelines and conventional Hamiltonian Monte Carlo (HMC). Flinch is characterized by end-to-end differentiability, high-dimensional gradient-based sampling, and tight integration with differentiable cosmology emulators such as Capse.jl, enabling efficient, direct exploitation of the statistical information contained in masked sky maps (Crespi et al., 30 Oct 2025).

1. Architectural Principles and Statistical Model

Flinch is designed for full field-level inference on the sphere, where instead of compressing the data into summary statistics (such as pseudo-$C_\ell$), the latent sky field is modeled directly as the object of inference. The fundamental inference hierarchy is expressed as:

$$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$

where:

• $\mathbf{d}$: observed, masked, noisy pixelized sky map,
• $\mathbf{a}$: spherical-harmonic coefficients of the latent field,
• $\mathbf{C} = \{C_\ell\}$: theoretical angular power spectrum, possibly parametrized by cosmological parameters $\boldsymbol{\theta}$,
• $\mathcal{G}(\mathbf{a}\mid\mathbf{C})$: Gaussian prior in harmonic space,
• $\mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$: Gaussian likelihood on unmasked pixels incorporating mask, beam, and noise.

The posterior over the latent field and cosmological parameters is sampled directly: $$\mathcal{P}(\mathbf{a},\boldsymbol{\theta}\mid \mathbf{d},\mathbf{N}) \propto \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})\, \mathcal{G}(\mathbf{a}\mid \mathbf{C}(\boldsymbol{\theta}))\, \pi(\boldsymbol{\theta})$$ This approach enables cosmological inference with full non-Gaussian treatment of low-multipole, masked-sky modes and pixel-level information.

2. Implementation: Differentiability and Computational Structure

Flinch is implemented in Julia, leveraging reverse-mode automatic differentiation (AD) via Zygote.jl. The entire computational pipeline—from map pixelization (HEALPix using HealpixMPI.jl), spherical harmonic transforms (with adjoint rules for the forward and backward passes), response functions (beam and pixel window), to map-to-spectrum emulation—is differentiable. Gradients flow from observation space to latent sky field and ultimately to cosmological parameters, allowing for efficient use of gradient-based MCMC and optimization.

The differentiable pipeline is constructed such that: $C_\ell$0 This guarantees that parameter gradients, including those propagating through spherical harmonic transforms and cosmology emulators (e.g., Capse.jl), are computed efficiently. A full gradient evaluation costs approximately 2.1 times a forward pass due to optimized adjoint rules for the spherical harmonic transforms (Crespi et al., 30 Oct 2025).

3. Sampling Algorithms and Efficiency

To tackle the high dimensionality of latent field inference (often exceeding $C_\ell$1 parameters at high resolution), Flinch uses gradient-based samplers. The main sampling algorithms compared include:

• Standard HMC,
• No-U-Turn Sampler (NUTS),
• MicroCanonical Langevin Monte Carlo (MCLMC),
• Pathfinder (for initialization only).

MCLMC is emphasized for its microcanonical (constant-energy) stochastic dynamics, which avoids the Metropolis accept/reject step, enabling more efficient exploration and higher effective sample rates in high dimension. The SDE governing MCLMC is: $C_\ell$2 where $C_\ell$3 and $C_\ell$4 projects velocities to ensure constant kinetic energy.

Empirically, MCLMC achieves orders-of-magnitude higher sampling efficiency than HMC or NUTS for $C_\ell$5 maps—about $C_\ell$6–$C_\ell$7 more efficient than NUTS and $C_\ell$8–$C_\ell$9 more than standard HMC, as measured in effective sample size per gradient evaluation. Wall-clock time for a typical chain is reduced from days to several hours while maintaining accurate posterior sampling (Crespi et al., 30 Oct 2025).

4. Statistical Performance: Map and Parameter Reconstruction

On simulated, masked, noisy CMB temperature maps, Flinch demonstrates:

• Accurate map reconstruction: Outside the mask, posterior means closely recover the input; inside the mask, only large-scale features are reconstructed, reflecting the prior's spatial coupling.
• Posterior over the angular power spectrum $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$0: Correctly reproduces the expected non-Gaussian structure at low $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$1 and converges to Gaussianity for high $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$2.
• Cosmological parameter inference: With end-to-end differentiation through theory emulation (e.g., Capse.jl), Flinch produces parameter constraints $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$3–$$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$4 tighter than pseudo-$$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$5 pipelines for the same data, mask, and $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$6 range.

Quantitative constraints estimated for fiducial cosmology $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$7 are:

• Flinch–NUTS: $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$8, $$\pi(\boldsymbol{\theta}) \rightarrow \delta_{\rm D}\!\bigl(\mathbf{C}-\mathbf{C}(\boldsymbol{\theta})\bigr) \rightarrow \mathcal{G}(\mathbf{a}\mid \mathbf{C}) \rightarrow \mathcal{L}(\mathbf{d}\mid \mathbf{a},\mathbf{N})$$9, $\mathbf{d}$0, $\mathbf{d}$1, $\mathbf{d}$2.
• Pseudo-$\mathbf{d}$3–NUTS: uncertainties on these parameters are consistently $\mathbf{d}$4–$\mathbf{d}$5 larger.

5. Reparameterization and Preconditioning

To facilitate efficient sampling and mitigate hierarchical funnel pathologies, several strategies are utilized:

• Spectrum reparameterization: Power spectra $\mathbf{d}$6 can be mapped to approximately normally distributed variables $\mathbf{d}$7, with appropriate Jacobian factors.
• Field whitening: Sampling in standardized coordinates, $\mathbf{d}$8, regularizes geometry.
• Likelihood regularization: Observed and model maps divided by $\mathbf{d}$9, yielding unit-variance likelihood.
• Parameter block-whitening: Cosmological parameter vector $\mathbf{a}$0 is whitened via estimated covariance for isotropic proposal scaling.
• Pathfinder: Used to initialize chains close to high-posterior-density regions, accelerating MCLMC thermalization by $\mathbf{a}$1.

6. Limitations and Future Extensions

Current demonstrated applications are to spin-0 (temperature) fields in simulated setups that include beam, pixel window, mask, and Gaussian noise, but do not encompass full real-data systematics or spin fields. While all components are differentiable, extensions to spin-weighted fields, additional cosmological probes (e.g., large-scale structure), and true multi-component field-level analyses are explicitly targeted for future work. Application to actual survey data, systematics validation, and interface with additional differentiable cosmology tools (e.g., LimberJack.jl, Blast.jl, Effort.jl) are underway.

7. Impact and Significance

Flinch represents a significant advance in field-level cosmological inference methodology, providing an AD-native, fully Bayesian map-space framework. By bypassing the need for pseudo-$\mathbf{a}$2 distributions and associated suboptimal or non-Gaussian likelihoods, and by directly propagating gradients from maps to cosmological parameters, Flinch extracts more statistical power from masked sky observations. The integration of high-dimensional efficient sampling (MCLMC), differentiable transformations (including SHT and pixel response), and emulators encapsulates a model for future scalable, robust cosmological analysis pipelines (Crespi et al., 30 Oct 2025).

Feature	Flinch Implementation	Reference
Field representation	Harmonic coefficients (alm, HEALPix)	(Crespi et al., 30 Oct 2025)
Gradient propagation	End-to-end via Zygote.jl	(Crespi et al., 30 Oct 2025)
Main sampler	MCLMC, NUTS, HMC	(Crespi et al., 30 Oct 2025)
Differentiable cosmology	Capse.jl interface (CMB emulation)	(Crespi et al., 30 Oct 2025)
Efficiency gain	MCLMC $\mathbf{a}$3–$\mathbf{a}$4 faster than NUTS	(Crespi et al., 30 Oct 2025)
Constraint improvement	$\mathbf{a}$5–$\mathbf{a}$6 narrower posteriors than pseudo-$\mathbf{a}$7	(Crespi et al., 30 Oct 2025)

Flinch exemplifies the convergence of modern probabilistic programming, automatic differentiation, and cosmological inference on the sphere, and forms a foundation for generalizable, information-efficient future analyses in CMB, large-scale structure, and other areas relying on masked, high-dimensional angular data.