Natural Scenes fMRI Dataset

Updated 29 May 2026

Natural Scenes fMRI Dataset is a large-scale, high-resolution neuroimaging resource capturing detailed brain responses to tens of thousands of natural scene images using 7T fMRI.
The dataset employs a rigorous experimental design with repeated image presentations and a 1-back task to boost signal reliability and support robust decoding analysis.
Advanced preprocessing and modeling pipelines, including ridge regression and cortical surface mapping, facilitate accurate encoding, decoding, and cross-subject evaluations.

The Natural Scenes fMRI Dataset (NSD) is a large-scale, high-resolution human neuroimaging resource designed to enable data-driven modeling of visual and semantic representation in the human brain. Leveraging ultra-high field (7 T) fMRI, NSD encompasses tens of thousands of unique naturalistic scene photographs and dense, single-image neural sampling in visual cortex, constituting the largest open dataset of its kind to date (Gifford et al., 2023, Lu et al., 2023, Guo et al., 4 May 2026).

1. Experimental Design and Scope

NSD comprises 8 healthy adults (age ≈ 20–35 years, mixed gender, right-handed), each completing approximately 30–40 one-hour scanning sessions. In total, NSD contains approximately 73,000 unique, full-color photographs of natural scenes, systematically sampled from the MS-COCO database to maximize semantic and contextual diversity (Gifford et al., 2023). Each subject viewed ∼9,000–10,000 distinct images, with each image typically presented twice (once per session) to enable within-item SNR boosting. No stimuli were shared across subjects, which is a distinctive aspect relative to other multi-subject datasets (Guo et al., 4 May 2026).

The experimental protocol is highly structured: every session contains ~8 runs of 100 trials each. Each trial presents an image for 3 s (central fixation, ∼10° visual angle), followed by a 1 s gray-background ISI. Subjects are instructed to maintain fixation and perform a 1-back repetition detection task orthogonal to high-level scene content.

2. MRI Acquisition and Preprocessing

Data acquisition utilized a Siemens Magnetom 7 T scanner equipped with a 32-channel head coil. Key functional sequence parameters include 1.8 mm isotropic spatial resolution, repetition time (TR) of either 1.0 s (Gifford et al., 2023) or 1.6 s (Lu et al., 2023), echo time of 25–30 ms, and multiband acceleration (factor 4–6) for rapid whole-brain coverage (occipital and temporal lobes prioritized). Each one-hour session comprises approximately 750 trials per subject (Guo et al., 4 May 2026).

Preprocessing was implemented using an fMRIPrep-based workflow with the following steps (Gifford et al., 2023):

Slice-timing correction
Rigid-body motion correction
Susceptibility-distortion correction (with fieldmaps or blip—up/down pairs)
Co-registration to the subject's T1-weighted anatomical scan
Surface projection with normalization to FreeSurfer's fsaverage cortex
Optional surface smoothing (2 mm FWHM)
ICA-AROMA for motion artifact removal
High-pass filtering (> 0.008 Hz)
Single-trial, mass-univariate GLMs to produce β-estimates (“beta-images”) for each image presentation

Vertex-wise β maps are z-scored within runs and averaged across repeats to maximize reliability.

3. Data Organization and Access

NSD is distributed in BIDS format, with both raw and preprocessed (“derivatives”) data available for download. The directory hierarchy is organized per subject and per session, with functional (func/), anatomical (anat/), and derivatives/fmriprep folders containing corresponding NIfTI and surface-format data. Key image- and session-level metadata are stored in accompanying JSON sidecars. Vertex-wise β responses are provided on the fsaverage cortical surface over a selection of ~20,000 reliable vertices, predominantly in early and high-level visual cortex (Gifford et al., 2023).

The open-access training data can be obtained via OpenNeuro (ds004141) and DataLad. Access to the challenge test set (subject-level held-out images) is gated by a data use agreement, without paywall, from https://naturalscenesdataset.org (Gifford et al., 2023).

4. Computational Modeling and Evaluation Methodologies

The NSD design emphasizes single-image decoding, encoding, and high-dimensional representational analysis. The standard modeling pipeline is as follows (Gifford et al., 2023):

Encoding model: At each cortical vertex $v$ , fit a linear model

$y_v = X \beta_v + \epsilon_v,$

where $y_v$ are β estimates, $X$ is the feature matrix (pixels, CNN features, semantic embeddings, etc.), and $\beta_v$ are model weights.

Cross-validation and regularization: Ridge regression (L2), with hyperparameter selection via leave-one-session-out nested cross-validation.
Test set evaluation: Prediction is performed on held-out images; performance is quantified via Pearson correlation between predictions and measured $y_v$ , squared to yield $R^2_v$ and normalized by empirical noise ceilings (split-half reliability across repeats) to produce $S_v = R^2_v / \text{NC}_v$ .

In challenge settings and benchmarks such as the Algonauts Project, overall metric is the average $S_v$ across selected vertices and subjects.

For image reconstruction and retrieval, additional metrics are prevalent (Guo et al., 4 May 2026, Lu et al., 2023):

Image retrieval accuracy: For each test β, retrieve the correct image embedding from a pool of 300 distractors (top-1 accuracy, chance = 0.33%).
Structural Similarity Index (SSIM) and pixelwise correlation for low-level image reconstruction quality.
High-level feature alignment: Retrieval accuracy in AlexNet, Inception-v3, or CLIP embedding spaces.

5. Applications in Decoding and Neuro-AI

NSD has catalyzed advances in both encoding and decoding paradigms:

Image reconstruction from fMRI: Models such as MindDiffuser use NSD β-maps to regress into latent spaces of generative models (e.g., VQ-VAE, CLIP, diffusion models), achieving state-of-the-art semantic and structural alignment between reconstructed and original images (Lu et al., 2023).
Cross-subject and source-free adaptation: The StableMind framework introduces regularized adaptation strategies (ridge reuse, Fourier-based brain feature augmentation, and difficulty-aware blur) to achieve robust cross-subject decoding with only one-hour of target-subject fMRI data, demonstrating image retrieval accuracies above 84% on NSD with strong resistance to inter-individual variability (Guo et al., 4 May 2026).
Comparative evaluations: NSD provides the basis for large-scale, leaderboard-based model comparison in the Algonauts Project, promoting transparent advancement in brain-vision modeling (Gifford et al., 2023).

6. Technical Innovations and Limitations

NSD's innovations include:

Ultra-dense sampling (∼73,000 unique images) per subject, versus other datasets like BOLD5000 (∼5,000 images) (Chang et al., 2018).
Uniform, high-SNR single-image β estimates via repeated presentations and rigorous denoising.
Cortical surface-based response extraction, enhancing cross-subject alignability.
Granular task structure (orthogonal 1-back) to discourage overt semantic strategies.

Limitations include a modest sample size (N=8), absence of shared stimulus sets across subjects (hinders direct cross-subject RDM analyses), and the high operational overhead of 7 T fMRI acquisition. The dataset's subject-specific design necessitates sophisticated approaches for cross-subject or population-level decoding.

NSD stands in contrast and complement to datasets such as BOLD5000 (Chang et al., 2018), which offers broader subject diversity but lower single-image sampling density, and domain-targeted datasets such as those supporting BigBiGAN-based decoding from fMRI (Mozafari et al., 2020).

Future directions articulated in associated literature include expansion to broader demographic cohorts, multimodal integration (e.g., EEG/MEG), and enhanced noise ceiling validation to refine model benchmarking. The dataset's scale, annotation richness, and cross-dataset compatibility establish NSD as a foundational benchmark for next-generation neuro-AI modeling and integrative visual neuroscience (Gifford et al., 2023, Lu et al., 2023, Guo et al., 4 May 2026).