Learnable latent embeddings for joint behavioral and neural analysis (2204.00673v2)

Abstract: Mapping behavioral actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioral data increases, there is growing interest in modeling neural dynamics during adaptive behaviors to probe neural representations. In particular, neural latent embeddings can reveal underlying correlates of behavior, yet, we lack non-linear techniques that can explicitly and flexibly leverage joint behavior and neural data. Here, we fill this gap with a novel method, CEBRA, that jointly uses behavioral and neural data in a hypothesis- or discovery-driven manner to produce consistent, high-performance latent spaces. We validate its accuracy and demonstrate our tool's utility for both calcium and electrophysiology datasets, across sensory and motor tasks, and in simple or complex behaviors across species. It allows for single and multi-session datasets to be leveraged for hypothesis testing or can be used label-free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, and rapid, high-accuracy decoding of natural movies from visual cortex.

• The paper introduces CEBRA, a novel self-supervised algorithm that uses non-linear ICA and contrastive learning to generate consistent, interpretable embeddings.
• It employs innovative sampling strategies by conditioning on behavioral and temporal data to outperform methods like UMAP and tSNE in cross-session analyses.
• CEBRA’s robust design enables reliable multi-session and multi-animal experiments, paving the way for real-time brain-machine interface applications.

CEBRA: Learnable Latent Embeddings for Joint Behavioral and Neural Analysis

The paper introduces CEBRA, a novel self-supervised learning algorithm designed to generate consistent and interpretable high-dimensional embeddings from joint behavioral and neural data. CEBRA addresses the challenges posed by traditional representation learning methods, which often rely on linear assumptions or generative models that fail to consistently embed across varying sessions or species.

Methodology and Theoretical Foundations

CEBRA leverages advances in non-linear independent component analysis (ICA) and contrastive learning to reveal latent embeddings by directly conditioning on behavior or temporal data. The model uses a neural network encoder optimized with an InfoNCE loss, refining the latent space using both positive and negative sample pairs. The positive sample distribution is defined by a user-determined auxiliary context, such as behavior or time, while the negative sample distribution captures the general dataset variability.

The paper extends standard InfoNCE objectives by incorporating novel sampling strategies, which allow the embedding to harness the joint statistical properties of behavioral and neural data. This flexibility is exemplified in CEBRA's capacity to manage multi-session and multi-animal datasets effectively, producing consistent embeddings across varying experimental setups.

Experimental Validation

The strength of CEBRA is demonstrated across several complex datasets: synthetic data, rat hippocampal recordings, primate somatosensory cortex activity during motor tasks, and mouse visual cortex recordings using both electrophysiology and calcium imaging. In each scenario, CEBRA exceeds benchmark alternatives such as UMAP, tSNE, and pi-VAE in generating consistent embeddings and decoding performance, particularly in cross-session or multi-animal settings.

A notable aspect is CEBRA's capacity to align embeddings across different modalities, as exhibited in its application to combined calcium and spike datasets. The model achieved high accuracy in decoding spatial positions from hippocampal data and frame information from mouse visual cortex recordings during passive movie viewing, illustrating its utility across a range of behavioral tasks and neural modalities.

Implications and Future Directions

CEBRA's capacity for high-performance decoding and its consistent embeddings offer significant implications for neuroscience research. The model's adaptability means it can be utilized in real-time brain-machine interface applications, enhancing decoding speed and accuracy. Further, the theoretical underpinning supporting identifiability and consistency bolsters confidence in CEBRA's robustness across diverse neural datasets.

Practically, CEBRA provides a reliable tool for uncovering neural population dynamics and mapping neural activity to behavior without the need for strong assumptions regarding data structure. The flexibility and precision in disentangling relevant neural signals promote its application in both exploratory and hypothesis-driven neuroscience research.

The future of CEBRA could see enhancements in identifying specific neural computations, as well as refinement in its theoretical models to extend beyond linear transformations. Expansions into wider applications, such as more diverse sensory tasks or more extensive behavioral interactions, could further cement its role as a critical tool in neural data analysis.

In summary, CEBRA represents a significant step forward in the analysis of joint behavioral and neural data, providing a robust framework for understanding the latent dynamics governing neural representations of behavior.