THOI: An efficient and accessible library for computing higher-order interactions enhanced by batch-processing (2501.03381v1)

Abstract: Complex systems are characterized by nonlinear dynamics, multi-level interactions, and emergent collective behaviors. Traditional analyses that focus solely on pairwise interactions often oversimplify these systems, neglecting the higher-order interactions critical for understanding their full collective dynamics. Recent advances in multivariate information theory provide a principled framework for quantifying these higher-order interactions, capturing key properties such as redundancy, synergy, shared randomness, and collective constraints. However, two major challenges persist: accurately estimating joint entropies and addressing the combinatorial explosion of interacting terms. To overcome these challenges, we introduce THOI (Torch-based High-Order Interactions), a novel, accessible, and efficient Python library for computing high-order interactions in continuous-valued systems. THOI leverages the well-established Gaussian copula method for joint entropy estimation, combined with state-of-the-art batch and parallel processing techniques to optimize performance across CPU, GPU, and TPU environments. Our results demonstrate that THOI significantly outperforms existing tools in terms of speed and scalability. For larger systems, where exhaustive analysis is computationally impractical, THOI integrates optimization strategies that make higher-order interaction analysis feasible. We validate THOI accuracy using synthetic datasets with parametrically controlled interactions and further illustrate its utility by analyzing fMRI data from human subjects in wakeful resting states and under deep anesthesia. Finally, we analyzed over 900 real-world and synthetic datasets, establishing a comprehensive framework for applying higher-order interaction (HOI) analysis in complex systems.

• The paper introduces THOI, a Python library leveraging batch processing and Gaussian copula methods to compute higher-order interactions efficiently.
• It employs robust heuristics and multi-platform compatibility to process complex systems with up to 30 variables faster than previous tools.
• Validation on synthetic and fMRI datasets confirms its accuracy in detecting intricate system dynamics and altered brain network states.

An Efficient Library for High-Order Interaction Analysis: THOI

The exploration of complex systems often demands a rigorous analysis of the intricate interdependencies across multiple variables. Traditional methods focusing on pairwise interactions tend to overlook these higher-order interactions, resulting in an incomplete understanding. The paper "THOI: An efficient and accessible library for computing higher-order interactions enhanced by batch-processing" introduces an innovative tool designed to address these challenges by offering a sophisticated framework to capture and analyze higher-order interactions (HOI) in complex systems.

Overview and Approach

The authors present THOI, a Python library that leverages PyTorch's processing capabilities to enhance the efficiency of HOI calculations. At its core, THOI utilizes concepts from multivariate information theory, extending the traditional analyses to capture collective dynamics marked by synergy, redundancy, and other key informational attributes. The library employs the Gaussian copula method for estimating joint entropies, sidestepping the need for direct estimation of joint probability distributions—a computationally expensive task in high-dimensional spaces.

THOI employs a batch-processing architecture, optimized for various computational environments using CPUs, GPUs, and TPUs. This allows for significant gains in the processing speed and scalability when performing exhaustive analyses on systems with a higher number of variables. For instance, THOI can analyze all interactions within a small system of up to 30 variables faster than existing tools such as JIDT, HOI toolbox, and others, which often struggle with the computational expense required for such analyses.

Key Features and Capabilities

1. Batch Processing and Gaussian Copula: The library performs joint entropy estimation using the Gaussian copula method, exploiting the simplicity of matrix operations, which are amenable to parallel processing. This significantly reduces computational complexity.
2. Robust Heuristics: For larger systems, where exhaustive computations become infeasible due to the exponential growth of combinations with increasing variables, THOI integrates heuristic optimization strategies, such as greedy algorithms and simulated annealing, to efficiently explore possible variable combinations.
3. Multi-platform Compatibility: THOI extends its applicability across different hardware setups, ensuring that it remains accessible even without high-performance computing facilities.

Validation and Applications

The performance of THOI was validated using both synthetic datasets with predefined interaction parameters and real-world fMRI data. In synthetic settings, the tool successfully identified known ground-truth high-order interactions, demonstrating its accuracy and reliability. Moreover, the practical utility of THOI was showcased through an fMRI data analysis, revealing altered brain network dynamics under different states of consciousness. Specifically, it detected a reduction in synergistic interactions during deep anesthesia compared to wakeful states, thus highlighting its potential utility in neuroscientific research.

Additionally, the extensive benchmarking across over 900 datasets emphasized THOI's capability to process large datasets efficiently within a short timeframe, further solidifying its position as a practical tool for higher-order analysis in a variety of complex systems.

Implications and Future Directions

The introduction of THOI represents a significant advancement in the computational paper of complex systems, providing a sophisticated approach to exploring multi-level interactions. Its implications extend across multiple disciplines, including neuroscience, economics, and social sciences, where understanding the dynamics of complex interdependencies is crucial.

Future developments may focus on expanding the library's capabilities to integrate more flexible statistical models that can account for non-Gaussian distributions, thereby enhancing its robustness across a wider spectrum of data types. Additionally, further optimization for multi-GPU environments could offer even greater improvements in processing efficiency, especially for very large systems that remain computationally challenging.

In summary, THOI provides an invaluable resource for researchers in the domain of complex systems, offering a scalable, efficient, and accessible means to explore the rich tapestry of interactions that characterize these systems.