Interaction Topological Transformer (ITT)

• Interaction Topological Transformer (ITT) is a multiscale porous-material learning framework that integrates persistent interaction topology with Transformer and CGCNN branches.
• It represents materials through structural, elemental, atomic, and pairwise-interaction tokens, enabling precise predictions for adsorption, transport, and stability.
• The framework leverages self-supervised pretraining on over 600K structures followed by supervised fine-tuning, achieving state-of-the-art results on diverse porous-material benchmarks.

Interaction Topological Transformer (ITT) is a multiscale porous-material learning framework that combines persistent interaction topology with a Transformer and an atom-level CGCNN branch. It was introduced for chemically and structurally diverse porous materials, including metal–organic frameworks (MOFs), covalent organic frameworks (COFs), zeolites (ZEOs), and porous polymer networks (PPNs), under the premise that adsorption, transport, and stability depend jointly on local chemical environments and global pore geometry and connectivity. ITT represents each structure through structural, elemental, atomic, and pairwise-elemental streams, fuses them through a built-in Transformer, and is trained by self-supervised pretraining on 0.6 million unlabeled structures followed by supervised fine-tuning. The reported outcome is state-of-the-art, accurate, and transferable prediction across a broad porous-material benchmark suite (Chen et al., 23 Sep 2025).

1. Scope, problem setting, and nomenclature

ITT addresses a setting in which porous-material properties are governed by multiscale structure–property relationships and where labeled data are sparse and unevenly distributed across material families and tasks. The motivating difficulties are explicit: handcrafted descriptors such as surface area, pore-size distributions, and chemical fingerprints only partially encode void topology and channel connectivity; standard graph neural networks and geometric neural networks tend to under-represent higher-order interactions and long-range effects mediated by connected pore networks; realistic porous frameworks may require multiple periodic images or large supercells, “frequently exceeding $10^4$ atoms”; and voxel or 3D-grid approaches suffer from sparsity, dimensionality, and periodicity-handling issues. Within this setting, ITT targets adsorption, transport, stability, and electronic-property prediction across MOFs, COFs, PPNs, and zeolites (Chen et al., 23 Sep 2025).

The exact term Interaction Topological Transformer should be distinguished from several adjacent transformer formulations. The autonomous-driving paper “End-to-end Contextual Perception and Prediction with Interaction Transformer” introduces an Interaction Transformer for multi-actor reasoning, but it explicitly does not use the exact term “Interaction Topological Transformer (ITT)” and does not formulate interaction using road-graph topology, lane-graph topology, or a named topological graph abstraction (Li et al., 2020). In human–object interaction detection, STIP introduces a structure-aware Transformer over interaction proposals and injects semantic and spatial structure into attention, making it a close conceptual neighbor rather than an exact ITT formulation (Zhang et al., 2022). In the porous-material literature, by contrast, ITT names a specific architecture whose core organizing idea is interaction topology (Chen et al., 23 Sep 2025).

2. Interaction topology and persistent interaction homology

The conceptual core of ITT is interaction topology, implemented through persistent interaction homology (PIH) and interaction complexes. The paper defines interaction topology as a multilevel interactive representation that integrates structural, elemental, and pairwise elemental information in one formalism. It is intended to go beyond both classical persistent homology, which analyzes the system as a whole and therefore lacks element-specific relational resolution, and graph-based representations, which are limited primarily to pairwise atomic interactions (Chen et al., 23 Sep 2025).

Let $K$ be the simplicial complex built on all atoms at a fixed scale, and let $\{K_i\}_{i=1}^n$ be subcomplexes induced by chosen element clusters. An interaction simplex is a tuple

$$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$

such that

$$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$

The resulting $n$-interaction complex is

$\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$

Two limiting cases are emphasized. When $n=1$, $\mathcal K=(K,\{K\})$ reduces to the ordinary simplicial complex $K$. When $K$0, $K$1 captures pairwise interactions between two element clusters (Chen et al., 23 Sep 2025).

Interaction homology is then defined analogously to ordinary homology by forming interaction chains from formal sums of interaction simplices and using a componentwise boundary operator. Its rank is the interaction Betti number

$K$2

Here $K$3 corresponds to shared connected components, $K$4 to co-supported loops or tunnels, and $K$5 to co-supported cavities. To capture scale dependence, ITT uses persistent interaction homology. As scale $K$6 increases, both the whole complex $K$7 and each subcomplex $K$8 grow, yielding a filtration

$K$9

For $\{K_i\}_{i=1}^n$0, the $\{K_i\}_{i=1}^n$1-persistent interaction homology group is

$\{K_i\}_{i=1}^n$2

with persistent Betti number

$\{K_i\}_{i=1}^n$3

The paper states that long-lived features correspond to stable, chemically meaningful patterns, such as channels framed by metals and linkers, whereas short-lived features are often noise or very local artifacts (Chen et al., 23 Sep 2025).

This formalism is the basis for four representational levels. Structural-level PIH captures global pore geometry and topological organization. Elemental-level PIH captures how chemically defined subsets contribute individually. Atomic-level encoding preserves fine-scale chemical and geometric information. Pairwise interaction PIH captures topological relationships between element-cluster pairs such as S–Tb. This suggests that ITT is designed to treat topology not as a single global descriptor, but as a hierarchy of interacting summaries across scales and chemical partitions (Chen et al., 23 Sep 2025).

3. Multilevel tokenization and Transformer architecture

Each material’s supercell is uniformly scaled to approximately

$\{K_i\}_{i=1}^n$4

to enable consistent topological analysis. Structural and elemental topological features are computed using the “HIP method with $\{K_i\}_{i=1}^n$5” over a grid from $\{K_i\}_{i=1}^n$6 to $\{K_i\}_{i=1}^n$7 with step size $\{K_i\}_{i=1}^n$8, recording Betti numbers in homology dimensions $\{K_i\}_{i=1}^n$9, $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$0, and $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$1. This produces $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$2 grid steps and therefore a feature length of

$$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$3

The structural embedding is a single token of shape $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$4, while the elemental embedding contains seven tokens of shape $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$5. The seven element clusters are denoted $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$6 and are formed using co-occurrence statistics and chemical similarity (Chen et al., 23 Sep 2025).

Interaction-topological features are derived from PIH with $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$7 over pairwise element-cluster interactions. On the same $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$8 to $$(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,$$9 grid, only $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$0 and $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$1 are used, giving

$$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$2

features per interaction pair. With seven elemental clusters, the model uses 42 distinct interaction pairs, yielding $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$3 interaction tokens, each of length $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$4. For the atomic stream, duplicate atoms are removed by comparing local neighbor environments. The remaining atoms are initialized following the CGCNN scheme with an $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$5 neighbor cutoff. The CGCNN branch uses four convolutional layers, each producing atomic embeddings of size $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$6, and retains up to 256 atomic tokens (Chen et al., 23 Sep 2025).

Representation level	Construction	Tokenization
Structural	PIH with $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$7, $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$8, $$\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.$$9–$n$0 grid	$n$1
Elemental	Seven clusters, PIH with $n$2, $n$3	$n$4
Atomic	CGCNN without global pooling, 4 conv layers, 128-dim	Up to 256 tokens
Interaction-topological	PIH with $n$5, $n$6, 42 pairs	$n$7

These descriptors are projected and fed as discrete tokens into the Transformer. The resulting inventory is 1 structural token, 7 elemental tokens, up to 256 atomic tokens, and 42 interaction-topological tokens. The ITT backbone contains eight hidden layers (ITT decoders) with hidden size $n$8, $n$9 attention heads, and feed-forward intermediate size $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$0. The paper describes self-attention across all tokens, allowing joint reasoning across structural, elemental, atomic, and interaction-topological inputs, and also states that cross-attention layers inject interaction-topology signals into the multilevel pathway, encouraging the integration of long-range pore-network dependencies with local information. The exact self-attention and cross-attention equations are not provided in the paper text, nor are token-type embeddings or positional encodings specified (Chen et al., 23 Sep 2025).

After multiple ITT layers, the fused representation is sent to property-specific output heads. In the representation analysis, the paper uses the first token embedding prior to the prediction head as the latent representation for visualization. The CGCNN branch is explicitly used without global pooling, preserving atom-resolved tokens. A plausible implication is that ITT is intended to preserve local atomic specificity while allowing topological streams to operate as long-range summary channels (Chen et al., 23 Sep 2025).

4. Pretraining, fine-tuning, and benchmark regime

ITT is trained with a two-stage strategy. The first stage is self-supervised pretraining on 602,396 unlabeled structures drawn from ARC-MOF (520,835), MC-COF (69,840), CoRE-COF v7 (1,242), PPN (10,237), and IZA-SC (242). These are used strictly as structure-only data. The self-supervised task masks a subset of the structure-level embeddings and reconstructs the masked portion from the remaining elemental, atomic, and interaction-topological context. The reported pretraining settings are 200 epochs, batch size 256, and random mask rate 50% for the first token. The paper explicitly does not describe contrastive loss, masked atomic prediction, or external priors in this stage (Chen et al., 23 Sep 2025).

The second stage is supervised fine-tuning on downstream tasks. The general settings are batch size 32, 200 epochs, AdamW, learning rate

$\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$1

weight decay

$\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$2

and warm-up over the first 10% of total epochs. The losses are mean squared error for regression and cross-entropy for classification. Labels are standardized before training, and for each downstream task the predictions from the top five models ranked by validation performance are averaged (Chen et al., 23 Sep 2025).

The downstream benchmarks span adsorption, transport, stability, and electronic properties. The MOF suite includes the $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$3-Select tasks, with sizes from 4,744 to 5,241, plus CO$\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$4-Henry (9,525), QMOF-BD band gap (20,375), solvent-removal stability (2,179), and thermal stability (3,132). The COF benchmark is MC-COF methane uptake (69,840), the PPN benchmark is hPPN methane uptake (17,846), and the zeolite benchmark is ZEO–H hydrogen adsorption (215/216 samples). For labeled datasets, the stated default protocol is an 8:1:1 train/validation/test random split for every task, while solvent-removal stability and thermal stability also use predefined source-paper splits in fixed-split comparisons. The principal metrics are MAE, RMSE, and $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$5 for regression, and Accuracy, Precision, and Recall for classification (Chen et al., 23 Sep 2025).

5. Empirical performance, transferability, and interpretation

The paper reports that ITT achieves state-of-the-art results on 17 of 18 tasks under the same evaluation protocol. On the MOF $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$6-Select suite, the pretrained model improves on scratch training across all eight listed adsorption and transport tasks: $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$7 vs. $\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$8 for Henry constant (N$\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).$9); $n=1$0 vs. $n=1$1 for Henry constant (O$n=1$2); $n=1$3 vs. $n=1$4 for uptake (N$n=1$5); $n=1$6 vs. $n=1$7 for uptake (O$n=1$8); $n=1$9 vs. $\mathcal K=(K,\{K\})$0 for self-diffusivity (N$\mathcal K=(K,\{K\})$1, 1 bar); $\mathcal K=(K,\{K\})$2 vs. $\mathcal K=(K,\{K\})$3 for self-diffusivity (N$\mathcal K=(K,\{K\})$4, $\mathcal K=(K,\{K\})$5 dilution); $\mathcal K=(K,\{K\})$6 vs. $\mathcal K=(K,\{K\})$7 for self-diffusivity (O$\mathcal K=(K,\{K\})$8, 1 bar); and $\mathcal K=(K,\{K\})$9 vs. $K$0 for self-diffusivity (O$K$1, $K$2 dilution). For MOF CO$K$3-Henry, ITT achieves MAE $K$4, compared with scratch $K$5 and source $K$6. On QMOF-BD band gap, however, scratch slightly exceeds pretrained ITT, with $K$7 versus ITT $K$8; this is one of the few explicit cases where pretraining does not help (Chen et al., 23 Sep 2025).

The cross-family results are similarly strong. On MC-COF, ITT reports methane-uptake MAE $K$9 at high pressure and $K$00 at low pressure, versus scratch $K$01 and $K$02, and baselines $K$03 and $K$04. On hPPN, the reported methane-uptake MAE is $K$05 at 1 bar and $K$06 at 65 bar, versus scratch $K$07 and $K$08, and baselines $K$09 and $K$10. On ZEO–H, the Henry constant at low pressure is $K$11 for ITT, $K$12 for scratch, and $K$13 for the baseline; for max loading at 403 bar, ITT yields $K$14, scratch $K$15, and baseline $K$16. Solvent-removal stability and thermal stability show more mixed behavior across fixed and randomized splits: SRS accuracy is $K$17 for ITT on the fixed split and $K$18 on the randomized split, versus source $K$19, scratch $K$20 fixed, and $K$21 randomized; TST MAE is $K$22 for ITT on the fixed split and $K$23 on the randomized split, versus source $K$24, scratch $K$25 fixed, and $K$26 randomized (Chen et al., 23 Sep 2025).

The paper presents the main evidence for transferability through the comparison between ITT with pretraining and ITT scratch. It states that self-supervised pretraining delivers consistent gains across medium and large datasets, with especially clear improvements on the eight $K$27 tasks, MC-COF methane uptake, hPPN methane uptake, and CO$K$28-Henry. The interpretation given for the band-gap exception is physical: band gap is dominated by localized electronic environments, whereas pretraining emphasizes broader structural regularities. A plausible implication is that ITT is most advantaged when long-range pore topology is a primary determinant of the target property (Chen et al., 23 Sep 2025).

Interpretability is treated through representation analysis and saliency. The t-SNE analysis of pretrained first-token embeddings organizes latent space by porous-material family—MOFs, COFs, PPNs, and zeolites—and within CoRE MOF 2019 it clusters structures by known topology classes such as ths and fof, despite the absence of topology labels during pretraining. Fine-tuning on a supervised topology classification task increases class separation further. Saliency analysis on the MOF example ABAXUZ_FSR shows that for band gap, atomic ($K$29) and elemental ($K$30) embeddings dominate, with interaction-topological embeddings next ($K$31); for transport, structural, elemental, and interaction-topological signals become more important. Scatter plots further show N$K$32 self-diffusion at 1 bar with MAE $K$33 in units of $K$34, and band-gap prediction with MAE $K$35 (Chen et al., 23 Sep 2025).

6. Relation to adjacent transformer literatures and stated limitations

ITT occupies a distinct position relative to adjacent research on interaction-aware and topology-aware transformers. In autonomous driving, the Interaction Transformer models actor-relative geometry and recurrent interaction reasoning, but does not explicitly encode lane-graph or road-graph topology (Li et al., 2020). In human–object interaction detection, STIP uses interaction proposals, an interaction-centric graph, semantic dependency classes, and layout-aware attention, making it a strong relational analogue rather than the same architecture (Zhang et al., 2022). In topological deep learning, the Cellular Transformer generalizes graph transformers to cell complexes using incidence-aware self- and cross-attention (Ballester et al., 2024), Persformer applies transformer-style interaction learning directly to persistence diagrams (Reinauer et al., 2021), TIGT augments graph transformers with cycle-informed topological positional embeddings and dual-path message passing (Choi et al., 2024), and tree-based Topological Transformers use topology-derived mask matrices and relative position encoding mechanisms defined on weighted trees (Choromanski et al., 2024). Relative to these lines of work, ITT is distinguished by its combination of persistent interaction homology, element-resolved porous-material tokenization, and multiscale Transformer fusion (Chen et al., 23 Sep 2025).

The limitations stated for ITT are specific and bounded. Some tasks benefit less from pretraining, most visibly QMOF-BD band gap. Small datasets such as ZEO–H max loading show high variance, with ITT and scratch nearly tied. Literature-mined labels in solvent-removal stability and thermal stability may contain non-structural information that a structure-only model does not consistently capture. In unsupervised representation analysis, hypothetical datasets such as hPPNs and MC-COFs show more overlap than curated families, suggesting that unsupervised pretraining alone may not fully resolve fine-grained compositional and structural distinctions. The model is validated on MOFs, COFs, PPNs, and zeolites; possible extension to POM-MOFs and other hybrid materials is discussed prospectively rather than demonstrated. The paper also does not provide explicit runtime or memory benchmarks, arguing qualitatively instead that ITT is more scalable than giant-supercell GNNs and more principled than voxelization. This suggests that ITT’s strongest current justification is empirical performance and representational fit, rather than a detailed efficiency analysis (Chen et al., 23 Sep 2025).