---
title: Interaction Topological Transformer (ITT)
url: https://www.emergentmind.com/topics/interaction-topological-transformer-itt
type: topic
---

# Interaction Topological Transformer (ITT)

Interaction Topological Transformer (ITT) is a multiscale porous-material learning framework that combines persistent interaction topology with a Transformer and an atom-level CGCNN branch. It was introduced for chemically and structurally diverse porous materials, including metal–organic frameworks (MOFs), covalent organic frameworks (COFs), zeolites (ZEOs), and porous polymer networks (PPNs), under the premise that adsorption, transport, and stability depend jointly on local chemical environments and global pore geometry and connectivity. ITT represents each structure through structural, elemental, atomic, and pairwise-elemental streams, fuses them through a built-in Transformer, and is trained by self-supervised pretraining on 0.6 million unlabeled structures followed by supervised fine-tuning. The reported outcome is state-of-the-art, accurate, and transferable prediction across a broad porous-material benchmark suite [2509.18573].

## 1. Scope, problem setting, and nomenclature

ITT addresses a setting in which porous-material properties are governed by multiscale structure–property relationships and where labeled data are sparse and unevenly distributed across material families and tasks. The motivating difficulties are explicit: handcrafted descriptors such as surface area, pore-size distributions, and chemical fingerprints only partially encode void topology and channel connectivity; standard graph neural networks and geometric neural networks tend to under-represent higher-order interactions and long-range effects mediated by connected pore networks; realistic porous frameworks may require multiple periodic images or large supercells, “frequently exceeding \(10^4\) atoms”; and voxel or 3D-grid approaches suffer from sparsity, dimensionality, and periodicity-handling issues. Within this setting, ITT targets adsorption, transport, stability, and electronic-property prediction across MOFs, COFs, PPNs, and zeolites [2509.18573].

The exact term **Interaction Topological Transformer** should be distinguished from several adjacent transformer formulations. The autonomous-driving paper “End-to-end Contextual Perception and Prediction with Interaction Transformer” introduces an **Interaction Transformer** for multi-actor reasoning, but it explicitly does **not** use the exact term “Interaction Topological Transformer (ITT)” and does **not** formulate interaction using road-graph topology, lane-graph topology, or a named topological graph abstraction [2008.05927]. In human–object interaction detection, STIP introduces a **structure-aware Transformer over interaction proposals** and injects semantic and spatial structure into attention, making it a close conceptual neighbor rather than an exact ITT formulation [2206.06291]. In the porous-material literature, by contrast, ITT names a specific architecture whose core organizing idea is **interaction topology** [2509.18573].

## 2. Interaction topology and persistent interaction homology

The conceptual core of ITT is **interaction topology**, implemented through **persistent interaction homology (PIH)** and **interaction complexes**. The paper defines interaction topology as a multilevel interactive representation that integrates structural, elemental, and pairwise elemental information in one formalism. It is intended to go beyond both classical persistent homology, which analyzes the system as a whole and therefore lacks element-specific relational resolution, and graph-based representations, which are limited primarily to pairwise atomic interactions [2509.18573].

Let \(K\) be the simplicial complex built on all atoms at a fixed scale, and let \(\{K_i\}_{i=1}^n\) be subcomplexes induced by chosen element clusters. An **interaction simplex** is a tuple
\[
(\sigma_1,\ldots,\sigma_n), \qquad \sigma_i \in K_i,
\]
such that
\[
\sigma_1 \cap \cdots \cap \sigma_n \neq \varnothing.
\]
The resulting **\(n\)-interaction complex** is
\[
\mathcal K = \big(K,\{K_i\}_{i=1}^n\big).
\]
Two limiting cases are emphasized. When \(n=1\), \(\mathcal K=(K,\{K\})\) reduces to the ordinary simplicial complex \(K\). When \(n=2\), \(\mathcal K=(K,\{K_1,K_2\})\) captures pairwise interactions between two element clusters [2509.18573].

Interaction homology is then defined analogously to ordinary homology by forming interaction chains from formal sums of interaction simplices and using a componentwise boundary operator. Its rank is the **interaction Betti number**
\[
\beta_p \;=\; \dim_{\mathbb K}\, H_p(\{K_i\};\mathbb K).
\]
Here \(p=0\) corresponds to shared connected components, \(p=1\) to co-supported loops or tunnels, and \(p=2\) to co-supported cavities. To capture scale dependence, ITT uses **persistent interaction homology**. As scale \(a\) increases, both the whole complex \(K(a)\) and each subcomplex \(K_i(a)\) grow, yielding a filtration
\[
\mathcal F(a)=(K(a),\{K_i(a)\}_{i=1}^n).
\]
For \(a \le b\), the \((a,b)\)-persistent interaction homology group is
\[
H_p^{a,b}(\mathcal{F}_i;\mathbb K) =\operatorname{im}\!\Big( H_p\big(\mathcal F_i(a)\big)\xrightarrow{\;\;} H_p\big(\mathcal F_i(b)\big) \Big),
\]
with persistent Betti number
\[
\beta_p^{a,b} \;=\; \dim_{\mathbb K}\, H_p^{a,b}(\mathcal F;\mathbb K).
\]
The paper states that long-lived features correspond to stable, chemically meaningful patterns, such as channels framed by metals and linkers, whereas short-lived features are often noise or very local artifacts [2509.18573].

This formalism is the basis for four representational levels. Structural-level PIH captures global pore geometry and topological organization. Elemental-level PIH captures how chemically defined subsets contribute individually. Atomic-level encoding preserves fine-scale chemical and geometric information. Pairwise interaction PIH captures topological relationships between element-cluster pairs such as S–Tb. This suggests that ITT is designed to treat topology not as a single global descriptor, but as a hierarchy of interacting summaries across scales and chemical partitions [2509.18573].

## 3. Multilevel tokenization and Transformer architecture

Each material’s supercell is uniformly scaled to approximately
\[
64 \,\text{\AA} \times 64 \,\text{\AA} \times 64 \,\text{\AA}
\]
to enable consistent topological analysis. Structural and elemental topological features are computed using the “HIP method with \(n=1\)” over a grid from \(0\) to \(25 \,\text{\AA}\) with step size \(0.1 \,\text{\AA}\), recording Betti numbers in homology dimensions \(H_0\), \(H_1\), and \(H_2\). This produces \(250\) grid steps and therefore a feature length of
\[
250 \times 3 = 750.
\]
The structural embedding is a single token of shape \(1 \times 750\), while the elemental embedding contains seven tokens of shape \(7 \times 750\). The seven element clusters are denoted \(C_0,\ldots,C_6\) and are formed using co-occurrence statistics and chemical similarity [2509.18573].

Interaction-topological features are derived from PIH with \(n=2\) over pairwise element-cluster interactions. On the same \(0\) to \(25 \,\text{\AA}\) grid, only \(H_0\) and \(H_1\) are used, giving
\[
250 \times 2 = 500
\]
features per interaction pair. With seven elemental clusters, the model uses **42 distinct interaction pairs**, yielding \(42\) interaction tokens, each of length \(500\). For the atomic stream, duplicate atoms are removed by comparing local neighbor environments. The remaining atoms are initialized following the CGCNN scheme with an \(8 \,\text{\AA}\) neighbor cutoff. The CGCNN branch uses **four convolutional layers**, each producing atomic embeddings of size \(128\), and retains up to **256 atomic tokens** [2509.18573].

| Representation level | Construction | Tokenization |
|---|---|---|
| Structural | PIH with \(n=1\), \(H_0,H_1,H_2\), \(0\)–\(25\,\text{\AA}\) grid | \(1 \times 750\) |
| Elemental | Seven clusters, PIH with \(n=1\), \(H_0,H_1,H_2\) | \(7 \times 750\) |
| Atomic | CGCNN without global pooling, 4 conv layers, 128-dim | Up to 256 tokens |
| Interaction-topological | PIH with \(n=2\), \(H_0,H_1\), 42 pairs | \(42 \times 500\) |

These descriptors are projected and fed as discrete tokens into the Transformer. The resulting inventory is 1 structural token, 7 elemental tokens, up to 256 atomic tokens, and 42 interaction-topological tokens. The ITT backbone contains **eight hidden layers (ITT decoders)** with hidden size \(256\), \(8\) attention heads, and feed-forward intermediate size \(1024\). The paper describes **self-attention across all tokens**, allowing joint reasoning across structural, elemental, atomic, and interaction-topological inputs, and also states that **cross-attention layers inject interaction-topology signals into the multilevel pathway**, encouraging the integration of long-range pore-network dependencies with local information. The exact self-attention and cross-attention equations are not provided in the paper text, nor are token-type embeddings or positional encodings specified [2509.18573].

After multiple ITT layers, the fused representation is sent to property-specific output heads. In the representation analysis, the paper uses the **first token embedding** prior to the prediction head as the latent representation for visualization. The CGCNN branch is explicitly used **without global pooling**, preserving atom-resolved tokens. A plausible implication is that ITT is intended to preserve local atomic specificity while allowing topological streams to operate as long-range summary channels [2509.18573].

## 4. Pretraining, fine-tuning, and benchmark regime

ITT is trained with a two-stage strategy. The first stage is self-supervised pretraining on **602,396 unlabeled structures** drawn from **ARC-MOF (520,835)**, **MC-COF (69,840)**, **CoRE-COF v7 (1,242)**, **PPN (10,237)**, and **IZA-SC (242)**. These are used strictly as structure-only data. The self-supervised task masks a subset of the structure-level embeddings and reconstructs the masked portion from the remaining elemental, atomic, and interaction-topological context. The reported pretraining settings are **200 epochs**, **batch size 256**, and **random mask rate 50% for the first token**. The paper explicitly does not describe contrastive loss, masked atomic prediction, or external priors in this stage [2509.18573].

The second stage is supervised fine-tuning on downstream tasks. The general settings are **batch size 32**, **200 epochs**, **AdamW**, learning rate
\[
1\times 10^{-4},
\]
weight decay
\[
0.01,
\]
and warm-up over the first **10%** of total epochs. The losses are **mean squared error** for regression and **cross-entropy** for classification. Labels are standardized before training, and for each downstream task the predictions from the **top five models ranked by validation performance** are averaged [2509.18573].

The downstream benchmarks span adsorption, transport, stability, and electronic properties. The MOF suite includes the \( \mathrm{O_2}/\mathrm{N_2} \)-Select tasks, with sizes from **4,744 to 5,241**, plus **CO\(_2\)-Henry (9,525)**, **QMOF-BD band gap (20,375)**, **solvent-removal stability (2,179)**, and **thermal stability (3,132)**. The COF benchmark is **MC-COF methane uptake (69,840)**, the PPN benchmark is **hPPN methane uptake (17,846)**, and the zeolite benchmark is **ZEO–H hydrogen adsorption (215/216 samples)**. For labeled datasets, the stated default protocol is an **8:1:1 train/validation/test random split** for every task, while solvent-removal stability and thermal stability also use predefined source-paper splits in fixed-split comparisons. The principal metrics are **MAE**, **RMSE**, and \(R^2\) for regression, and **Accuracy**, **Precision**, and **Recall** for classification [2509.18573].

## 5. Empirical performance, transferability, and interpretation

The paper reports that ITT achieves **state-of-the-art results on 17 of 18 tasks** under the same evaluation protocol. On the MOF \( \mathrm{O_2}/\mathrm{N_2} \)-Select suite, the pretrained model improves on scratch training across all eight listed adsorption and transport tasks: \(R^2=0.82\) vs. \(0.80\) for Henry constant (N\(_2\)); \(0.86\) vs. \(0.85\) for Henry constant (O\(_2\)); \(0.83\) vs. \(0.83\) for uptake (N\(_2\)); \(0.87\) vs. \(0.85\) for uptake (O\(_2\)); \(0.81\) vs. \(0.77\) for self-diffusivity (N\(_2\), 1 bar); \(0.82\) vs. \(0.78\) for self-diffusivity (N\(_2\), \(\infty\) dilution); \(0.76\) vs. \(0.71\) for self-diffusivity (O\(_2\), 1 bar); and \(0.79\) vs. \(0.75\) for self-diffusivity (O\(_2\), \(\infty\) dilution). For MOF CO\(_2\)-Henry, ITT achieves MAE \(=3.20\times 10^{-1}\), compared with scratch \(=3.38\times 10^{-1}\) and source \(=4.94\times 10^{-1}\). On QMOF-BD band gap, however, scratch slightly exceeds pretrained ITT, with \(2.63\times 10^{-1}\,\mathrm{eV}\) versus ITT \(2.72\times 10^{-1}\,\mathrm{eV}\); this is one of the few explicit cases where pretraining does not help [2509.18573].

The cross-family results are similarly strong. On MC-COF, ITT reports methane-uptake MAE \(=2.35\) at high pressure and \(=9.40\times 10^{-1}\) at low pressure, versus scratch \(2.95\) and \(1.20\), and baselines \(21.7\) and \(6.63\). On hPPN, the reported methane-uptake MAE is \(=4.55\times 10^{-1}\) at 1 bar and \(=3.09\) at 65 bar, versus scratch \(7.21\times 10^{-1}\) and \(3.52\), and baselines \(1.23\) and \(7.70\). On ZEO–H, the Henry constant at low pressure is \(2.18\times 10^{-1}\) for ITT, \(2.26\times 10^{-1}\) for scratch, and \(2.87\times 10^{-1}\) for the baseline; for max loading at 403 bar, ITT yields \(2.19\), scratch \(2.13\), and baseline \(4.75\). Solvent-removal stability and thermal stability show more mixed behavior across fixed and randomized splits: SRS accuracy is \(0.70\) for ITT on the fixed split and \(0.77\) on the randomized split, versus source \(0.76\), scratch \(0.71\) fixed, and \(0.74\) randomized; TST MAE is \(43\) for ITT on the fixed split and \(48.8\) on the randomized split, versus source \(44\), scratch \(46\) fixed, and \(50.2\) randomized [2509.18573].

The paper presents the main evidence for transferability through the comparison between **ITT with pretraining** and **ITT scratch**. It states that self-supervised pretraining delivers consistent gains across medium and large datasets, with especially clear improvements on the eight \( \mathrm{O_2}/\mathrm{N_2} \) tasks, MC-COF methane uptake, hPPN methane uptake, and CO\(_2\)-Henry. The interpretation given for the band-gap exception is physical: band gap is dominated by localized electronic environments, whereas pretraining emphasizes broader structural regularities. A plausible implication is that ITT is most advantaged when long-range pore topology is a primary determinant of the target property [2509.18573].

Interpretability is treated through representation analysis and saliency. The t-SNE analysis of pretrained first-token embeddings organizes latent space by porous-material family—MOFs, COFs, PPNs, and zeolites—and within CoRE MOF 2019 it clusters structures by known topology classes such as **ths** and **fof**, despite the absence of topology labels during pretraining. Fine-tuning on a supervised topology classification task increases class separation further. Saliency analysis on the MOF example **ABAXUZ_FSR** shows that for band gap, atomic (\(\sim 32\%\)) and elemental (\(\sim 33\%\)) embeddings dominate, with interaction-topological embeddings next (\(\sim 19\%\)); for transport, structural, elemental, and interaction-topological signals become more important. Scatter plots further show **N\(_2\)** self-diffusion at 1 bar with MAE \(=3.23\times 10^{-5}\) in units of \(10^{-4}\,\mathrm{cm}^2/\mathrm{s}\), and band-gap prediction with MAE \(=0.263\,\mathrm{eV}\) [2509.18573].

## 6. Relation to adjacent transformer literatures and stated limitations

ITT occupies a distinct position relative to adjacent research on interaction-aware and topology-aware transformers. In autonomous driving, the Interaction Transformer models actor-relative geometry and recurrent interaction reasoning, but does not explicitly encode lane-graph or road-graph topology [2008.05927]. In human–object interaction detection, STIP uses interaction proposals, an interaction-centric graph, semantic dependency classes, and layout-aware attention, making it a strong relational analogue rather than the same architecture [2206.06291]. In topological deep learning, the Cellular Transformer generalizes graph transformers to cell complexes using incidence-aware self- and cross-attention [2405.14094], Persformer applies transformer-style interaction learning directly to persistence diagrams [2112.15210], TIGT augments graph transformers with cycle-informed topological positional embeddings and dual-path message passing [2402.02005], and tree-based Topological Transformers use topology-derived mask matrices and relative position encoding mechanisms defined on weighted trees [2406.15881]. Relative to these lines of work, ITT is distinguished by its combination of persistent interaction homology, element-resolved porous-material tokenization, and multiscale Transformer fusion [2509.18573].

The limitations stated for ITT are specific and bounded. Some tasks benefit less from pretraining, most visibly QMOF-BD band gap. Small datasets such as ZEO–H max loading show high variance, with ITT and scratch nearly tied. Literature-mined labels in solvent-removal stability and thermal stability may contain non-structural information that a structure-only model does not consistently capture. In unsupervised representation analysis, hypothetical datasets such as hPPNs and MC-COFs show more overlap than curated families, suggesting that unsupervised pretraining alone may not fully resolve fine-grained compositional and structural distinctions. The model is validated on MOFs, COFs, PPNs, and zeolites; possible extension to POM-MOFs and other hybrid materials is discussed prospectively rather than demonstrated. The paper also does not provide explicit runtime or memory benchmarks, arguing qualitatively instead that ITT is more scalable than giant-supercell GNNs and more principled than voxelization. This suggests that ITT’s strongest current justification is empirical performance and representational fit, rather than a detailed efficiency analysis [2509.18573].

Source: https://www.emergentmind.com/topics/interaction-topological-transformer-itt