Tensor Programs Formalism

• Tensor Programs Formalism is a unified framework for representing and analyzing multidimensional arrays using advanced tensor algebra and Einstein notation.
• It employs multi-level indexing and algebraic lifting to accurately model complex interactions in fields like radio astronomy, physics, and machine learning.
• The approach enables systematic optimization of high-order tensor operations and supports the development of domain-specific computation languages.

Tensor Programs Formalism is a mathematical and computational framework designed to systematically represent, manipulate, and analyze programs involving multidimensional arrays (tensors), with an emphasis on unifying notation, extending expressiveness beyond matrix algebra, and providing explicit methodologies for both theoretical investigations and high-performance applications. The formalism is grounded in advanced tensor algebra—including Einstein summation, index bookkeeping, and the design of domain-specific computation languages—and is increasingly adopted within radio astronomy, theoretical physics, machine learning, and high-performance computing.

1. General Principles and Notational Structure

At its core, the Tensor Programs Formalism abstracts computation over multidimensional arrays using tensor algebra, enabling concise representations that generalize and subsume standard matrix-based approaches. This formalism makes extensive use of:

• Einstein Notation: Implicit summation over repeated indices, supporting expressions such as $y^i = A^i_j x^j$ without explicit summation symbols.
• Multi-level Indexing: Distinct index sets (e.g., Greek letters for physical field variables, Latin letters for station or receptor identifiers), enabling clean separation of vector space and spatial/elemental structure.
• Algebraic Lifting: “Lifting” operations from low-rank (e.g., 2×2 matrices) to higher-rank or block-indexed tensors via multilinear algebra, supporting increased heterogeneity and complexity.

These aspects allow one to encode not only straightforward linear transformations but also nested, hierarchical, and multi-site interactions in a consistent language.

For example, in advanced radio interferometry, the visibility measurement for interferometer stations $p$ and $q$ is lifted from the 2×2 matrix equation

$V_{pq} = J_p B J_q^*$

to the tensor notation

$$[\mathcal{V}_{pq}]^{i}_{j} = [\mathcal{J}_p]^{i}_{\alpha} B^{\alpha}_{\beta} [\mathcal{J}_q]^{\beta\ast}_{j}$$

(see (Smirnov, 2011), Eq. 1), where the tensor $\mathcal{J}_p$ may encode arbitrary per-element, direction-dependent, or multi-receptor effects.

2. Motivations: Going Beyond Matrix Algebra

The primary impetus for the development of the tensor programs approach is the recognition that traditional matrix frameworks (such as the classical Radio Interferometer Measurement Equation (RIME) based on Jones matrices) are insufficient for accurately describing modern systems:

• Phased Array Feeds (PAFs) and Aperture Arrays (AAs): Require support for more than two receptors per station, and for mutual coupling between receptors.
• Non-colocated Receptors and Wide-field Effects: Necessitate representing direction-dependent effects whose variation across the aperture cannot be captured by simple matrix multiplication.
• Beamforming, Mutual Coupling, and Cross-station Interactions: Demand formalism that tracks per-element transformations and the composition of effects that do not admit a factorization into 2×2 chains.

Matrix algebra cannot express operations with varying dimensionality, coupled indices, or non-scalar field representations (e.g., 3D polarization in wide-field polarimetry); tensor programs provide a uniform solution.

3. Expressive Power: Architecture and Operations

Tensor Programs Formalism encompasses the following:

a. Generalized Signal Path Representation

Physical signal propagation—including image-plane effects ($Z$), beam responses ($E$), phase terms ($K$), and mutual coupling ($Q$)—is encoded as a contraction of multi-index tensors,

$$v^p = Q_s^p \cdot E_{\beta}^s \cdot K^s \cdot Z_{\alpha}^{s\beta} \cdot e^{\alpha}$$

with Einstein summation over repeated indices.

b. Elemental and Station Indices

Station or element labels become free indices in the expressions, allowing explicit modeling of individual hardware characteristics and their coupling.

c. Reduction to Classical Limits

When specialized to the standard assumptions (dual receptors, closed stations, colocated feeds), the tensor formalism reduces to the conventional 2×2 RIME, preserving backward compatibility and providing a mechanism for formal limit-taking (Smirnov, 2011).

4. Einstein Notation and Multilinear Structure

The formalism leverages Einstein notation for succinct and generalized expression of multi-linear operations:

• Summation is rigorously handled via index repetition, avoiding notational clutter and making dependency structure explicit.
• Index sets can be freely extended: for example, moving from 2D Jones algebra (indices range over 1,2) to 3D Wolf formalism (indices over 1,2,3) via

$B^{(3)} = T \cdot B^{(2)} \cdot T^T$

where $T$ implements the basis transformation (see (Smirnov, 2011)).

The notation enables analysis of coordinate transformation properties, such as handling of representations in linear versus circular polarization, and facilitates manipulation under symmetry and commutation properties.

5. Applications: Physical and Computational Scenarios

a. Radio Interferometry

• Beamforming: Combination of many receptor signals using a compound beam tensor $S$, integrating beamformer weights, per-receptor gains, and mutual coupling. The beamformer output is encoded as $$f = w_a v^a = S_{\beta} \cdot K \cdot Z_{\alpha}^{(\beta)} \cdot e^{\alpha}$$.
• Mutual Coupling: Full incorporation of voltage cross-talk or even field-level coupling between receptors via higher-order tensors ($Q$).
• Wide-field Polarimetry: Handling of fully three-dimensional polarization effects by range expansion of the relevant tensor indices and recasting brightness in Wolf formalism, accommodating off-axis or non-transverse field components.

b. Theoretical Physics and Beyond

Although developed for radio astronomy, the formalism’s construction of multi-index invariants and transformation rules—as seen in gravitational theories (Jarv et al., 2015) and optimization/ensemble modeling in neuroscience (Biswas et al., 2023)—demonstrates its utility for encoding physical invariants, constraints, and basis transformations.

6. Computational Strategies and Implementation

As explicit contraction of high-rank tensors can quickly lead to computational inefficiency, several optimization strategies are advocated (Smirnov, 2011):

Strategy	Principle	Impact
Partitioned Contractions	Split full contractions into smaller blocks, enabling reuse/results	Reduces FLOPs
Loop/Dependence Hoisting	Move summations with time/frequency invariance outside inner loops	Lowers inner loop cost
Commutation Optimization	Exploit algebraic commutativity for reordering of operations	Further optimization

For parallel or high-throughput scenarios, tensor programs can structure evaluation to maximize data locality and exploit hardware features (e.g., GPU parallelism).

7. Theoretical Implications and Future Directions

The adoption of the tensor programs formalism exposes previously hidden mathematical assumptions (e.g., colocation, duality, closure), facilitating:

• Explicit definition of domains of validity for existing formalisms.
• Systematic extension to new device architectures, propagation models, and cross-disciplinary applications.
• A platform for robust algorithm development, automatic program manipulation, and the design of simulation/calibration frameworks for large-scale array systems.

By providing a rigorous and extensible foundation, the tensor programs formalism supports both theoretical analyses and the practical demands of next-generation instrumentation and computational pipelines.

Summary Table: Role of Tensor Programs Formalism

Domain	Role of Tensor Programs	Limitation of Prior Approaches
Radio Astronomy	Unified representation of heterogeneous arrays	2×2 RIME cannot handle cross-coupling
Gravitational Theory	Invariance, frame-independent expressions	Matrix/coordinate forms frame-dependent
Machine Learning	Explicit high-order index structure via notation	Implicitly ordered axes, risk of errors
Optimization/Neuroscience	Dual basis, quadratic constraints, ensemble analysis	Standard vector view not geometrically explicit

In conclusion, the Tensor Programs Formalism establishes a comprehensive and unifying mathematical language for representing, analyzing, and efficiently implementing complex tensor computations, facilitating significant advances in the modeling and simulation of modern physical, engineering, and computational systems.