Exploiting tensor symmetries and tensor compression within algorithmic differentiation for HPC

Determine how to leverage tensor symmetries and tensor compression within algorithmic differentiation workflows used in high-performance computing settings that impose pure-function constraints and restrict primitives such as mutation and inter-process communication, so that these sophisticated, problem-specific optimizations can be achieved without major manual effort.

Background

The paper critiques algorithmic differentiation (AD) systems that rely on partial evaluation and runtime tracing, noting that such systems often enforce pure-function constraints that exclude mutating operations and communication primitives essential for high-performance computing (HPC). This separation hinders the development of performance optimizations because gradients are not available in a human-readable analytic form during optimization design.

Within this context, the authors point out that even basic HPC techniques (e.g., pipelining and recomputation) require substantial manual engineering. They then explicitly state that more sophisticated, problem-specific optimizations—specifically the exploitation of tensor symmetries and tensor compression—remain open questions under these constraints. This highlights a gap in current AD tooling for HPC applications where symmetry-aware and compression-aware optimizations are critical for efficiency.