PauLIB: A High-Performance Library for Processing Pauli Strings

Abstract: Processing large Pauli sums is a significant bottleneck in quantum chemistry, Pauli propagation, and Pauli-based compilation. Existing frameworks often suffer from Python interpreter overhead or utilize hash-map data structures that hinder SIMD vectorization and complicate multi-threaded merging. We present PauLIB, a header-only C++20 library designed to eliminate these bottlenecks through three key architectural choices. A bit-packed binary symplectic representation that encodes each qubit in two bits, reducing Pauli multiplication to a bitwise XOR and a population count; a sorted array layout that replaces hash maps to enable branch-predictable SIMD bulk operations; and a struct-of-arrays (SoA) memory layout that exposes contiguous word arrays for explicit SIMD vectorization. Benchmarks at 500 qubits show that single Pauli string multiplication runs at 25ns per operation-14 times faster than PauliEngine and 660 times faster than Qiskit-flat across all pair counts tested. Hamiltonian outer-product multiplication is approximately 10 times faster than PauliEngine and 45 times faster than Qiskit at all tested sizes. Greedy commutation grouping, the dominant preprocessing cost in variational algorithms, achieves up to 21,000 times speedup over PennyLane, driven by the compact bit-packed representation. The compact layout reduces the memory footprint of a one-million-term Hamiltonian at 500 qubits from 1,036MB (Qiskit) to 142MB, a 7.3 times reduction that directly enables larger problem sizes within a fixed memory budget. PauLIB is open source and provides C++ and Python interfaces.

• The paper introduces a bit-packed symplectic representation for Pauli strings, enabling efficient bitwise operations and constant execution time per qubit.
• It leverages both AoS and SoA memory layouts to optimize SIMD vectorization and enable high-throughput deduplication and gate operations.
• Benchmark results on 500-qubit scales show up to 660× speedup and significant memory savings compared to Python and Julia-based frameworks.

PauLIB: High-Performance Processing of Pauli Strings

Motivation and Context

Quantum computing workflows in quantum chemistry, circuit compilation, and Pauli propagation are heavily reliant on efficient manipulation of large sets of Pauli strings. The operational bottleneck in these domains arises from the rapid scaling of Pauli terms, often reaching millions for chemistry problems and substantial counts in error correction or variational algorithms. Prior frameworks, predominantly written in Python or Julia, incur significant interpreter overhead, suffer from non-contiguous memory layouts (hash maps, character strings), and present barriers to SIMD vectorization, resulting in limited scalability and throughput.

PauLIB addresses these bottlenecks through a C++20 header-only library that spans the full generalized Pauli algebra, supporting complex coefficients, non-Clifford rotations, memory-efficient storage, and exploiting hardware parallelism at all levels.

Core Architecture and Data Layout

The central innovation of PauLIB lies in its bit-packed binary symplectic representation. Each qubit is mapped to two bits: one for the Pauli $X$ operator and another for $Z$, such that an $n$-qubit string fits in $\lceil n/64 \rceil$ 64-bit words. This representation allows Pauli multiplication to be reduced to a bitwise XOR—eliminating phase calculations to population count instructions—yielding performance independent of the qubit count.

The library provides two principal memory layouts:

• Array-of-Structures (AoS): Each term is stored in a packed struct containing X-words, Z-words, a phase byte, and coefficient.
• Struct-of-Arrays (SoA): Transposes data, organizing all X-words, Z-words, and flag bytes across contiguous arrays, catalyzing SIMD vectorization.

  Figure 1: Memory layout of PauliString, illustrating packed 64-bit Z-words, X-words, and a byte for sign and imaginary phase.

The SoA layout is critical for gate operations, as single-qubit gates access only relevant word arrays, enabling vectorized operations on batches of Pauli strings without memory pollution. The sorted array layout replaces hash maps, facilitating branch-predictable linear scans for deduplication, eliminating cache misses and pointer-heavy structures.

Figure 2: Memory layout of PauliSumSoA, showing contiguous arrays for Z/X-words and flags enabling parallel manipulation of sums with $M$ terms.

Efficient Pauli Algebra Operations

All single-string operations (multiplication, commutation checks, and symplectic inner product) are reduced to bitwise kernels. Each uses an XOR for word updates and population counts for phase/parity, yielding constant execution time per qubit.

Pauli propagation, gate application, and Hamiltonian multiplication leverage both layouts:

• Clifford gates: In-place relabeling using bit arithmetic.
• Non-Clifford rotations: Split each anti-commuting term according to $\cos(\theta)$ and $-i\sin(\theta)$, doubling the sum size in worst-case scenarios.
• Hamiltonian outer-product: Pre-allocates output, parallelizes across threads (OpenMP), and deploys explicit SIMD via Google Highway to process multiple terms per cycle.

Sorting and deduplication are managed by sorting term indices lexicographically—enabled by fixed-width keys—and linearly merging duplicates, giving $O(M \log M)$ complexity and sequential memory access.

Benchmark Analysis

PauLIB's performance profile is anchored by benchmarks at the 500-qubit scale, comparing against PauliEngine, Qiskit, and PennyLane. Notable results include:

• Single Pauli multiplication: 25 ns per operation (SoA layout), 14$\times$ faster than PauliEngine, 660$\times$ faster than Qiskit.
• Hamiltonian multiplication (AoS layout): 10$Z$0 faster than PauliEngine, 45$Z$1 faster than Qiskit, 140$Z$2 speedup at $Z$3 term products versus pythonic backends.
• Greedy commutation grouping: Up to 21,000$Z$4 faster than PennyLane, with scaling dominated by symplectic bit-packing rather than SIMD/threading at large qubit counts.
• Memory footprint: A million-term Hamiltonian at 500 qubits occupies 142 MB in PauLIB versus 1,036 MB for Qiskit (7.3$Z$5 reduction), tightly coupling memory savings to data locality and bit-packing.

Practical and Theoretical Implications

PauLIB constitutes a unified, high-throughput back-end for large-scale Pauli processing in quantum software. Its architectural choices enable:

• Bulk throughput: Efficient manipulation of multi-million-term Hamiltonians and propagation tallies for variational algorithms.
• Hardware exploitation: Fully taps SIMD and OpenMP parallelism, scalable across modern NUMA and HBM-enabled systems.
• Memory scalability: Supports expansion to hundreds of millions of terms, crucial for extending the quantum software stack to emerging application scales.
• Interface extensibility: BLAS-like kernels, operator exponentiation, and distributed-memory propagation are explicitly planned, promising further extensions to circuit simulation and expectation-batching.

On a theoretical level, PauLIB's symplectic logic maps bitwise Pauli algebra directly onto hardware primitives, matching the minimal arithmetic cost—an optimality unattainable with character or hash-based representations.

Future Directions

Successors to PauLIB will focus on distributed-memory scaling, further lowering the barrier for quantum chemistry and error correction problems requiring hundreds of millions of Pauli terms. Parallel propagation simulators and composable algebraic kernels (commutators, exponentiation, batch expectation evaluation) are under development, broadening the suite for quantum circuit simulation.

Conclusion

PauLIB establishes a high-performance foundation for Pauli string manipulation, bridging quantum software and hardware efficiency. Through compact binary representation, sorted array layouts, and struct-of-arrays traversal, it achieves superior throughput, minimal latency, and memory savings across quantum simulation workloads. Its extensibility and alignment with hardware capabilities position it as a pivotal component for advancing scalable quantum applications and research (2605.25974).