Adaptive Sampling CI for Electronic Structure

• Adaptive Sampling Configuration Interaction (ASCI) is a selected CI algorithm that constructs a compact variational wave function by retaining the most important Slater determinants.
• It employs an iterative workflow of determinant ranking, search, and Hamiltonian diagonalization to efficiently approximate full configuration interaction for large electronic active spaces.
• Deterministic second-order perturbation correction and scalable parallelization techniques enable ASCI to achieve chemically accurate results in previously intractable multiconfigurational calculations.

Adaptive Sampling Configuration Interaction (ASCI) is a selected configuration interaction (SCI) algorithm designed to efficiently approximate the full configuration interaction (FCI) solution to the electronic structure problem, with emphasis on scalability to large active spaces and extensibility to correlated electronic structure and dynamical observables. The method iteratively constructs a compact variational wave function by identifying and retaining only the most important Slater determinants from the exponentially large Hilbert space. ASCI is a fully deterministic method that enables chemically accurate calculations in active spaces previously intractable for exact diagonalization and provides a robust framework for integration into multiconfigurational and embedding electronic structure approaches (Levine et al., 2019, Tubman et al., 2018, Tubman et al., 2018).

1. Variational Ansatz and Determinant Selection

The ASCI approach formulates a variational CI wave function as

$|\Psi\rangle = \sum_I C_I |D_I\rangle,$

where the determinants $\{|D_I\rangle\}$ define the variational subspace $S$ and are selected for their large contributions to the CI expansion. The associated variational energy is given by

$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$

with Hamiltonian matrix elements $H_{IJ}$ computed between determinants. Determinant selection, central to the ASCI scheme, exploits a first-order perturbative estimate derived from the CI eigenvalue equation,

$$C_i = \frac{\sum_{j\neq i} H_{ij} C_j}{H_{ii} - E},$$

providing a ranking metric

$$A_i = \frac{\sum_{j\in S} H_{ij} C_j}{H_{ii} - E_{\mathrm{var}}},$$

to identify external determinants most likely to significantly lower the energy upon adding them to the variational subspace. Iterative pruning and growth of $S$ using this score ensures that the most important correlation effects are efficiently captured (Levine et al., 2019, Tubman et al., 2018).

2. Iterative Workflow, Search, and Hamiltonian Construction

The ASCI workflow consists of the following key steps:

1. Ranking and selection of reference determinants: Retain the top $c$ determinants in $S$ by $\{|D_I\rangle\}$0 as "core" parents for excitation generation.
2. Search space generation: Produce all singly and doubly excited determinants from the core parents.
3. Scoring and ranking: Compute $\{|D_I\rangle\}$1 for each generated determinant, discarding those below a cutoff.
4. Variational subspace update: Merge $\{|D_I\rangle\}$2 with the top $\{|D_I\rangle\}$3 ranked candidates (by $\{|D_I\rangle\}$4) to form an expanded subspace $\{|D_I\rangle\}$5.
5. Diagonalization: Construct and diagonalize the Hamiltonian in $\{|D_I\rangle\}$6 to obtain new coefficients and energy.
6. Convergence: Repeat the search-diagonalize cycle until $\{|D_I\rangle\}$7 converges or $\{|D_I\rangle\}$8 reaches a target $\{|D_I\rangle\}$9 (Levine et al., 2019, Tubman et al., 2018).

Efficient Hamiltonian construction exploits data structures such as dynamic bit masking and residue arrays to identify connected determinant pairs rapidly and reduce computational bottlenecks (Tubman et al., 2018).

3. Deterministic Second-Order Perturbation (PT2) Correction

Residual correlation energy missing from the truncated CI expansion is accounted for using second-order Epstein-Nesbet perturbation theory,

$S$0

where the sum is over external determinants linked to $S$1 by single or double excitations. Deterministic evaluation exploits partial summation, constraint partitioning (triplet/quadruplet indices), and fast sorting algorithms (including hash-based compression) for high throughput and perfect scaling on multi-node or GPU architectures. For large-scale models, this delivers orders-of-magnitude speed-up compared to stochastic PT2 approaches (e.g., SHCI), while providing arbitrarily small statistical error (Tubman et al., 2018, Tubman et al., 2018).

The FCI limit can be reliably estimated by extrapolating $S$2 to $S$3 over several $S$4 (Levine et al., 2019, Park, 2021).

4. Orbital Optimization and CASSCF Integration (ASCI-SCF)

ASCI can serve as an FCI-solver in CASSCF-like multiconfigurational approaches for large active spaces, enabling efficient self-consistent field (SCF) optimization of the orbital basis ("ASCI-SCF"). The standard double-loop scheme consists of:

• Outer loop: Orbital optimization using variational ASCI energy $S$5, typically via BFGS quasi-Newton steps with diagonal Hessian preconditioning.
• Inner loop: For fixed orbitals, repeat the ASCI variational/selection cycle to update the determinant list and coefficients.
• Periodic determinant refresh: Every $S$6 orbital steps, re-optimize the determinant list to adapt to changing orbital character.

Key distinctions from exact CASSCF arise because the ASCI variational space is not invariant under active-active orbital rotations, producing numerous local extrema. Mitigation strategies include starting from natural orbitals, staged growth of the $S$7 parameter, spin-state cycling, and selection by minimum variational energy (Levine et al., 2019, Park, 2021).

Analytic energy gradients and Z-vector (Lagrangian) methods enable efficient geometry optimization and electronic property evaluation at near-CASSCF accuracy for active spaces with 30–50 orbitals (Park, 2021).

5. Parallelization and Computational Scaling

Distributed memory parallelization of ASCI leverages memory-efficient constraint partitioning, static load balancing, and block-row storage for the Hamiltonian and SpMV operations. The ASCI search is parallelized by dividing constraints (e.g., triplet indices) among processors, and collective quickselect algorithms efficiently extract the global top-ranked determinants after local partial scoring. The dominant cost at scale is the Hamiltonian construction; the parallel Davidson diagonalization step achieves high efficiency, with weak scaling limits set mainly by synchronization and load imbalance in nonzero block distribution. Benchmark calculations demonstrate variational ASCI calculations with up to $S$8 determinants on 16,384 cores at $S$9 parallel efficiency for systems such as Cr$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$0(24e,30o) (Williams-Young et al., 2023).

6. Applications and Performance Benchmarks

ASCI and ASCI-SCF have enabled multiconfigurational calculations in active spaces (orbital, electron pairs) inaccessible to conventional methods:

• Periacenes and PAHs: Active spaces of up to (52e,52o); chemically accurate singlet-triplet gaps and electronic structure up to the 6-periacene ($$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$1 kcal/mol), with wall times of $$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$2–$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$3s per SCF iteration on 24 CPU cores; analysis confirmed biradical but not strong polyradicaloid character (Levine et al., 2019, Park, 2021).
• Iron Porphyrin: Balanced active spaces (40e,42o); ASCI-SCF energies lower than HCISCF with fewer determinants, ground-state and excited-state splittings in quantitive agreement with experiments (Levine et al., 2019).
• G1 Test Set, Cr$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$4, F$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$5, C$$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$6: ASCI+PT2 achieves $$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$7mHa error for atomization energies and energies, with PT2 correction and natural orbital rotations further accelerating convergence relative to alternative SCI schemes (Tubman et al., 2018, Tubman et al., 2018).
• DMFT and Dynamical Simulations: As an impurity solver in DMFT, ASCI enables simulations of cluster models (e.g., $$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$8 clusters with 17–24 baths) at a cost several orders of magnitude lower than prior CI-based approaches; accurate Green's functions and self-energies are obtained with a single or two-layer zero-state augmentation (Mejuto-Zaera et al., 2017).
• Time-dependent Propagation: The TD-ASCI extension combines the static selection with a short iterative Lanczos algorithm for unitary propagation of the CI wave function, enabling the computation of dipole autocorrelation functions and absorption spectra to within $$E_{\mathrm{var}} = \langle \Psi | \hat{H} | \Psi \rangle = \sum_{IJ} C_I H_{IJ} C_J,$$9 eV of EOM-CCSDT(T) accuracy using $H_{IJ}$0 Krylov vectors and up to $H_{IJ}$1 determinants (Shee et al., 2024).

7. Algorithmic Innovations and Future Challenges

ASCI algorithms have introduced several innovations, including:

• Sorting-based determinant selection, memory-efficient search constraints, residue array and dynamic bit-masking Hamiltonian construction, hash-based PT2 evaluation, and scalable parallelization approaches (Tubman et al., 2018, Tubman et al., 2018, Williams-Young et al., 2023).
• Deterministic PT2 correction and extrapolation to the FCI limit, enabling robust energy estimates without stochastic error.
• Analytic gradients and Z-vector implementation for geometry optimization at the ASCI-SCF-PT2 level (Park, 2021).
• Extensions to time-dependent propagation for dynamical observables (Shee et al., 2024).

Remaining challenges include integration of dynamical correlation beyond PT2 (e.g., multireference perturbation theory, coupled-cluster methods), more robust orbital optimization strategies for excited states, further reduction of computational cost (block-sparse tensors, advanced pruning), and optimization for exascale hardware and GPU acceleration (Levine et al., 2019, Park, 2021, Williams-Young et al., 2023).

---

Key References:

• "CASSCF with Extremely Large Active Spaces using the Adaptive Sampling Configuration Interaction Method" (Levine et al., 2019)
• "An efficient deterministic perturbation theory for selected configuration interaction methods" (Tubman et al., 2018)
• "Modern Approaches to Exact Diagonalization and Selected Configuration Interaction with the Adaptive Sampling CI Method" (Tubman et al., 2018)
• "A Parallel, Distributed Memory Implementation of the Adaptive Sampling Configuration Interaction Method" (Williams-Young et al., 2023)
• "Near-Exact CASSCF-Level Geometry Optimization with a Large Active Space using Adaptive Sampling Configuration Interaction Self-Consistent Field Corrected with Second-Order Perturbation Theory (ASCI-SCF-PT2)" (Park, 2021)
• "Real-time propagation of adaptive sampling selected configuration interaction wave function" (Shee et al., 2024)
• "Dynamical Mean-Field Theory Simulations with the Adaptive Sampling Configuration Interaction Method" (Mejuto-Zaera et al., 2017)