Core-Optimized Orbitals (COO) in Spectroscopy and CI

• COO is a variational strategy that adapts the orbital basis to target specific electronic excitations or many-body correlations, moving beyond fixed Hartree–Fock/Kohn–Sham orbitals.
• In core-level spectroscopy, COO optimizes core-hole states via ΔSCF/ROKS with a spin-free X2C Hamiltonian, achieving accurate K-edge energies without empirical shifts.
• In sparse CI applications, COO co-optimizes orbital rotations and CI coefficients to dramatically compress the determinant count, enhancing computational efficiency and accuracy.

Core-Optimized Orbitals (COO) denotes an orbital optimization strategy in which the one-particle basis is variationally adapted to a targeted electronic structure problem rather than inherited unchanged from a reference Hartree–Fock or Kohn–Sham solution. In the arXiv literature, the acronym appears in two closely related but technically distinct settings. In core-level spectroscopy, COO refers to the orbital sets obtained by $\Delta$SCF or ROKS optimization for a $1s\!\to\!\text{virtual}$ excitation within an orbital-optimized DFT framework combined with the spin-free exact two-component (X2C) Hamiltonian (Cunha et al., 2021). In strongly correlated many-body calculations, COO denotes a co-optimization of sparse CI coefficients and orbital rotations, with the aim of absorbing a large fraction of the dynamical correlation directly into the single-particle basis (Zhang et al., 21 May 2026). In both usages, the central object is a unitary orbital rotation $U=e^\kappa$, and the central principle is that orbital flexibility can materially alter the compactness and accuracy of the many-electron ansatz.

1. Terminological scope and conceptual role

The term COO is used for two variational constructions that share a common mathematical mechanism but target different physical regimes. In the spectroscopy setting, one constructs a core-hole orbital manifold for K-edge calculations by enforcing a target occupancy pattern and optimizing the orbitals in the presence of a spin-free X2C one-electron operator (Cunha et al., 2021). In the sparse-CI setting, one seeks the ground state by varying both a selected determinant expansion and the orbital basis, so that the same many-body accuracy can be reached with substantially fewer determinants (Zhang et al., 21 May 2026).

Context	Optimization target	Principal outcome
OO-DFT/X2C spectroscopy	A $1s$ core-hole determinant or ROKS state	Accurate K-edge energies and spectra
TrimCI+COO many-body theory	Sparse CI coefficients and orbital rotations	Large compression in determinant count

This dual usage is not merely terminological. In both cases, the orbital basis is treated as an active variational degree of freedom, and the orbital optimization is coupled to an explicitly constrained electronic ansatz. A plausible implication is that COO is best understood as a family of orbital-adaptive methods rather than a single formalism.

2. Variational structure in core-level spectroscopy

In the OO-DFT/X2C formulation for spectroscopy, the simplest $\Delta$SCF approach targets a single-determinant wavefunction $\Phi^*$ in which one or two $1s$ molecular orbitals on the chosen atom is emptied, and one electron is placed in a chosen virtual (Cunha et al., 2021). The total energy functional is

$$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$

where $\hat H_{\rm X2C}$ is the spin-free X2C one-electron operator, $\hat J[\rho^*]$ is the Coulomb operator built from the density $1s\!\to\!\text{virtual}$0, and $1s\!\to\!\text{virtual}$1 is the chosen DFT exchange-correlation potential.

Because the orbitals $1s\!\to\!\text{virtual}$2 must remain orthonormal and must carry the desired occupation pattern, the formulation introduces a matrix of Lagrange multipliers $1s\!\to\!\text{virtual}$3 enforcing

$1s\!\to\!\text{virtual}$4

together with additional constraints $1s\!\to\!\text{virtual}$5 that enforce that a particular orbital on atom A remains empty. The corresponding Lagrangian is

$1s\!\to\!\text{virtual}$6

and the stationarity conditions

$1s\!\to\!\text{virtual}$7

yield the usual DFT-like Fock equations with the desired core-hole occupancy.

This formulation makes explicit that the spectroscopy version of COO is not a perturbative excited-state correction layered onto a fixed ground-state calculation. Rather, it is a constrained variational optimization in which the target core-excited occupancy is embedded directly into the orbital equations.

3. Relativistic Hamiltonian, orbital optimization, and practical construction

For elements heavier than Ne, the spectroscopy framework combines orbital optimization with a scalar-relativistic treatment based on the spin-free X2C model (Cunha et al., 2021). The construction begins from the four-component one-electron Dirac equation in a restricted-kinetic-balance basis,

$1s\!\to\!\text{virtual}$8

with $1s\!\to\!\text{virtual}$9, $U=e^\kappa$0, and $U=e^\kappa$1 the usual kinetic, nuclear-attraction, and overlap integrals, and $U=e^\kappa$2 the spin-free piece of $U=e^\kappa$3. A one-step exact two-component unitary decoupling then yields an effective two-component Hamiltonian

$U=e^\kappa$4

with

$U=e^\kappa$5

where $U=e^\kappa$6, $U=e^\kappa$7, and

$U=e^\kappa$8

The orbital optimization itself is expressed through an infinitesimal unitary rotation generated by an anti-Hermitian matrix $U=e^\kappa$9,

$1s$0

together with the excited-state energy gradient

$1s$1

and, optionally, the orbital-rotation Hessian

$1s$2

The workflow described for practical optimization uses quasi-Newton or trust-region methods rather than an explicit full Hessian. A convenient robust algorithm is square-gradient minimization, initialized with a guess that has the $1s$3 core hole on the target atom, for example by the maximum-overlap method IMOM. The loop proceeds by building $1s$4, $1s$5, and $1s$6; forming

$1s$7

computing the orbital-rotation gradient; proposing $1s$8 with an approximate Hessian such as BFGS; enforcing a trust region; optionally level-shifting the virtual block by $1s$9 $\Delta$0–$\Delta$1 to avoid collapse to the ground state; and rotating the orbitals. The stated convergence controls are $\Delta$2, for example $\Delta$3, and $\Delta$4.

The practical recipe is correspondingly specific: a radial grid of $\Delta$5 points and a Lebedev angular grid of $\Delta$6 points; a decontracted triple-$\Delta$7 “pcX-n” or core-valence Dunning set on the core-hole atom, such as aug-pcX-2 or cc-pCVTZ; aug-pcseg-1 on all other atoms; and core-hole targeting by IMOM followed by freezing that occupation throughout the SCF loop, or equivalently by adding a small penalty functional $\Delta$8 to the energy. Square-gradient minimization is stated to guarantee descent of $\Delta$9 and avoid “variational collapse.”

4. COO as a co-optimized sparse-CI orbital basis

In the many-body setting, COO is defined by simultaneous variation of a sparse CI expansion and the single-particle basis (Zhang et al., 21 May 2026). Starting from any initial orthonormal orbitals $\Phi^*$0, one applies a unitary rotation $\Phi^*$1 generated by a real antisymmetric matrix $\Phi^*$2, with $\Phi^*$3 and $\Phi^*$4 independent angles, to define

$\Phi^*$5

The CI ansatz over a selected set of determinants $\Phi^*$6 is then

$\Phi^*$7

and the variational objective is the Rayleigh quotient

$\Phi^*$8

subject to $\Phi^*$9. Equivalently,

$1s$0

where $1s$1 and $1s$2 are the $1s$3- and $1s$4-RDMs of $1s$5.

The baseline sparse ansatz is TrimCI, which builds a variational space $1s$6 of $1s$7 determinants by alternately “expanding” in large Hamiltonian couplings $1s$8 and “trimming” by retaining the largest weight configurations after block-diagonal and global diagonalization. Its wavefunction is

$1s$9

At fixed orbitals, stationarity with respect to the CI coefficients gives the projected eigenproblem

$$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$0

For the orbital variables, the generator

$$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$1

leads to the gradient

$$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$2

which can be written in closed form from the $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$3- and $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$4-RDMs.

The joint update uses BFGS on $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$5, but every line-search trial step immediately re-optimizes $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$6 via Davidson on the fixed core. The stated purpose is that the trial energy $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$7 sees the full coupled response rather than a linear surrogate. The algorithm is staged: a global COO phase with repeated TrimCI core construction and BFGS orbital updates; a local-refinement phase in which the determinant count is grown gradually; and an expansion phase in which $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$8 is frozen while the CI space is enlarged further, optionally with semistochastic PT2 correction. Practical notes include using a small initial core $$E[\Phi^*] = \bigl\langle\Phi^*\bigl|\hat H_{\rm X2C}+\hat J[\rho^*]+\hat V_{\rm XC}[\rho^*]\bigr|\Phi^*\bigr\rangle,$$9–$\hat H_{\rm X2C}$0, seeding with HF and random determinants, and preferring BFGS over L-BFGS and CG for the orbital optimization.

5. Accuracy, compactness, and asymptotic scaling

For K-edge spectroscopy with OO-DFT/X2C, the reported benchmark is that over a broad set of third-row K-edges (Ne, Si–Cl, Ar), ROKS or $\hat H_{\rm X2C}$1SCF + X2C + SCAN yields RMS errors of $\hat H_{\rm X2C}$2–$\hat H_{\rm X2C}$3 versus experiment, while conventional TDDFT routinely errs by $\hat H_{\rm X2C}$4–$\hat H_{\rm X2C}$5 for heavy-atom K-edges and must be post-shifted (Cunha et al., 2021). The abstract further states that OO-DFT/X2C reproduces experimental spectra quite well, sans empirical shifts for alignment, and that all steps remain $\hat H_{\rm X2C}$6 or at worst $\hat H_{\rm X2C}$7, with only a modest prefactor overhead of $\hat H_{\rm X2C}$8 relative to ground-state KS-DFT or TDDFT.

For the sparse-CI COO method, the central numerical claim is that the apparent parameter-heaviness of CI is strongly basis-dependent (Zhang et al., 21 May 2026). In a localized-MO basis, reaching $\hat H_{\rm X2C}$9 accuracy on $\hat J[\rho^*]$0 $\hat J[\rho^*]$1 requires

$\hat J[\rho^*]$2

whereas in the COO basis the same accuracy is reached with

$\hat J[\rho^*]$3

The paper describes this as a factor $\hat J[\rho^*]$4 compression in determinant count. At matched accuracy on the same cluster, TrimCI+COO is reported to be $\hat J[\rho^*]$5 more compact than the largest unrestricted-DMRG benchmark and $\hat J[\rho^*]$6 more compact with PT2. Across the iron-sulfur series, from $\hat J[\rho^*]$7 $\hat J[\rho^*]$8 to the P-cluster $\hat J[\rho^*]$9, TrimCI+COO is reported to be $1s\!\to\!\text{virtual}$00–$1s\!\to\!\text{virtual}$01 more compact than SU(2)-adapted DMRG with entanglement-minimized orbitals at matched accuracy.

Setting	Metric	Reported result
OO-DFT/X2C K-edge spectroscopy	RMS error vs experiment	$1s\!\to\!\text{virtual}$02–$1s\!\to\!\text{virtual}$03
OO-DFT/X2C vs TDDFT	Heavy-atom K-edge error	TDDFT routinely errs by $1s\!\to\!\text{virtual}$04–$1s\!\to\!\text{virtual}$05
$1s\!\to\!\text{virtual}$06 TrimCI+COO	Determinants at matched accuracy	$1s\!\to\!\text{virtual}$07 vs $1s\!\to\!\text{virtual}$08 in LMO basis
$1s\!\to\!\text{virtual}$09 vs UDMRG	Compactness at matched accuracy	$1s\!\to\!\text{virtual}$10; $1s\!\to\!\text{virtual}$11 with PT2
Iron-sulfur series vs SU(2)-DMRG+EMO	Compactness at matched accuracy	$1s\!\to\!\text{virtual}$12–$1s\!\to\!\text{virtual}$13

The many-body paper also gives an asymptotic framing: if $1s\!\to\!\text{virtual}$14, then orbitals that increase $1s\!\to\!\text{virtual}$15 improve the asymptotic scaling of $1s\!\to\!\text{virtual}$16 required for a fixed $1s\!\to\!\text{virtual}$17. This suggests that COO is not only a finite-size compression device but also a means of changing the effective convergence law of sparse CI.

6. Factorization of gains, limitations, and open directions

A distinctive analysis in the many-body COO work is the separation of the total advantage into an orbital-basis gain and an ansatz gain using a tunable Hubbard-on-graph model on $1s\!\to\!\text{virtual}$18 sites at half-filling with $1s\!\to\!\text{virtual}$19 and topology parameter $1s\!\to\!\text{virtual}$20 (Zhang et al., 21 May 2026). At fixed $1s\!\to\!\text{virtual}$21, the paper defines

$1s\!\to\!\text{virtual}$22

so that

$1s\!\to\!\text{virtual}$23

As $1s\!\to\!\text{virtual}$24 grows from $1s\!\to\!\text{virtual}$25, $1s\!\to\!\text{virtual}$26 rises from $1s\!\to\!\text{virtual}$27 and $1s\!\to\!\text{virtual}$28 from $1s\!\to\!\text{virtual}$29, giving total $1s\!\to\!\text{virtual}$30 from $1s\!\to\!\text{virtual}$31. Within the paper’s interpretation, the ansatz gain captures multi-center entanglement that resists MPS localization.

The limitations are likewise explicit. In the many-body setting, accuracy remains variational; very large $1s\!\to\!\text{virtual}$32 expansions can be costly, though COO cuts $1s\!\to\!\text{virtual}$33 by $1s\!\to\!\text{virtual}$34; line-search re-diagonalization requires the core to remain small; selected-CI matvecs at $1s\!\to\!\text{virtual}$35 require distributed Davidson; and orbital rotations cost $1s\!\to\!\text{virtual}$36 per gradient step, which may dominate for very large active spaces. The proposed future directions are to apply COO to larger targets such as the FeMo cofactor and P-cluster refinement, combine COO with tensor networks, coupled-cluster, and neural quantum states, investigate hybrid schemes in which small COO cores seed VQE or quantum Monte Carlo, explore automated basin discovery, and extend the multi-node GPU Davidson to $1s\!\to\!\text{virtual}$37.

In the spectroscopy setting, the paper reports that it also explored K and L edges of $1s\!\to\!\text{virtual}$38 transition metals to identify limitations of the OO-DFT/X2C approach in modeling the spectra of heavier atoms (Cunha et al., 2021). More broadly, the spectroscopy workflow makes clear that the method’s success depends on maintaining the target core occupancy during optimization and on controlling variational collapse through occupation freezing, penalty functionals, trust regions, or level shifts.

Taken together, these two lines of work establish COO as a general orbital-adaptive paradigm. In one application, it produces core-hole orbitals for relativistic K-edge spectroscopy at roughly ground-state DFT cost; in the other, it uses a small variational core to learn an orbital basis that can compress sparse CI by three to five orders of magnitude. A plausible unifying interpretation is that COO transfers correlation or excitation specificity from the many-electron expansion into the orbital manifold itself, thereby changing which degrees of freedom must be represented explicitly.