Coherence Optimization in Quantum, ML & Signals

• Coherence Optimization is the process of maximizing a quantifiable coherence measure—balancing superposition, predictability, and alignment across quantum, signal, and machine learning domains.
• It employs mathematical formulations like relative entropy, l1-norm, and semidefinite programming to optimize basis selection and signal alignments under resource constraints.
• Practical applications include quantum state conversion, enhanced communication code designs, and improved neural representation, yielding significant precision and performance gains.

Coherence Optimization is the mathematical and algorithmic problem of maximizing or designing for “coherence” within a given system, where coherence refers to the quantifiable degree of superposition, predictability, alignment, or cross-mode compatibility in quantum physics, signal processing, machine learning, or communications contexts. The precise operationalization, objective function, and optimality criteria are domain-specific but share the underlying theme of optimizing an appropriate coherence measure under structural or resource constraints.

1. Quantum Coherence Optimization: Optimal Bases and Measures

In the quantum resource-theoretic setting, coherence is basis-dependent. For a fixed state $\rho$ in %%%%1%%%%-dimensional Hilbert space and a choice of incoherent reference basis, canonical coherence measures include the relative entropy of coherence $C_{\rm rel}(\rho)$, $l_1$-norm of coherence $C_{\ell_1}(\rho)$, robustness of coherence $C_R(\rho)$, coherence weight $C_w(\rho)$, and the skew-information based measure $C_{\rm sk}(\rho)$. Each of these quantifies off-diagonal magnitude or functional resourcefulness of quantum superpositions.

The central optimization problem is to determine, for each measure $C$, the maximal possible coherence for $\rho$ over all possible reference bases (i.e., unitarily related incoherent bases): $$C^{\max}(\rho) = \max_{U\in U(d)} C(U\rho U^\dagger)$$ Key analytical results (Hu et al., 2017):

• For measures $C_{\rm rel}$, $C_R$, $C_w$, and $C_{\rm sk}$, explicit closed forms for $C^{\max}(\rho)$ exist:
  • $C_{\rm rel}^{\max}(\rho) = \log d - S(\rho)$, where $S$ denotes von Neumann entropy.
  • $C_R^{\max}(\rho) = d\lambda_{\max} - 1$, $\lambda_{\max}$ being the largest eigenvalue of $\rho$.
  • $C_w^{\max}(\rho) = 1 - d\lambda_{\min}$.
  • $$C_{\rm sk}^{\max}(\rho) = 1 - (1/d)(\sum_i \sqrt{\lambda_i})^2$$.
  • For the $l_1$-norm, only tight upper bounds are available for generic mixed $\rho$ with $d\ge 4$.

Strikingly, these maxima are attained when the incoherent basis is mutually unbiased with respect to the eigenbasis of $\rho$. That is, the columns of any rescaled complex Hadamard matrix $H_d$ produce such optimal bases—mutually unbiased bases (MUBs). This insight links coherence optimization to the structure of complex Hilbert spaces and leads to efficient procedures: diagonalize $\rho$, apply $H_d$, and evaluate coherence for the resulting basis. This approach works for all major faithful measures (with pure states and qubits as special cases) (Hu et al., 2017).

2. Coherence Optimization in Quantum Resource Theories

Beyond basis maximization, coherence optimization addresses operational resource-theoretic tasks, such as single-shot and asymptotic conversion rates. Within the framework of incoherent operations (IC), two fundamental problems are coherence distillation (extracting maximally coherent states from arbitrary input) and coherence cost (creating an input from maximally coherent states).

The optimal single-shot probability $P_{\max}$ of transforming $|\psi\rangle$ to $|\phi\rangle$ under IC is given by

$$P_{\max}(|\psi\rangle \to |\phi\rangle) = \min_{1 \le \ell \le d} \frac{\sum_{i=\ell}^d |\psi_i|^2}{\sum_{i=\ell}^d |\phi_i|^2}$$

with deterministic (probability 1) conversion possible only if the initial basis population vector majorizes the target (Du et al., 2015). The corresponding coherence measures are constructed as convex roofs over concave, permutation-invariant, vanishing-on-basis probability functionals.

In the asymptotic regime, the operational rates are also given by coherence measures:

• Distillable coherence: $C_d(\rho) = C_r(\rho) = S(\Delta(\rho)) - S(\rho)$,
• Coherence cost: $C_c(\rho) = C_f(\rho)$ (coherence of formation, the minimum average $C_r$ over pure-state decompositions) (Winter et al., 2015).

No bound coherence states exist; for every $\rho$, $C_f(\rho) = 0$ iff $C_d(\rho) = 0$. Reversibility of distillation/formation holds exactly when $\rho$ is block-pure in the incoherent basis.

3. Coherence Optimization in Classical and Hybrid Signal Processing

The concept of coherence optimization generalizes to the design of vector sets—frames, codes, or matrices—with minimum mutual (cross-)coherence. For a set of unit vectors $A=[a_1,\dots,a_M]$ in $\mathbb{C}^N$, mutual coherence is

$\mu(A) = \max_{i\neq j} |\langle a_i, a_j\rangle|$

Minimizing $\mu(A)$ is central to compressed sensing, communication codebooks, and Grassmannian line-packing. The Welch bound provides a fundamental lower bound, with equality attained by equiangular tight frames (ETFs) when such frames exist.

Best Complex Antipodal Spherical Codes (BCASCs) are numerically constructed via force-equilibrium flows that drive code points to globally minimal-coherence configurations, efficiently approaching theoretical bounds (Zörlein et al., 2014). Analytical and fast approximate algorithms exploit the symmetries and antipodal equivalence of codewords.

For application to algebraic lattices, average and maximal coherence are computed explicitly for cyclotomic lattices, supporting their use in low-coherence, high-density signal designs (Fukshansky et al., 2020).

4. Optimization Algorithms and Computational Strategies

Methodologically, coherence optimization is frequently cast as a non-convex optimization or, for convex relaxations, as efficient convex or semidefinite programs. Key computational approaches include:

• Derivative-free SPA-optimization (Sequential Parameterization) for constrained maximization of coherence in attosecond photoionization, sequencing Fourier parameters only as needed while enforcing practical pulse constraints (Goetz et al., 2016).
• Trace-regularized convex programming for reconstructing mutual intensity matrices in optical coherence retrieval, which leverages nuclear norm regularization for noise suppression and low-rank recovery; solved efficiently via adaptive accelerated proximal gradient schemes with global convergence guarantees (Bao et al., 2017).
• Semidefinite programming (SDP) for computing maximal expectation values of observables and corresponding coherence witnesses under IO or MIO (maximally incoherent operations), with all operational coherence measures arising as specific instances (Tan et al., 2018).

Representative pseudocode for BCASC search and SPA-optimization, as well as step-by-step algorithms for quantum tomography with minimal data, are explicitly detailed in the corresponding sources.

5. Coherence Optimization in Machine Learning and Neural Models

In machine learning, coherence optimization underpins unsupervised self-improvement and better internal representation structure:

• In LLMs and semi-supervised learning, maximizing the predictive coherence of context-to-behavior mappings is equivalent to description-length regularization, yielding optimal generalization bounds when the regularizer is the pretrained prior (Qiu et al., 20 Jan 2026). All major self-improvement heuristics—debate, bootstrap, internal coherence maximization—emerge as special cases of this framework, which trades off fit and compressibility, optimizing both held-out accuracy and mutual predictability.
• Statistical Coherence Alignment (SCA) introduces a continuous tensor-field loss to align token representations in the embedding space, augmenting the standard LM objective with a penalty on deviations from global field structure; this improves perplexity, accuracy, and rare-word embedding quality, with convergence to high-coherence manifolds established via tensor-field contraction arguments (Gale et al., 13 Feb 2025).
• In neural topic modeling, a differentiable surrogate for semantic topic coherence is added to the variational objective to optimize interpretability directly, yielding models with higher NPMI coherence at fixed perplexity (Ding et al., 2018).
• PiDAn leverages a quadratic projected-coherence maximization to distinguish and mitigate poisoned vs authentic data in DNNs for backdoor detection by maximizing energy outside principal class subspaces; sample-wise optimal weights group by subspace, enabling effective, low-overhead defense schemes (Wang et al., 2022).

6. Application Domains and Experimental Results

Coherence optimization finds practical utility in diverse domains:

• Quantum metrology, where concentrating available resources into maximally coherent probe states directly enhances precision (Stratton et al., 3 Dec 2025).
• Quantum state tomography under data scarcity, where convex relaxations yield tight, tractable lower bounds on coherence measures (e.g., relative entropy of coherence) with computational cost insensitive to system size, outperforming full-tomography and polynomial-shadow schemes (Ding et al., 24 Oct 2025).
• Quantum control, where nearly perfect coherence/prescribed population ratios are realized for hole states in argon using SPA-optimization in the presence of physical constraints (Goetz et al., 2016).
• Neutron interferometry, where coherence (contrast) is optimized via refocusing internal diffraction phases through geometric blade adjustment in four-blade architectures, restoring near-unity contrast while preserving vibrational immunity (Nsofini et al., 2019).
• Wireless networks, where distributed group beamforming and node assignment algorithms maximize communication group coherence with low-complexity, yielding large power/SIR gains and extending operational range (Shi et al., 2019).

7. Limitations, Theoretical Boundaries, and Future Directions

Fundamental limitations persist:

• For the $l_1$-norm of coherence in dimensions $d\ge4$, only tight upper bounds are known for general mixed states, with closed-form maxima still elusive (Hu et al., 2017).
• Overoptimization and context dimension imbalances in ML can cause coherence “Goodharting” and collapse, mitigable through model regularization and early stopping (Qiu et al., 20 Jan 2026).
• In coherence concentration under symmetry constraints (unspeakable coherence), some global correlations remain non-extractable as local coherence (a formal no-go theorem), and only certain architectures permit unbounded amplification via concatenation (Stratton et al., 3 Dec 2025).
• Approximate-derivative coherence strategies like WASP are not exact, require small inter-sample distances, and scale poorly in high-dimensional neural contexts (Rakita et al., 26 Apr 2025).

Experimental and theoretical results converge on the importance of structure—MUBs in quantum, equiangular codes in classical, and field-theoretic manifolds in ML—in achieving optimal coherence. Ongoing work addresses scaling to massive systems, extending to nontrivial thermodynamic or multi-agent settings, and integrating value-alignment regularizers into the core coherence optimization framework.

---

References:

• Maximum coherence and MUBs in quantum resource theory (Hu et al., 2017)
• General construction of coherence measures and optimal state conversion (Du et al., 2015)
• Operational resource theory and asymptotic optimality (Winter et al., 2015)
• Coherence optimization in complex frames and codes (Zörlein et al., 2014)
• Scalable convex relaxation for sparse quantum coherence estimation (Ding et al., 24 Oct 2025)
• Self-improvement and description-length in ML via coherence optimization (Qiu et al., 20 Jan 2026)
• Statistical Coherence Alignment in LLMs (Gale et al., 13 Feb 2025)
• SPA-optimization for quantum control (Goetz et al., 2016)
• PiDAn for backdoor detection in DNNs (Wang et al., 2022)
• Neutron interferometer contrast optimization (Nsofini et al., 2019)
• Distributed beamforming and group coherence in wireless networks (Shi et al., 2019)
• Boundaries to extractable unspeakable coherence (Stratton et al., 3 Dec 2025)
• Trace-regularized coherence retrieval in optics (Bao et al., 2017)
• Coherence measure optimization for observable maximization (Tan et al., 2018)