Annealing-Augmented Variants (T-GCG)

• T-GCG is a family of optimization methods that inject annealing-inspired stochasticity, temperature scheduling, and quantum sampling into standard algorithms.
• It improves exploration in rugged, non-convex landscapes across machine learning, combinatorial optimization, adversarial attacks, and materials computation.
• Key implementations include temperature-augmented gradient attacks, quantum-annealer genetic algorithms, and directed mutation strategies demonstrating superior performance over classical heuristics.

Annealing-Augmented Variants (T-GCG) encompass a family of algorithms and encodings that inject annealing-inspired stochasticity, temperature scheduling, or quantum sampling into core optimization routines. Across machine learning, combinatorial optimization, adversarial attack, and materials computation, T-GCG instantiations have emerged as competitive and often unifying frameworks for navigating rugged, non-convex landscapes. Representative instances include temperature-augmented gradient-based adversarial attacks for LLMs, quantum-annealer-embedded genetic algorithms for combinatorial search, annealing-inspired STE variants for quantized deep networks, and Hamiltonian mappings of physical domain variants for quantum annealing hardware. Core to these approaches is the exploitation of temperature or annealer-control parameters to facilitate exploration, combat premature convergence, and structure the scheduling of randomness in a systematically decaying fashion.

1. Fundamental Principles and Mathematical Constructs

T-GCG approaches leverage the principles of simulated or quantum annealing—controlled stochasticity or quantum tunneling guided by an annealing schedule—to augment baseline heuristics. In discrete combinatorial settings, e.g., the Chinese Postman Problem (CPP) and multi-agent routing, the optimization objective is formulated as a quadratic unconstrained binary optimization (QUBO) problem. The QUBO encodes constraints and costs through binary variables with penalty-weighted quadratic terms:

$$E(x, s) = M(x) + P_{req}\,C_{req}(x,s) + P_{adj}\,C_{adj}(x) + P_{one}\,C_{one}(x) + \cdots$$

where $M(x)$ is the cost function (e.g., postman tour length), and each $C_{\star}$ is a constraint. For neural optimization (QNNs), T-GCG leverages temperature-parameterized additive noise on quantizer inputs, with layer-wise or blockwise annealing schedules to guide training dynamics (Spallanzani et al., 2022). In adversarial prompting for LLMs, T-GCG introduces coordinate-wise temperature stochasticity in token sampling and acceptance of adversarial candidates using Gibbs measures (Tan et al., 30 Aug 2025).

2. T-GCG in Quantum Annealing and Combinatorial Optimization

Quantum annealing-based T-GCG solves variants of the CPP and related multi-agent routing problems by mapping the combinatorial constraints to QUBO and embedding the resultant Ising Hamiltonians on hardware such as D-Wave. Model variants include:

• Undirected, Directed, Windy, and K-Postman Variants: All edge types are encoded via directed copies; directionally varying (windy) weights are captured by asymmetric cost matrices; k-postman formulations introduce additional agent indices and slack variables.
• Constraint Reductions: Exploiting symmetries halves variable counts for undirected graphs; pruning based on fixed start/stop vertices sharply reduces the search space; terminal-vertex encodings with rest bits substitute costly repeat-edge constraints.

Parameter tuning on QA devices (D-Wave 2000Q, Advantage) involves optimization of penalty weights ($P_{req}\approx 35\times$maxW, $P_{adj}\approx 75\times$maxW, $P_{one}\approx 40\times$maxW), chain strengths, and anneal times. Performance comparison with classical heuristics (greedy, tabu, simulated annealing) and hybrid schemes (greedy+QA, qbsolv) shows QA hybrids matching or outperforming state-of-the-art approaches for medium/large-scale problems. For example, in closed-CPP with node counts 8–10, hybrids achieve 100% optimality, whereas direct quantum annealing alone achieves much lower rates (4.5–2.3%) (Pion et al., 2022).

3. Annealing-Augmented Algorithms in Machine Learning and Robust AI

In quantized neural network (QNN) optimization, T-GCG is instantiated as Additive Noise Annealing (ANA). Here, quantizer surrogates are interpreted as convolutions with temperature-controlled noise, and each network layer follows a scheduled decay of its noise (“temperature”):

$$\phi_{T}(x) = \mathbb E_{\xi\sim p(\cdot;T)}[q(x+\xi)] = (p(\cdot;T) * q)(x)$$

Synchronization of annealing—especially with partitioned progressive cooling schedules—ensures that convergent approximations to discontinuous quantized mappings are obtained in the limit $T_\ell\to 0$. CIFAR-10 experiments demonstrate that accuracy is highly sensitive to annealing schedule synchronization (partition and progressive decay of $T_\ell$), whereas the qualitative type of surrogate regularization (uniform, logistic, Gaussian) is of secondary importance. The temperature-Gaussian convolutional gradient (T-GCG) variant (Gaussian noise) matches static straight-through estimator performance when schedules are synchronized (Spallanzani et al., 2022).

4. Annealing-Augmented Adversarial Attacks on LLMs

The T-GCG attack for LLMs augments the deterministic Greedy Coordinate Gradient (GCG) method by introducing two annealing-like mechanisms:

• Temperature $T_1$: Controls stochasticity in token choice, exponentially cooled across epochs ($M(x)$0).
• Temperature $M(x)$1: Governs Gibbs-distributed acceptance of candidate adversarial suffixes, typically tied to the current surrogate loss ($M(x)$2).

This double temperature approach enables diversified candidate exploration and stochastic acceptance to escape local minima. When evaluated on AdvBench and coding-related prompts across multiple LLMs (Qwen2.5-0.5B, LLaMA-3.2-1B, GPT-OSS-20B), T-GCG increases surface-level attack success rates under prefix-refusal heuristics (e.g., LLaMA-3.2-1B, $M(x)$3 rises from 68.7% to 73.6%), but yields marginal or negative gains under semantic evaluation by GPT-4o. This suggests the annealing augmentation mainly increases variability and “trick-like” outputs rather than deep semantic jailbreaks (Tan et al., 30 Aug 2025).

5. Genetic Quantum Annealing and Directed Mutation

T-GCG in meta-heuristic search contextifies genetic algorithms within a quantum annealing environment. Each individual’s genotype is encoded as a vector of annealer biases; via nepotism (fitness-inheritance) and quantum polyandry (inter-individual Ising couplings), the system exploits quantum hardware for directed mutational search. Annealing schedule parameters (e.g., $M(x)$4, total anneal time, ramp durations) are carefully chosen and hardware-specific (e.g., D-Wave Advantage). On multimodal and Diophantine combinatorial benchmarks, this hybrid T-GCG strategy yields up to an order-of-magnitude reduction in fitness function calls, reflecting sharper fitness jumps and superior sampling of high-quality solutions relative to classical GA baselines (Abel et al., 2022).

6. Quantum Annealing for Multivariant Selection in Physical Systems

In microstructural materials science, T-GCG manifests as mapping elastic-variant selection problems (e.g., martensite multivariant equilibrium) to dense Ising models tractable on quantum annealers. Variant configurations are encoded by spin arrays, and the total elastic energy is expressed in quadratic Hamiltonian form. The practical annealing protocol comprises hardware annealing schedules (D-Wave platforms), repeated sampling, and classical pre/post-processing for large-scale problems. This approach delivers exact global minimization for systems with thousands of grains and variants, providing >$M(x)$5 computational speedup over classical simulation for dense, frustrated networks (Santos et al., 2024).

7. Synthesis, Limitations, and Outlook

Across domains, T-GCG approaches unite annealing-infused optimization, variable pruning, hardware-aware scheduling, and hybridization with classical heuristics in pursuit of global or near-global optima. The unifying mechanism is the systematic application of controlled stochasticity or quantum fluctuations to encourage exploration over premature exploitation, supported by a theoretical and experimental emphasis on schedule synchronization, variable compactness, and hardware-specific constraint management.

Empirical results consistently show that T-GCG variants are competitive or superior to their vanilla counterparts and classical algorithms, especially in combinatorially hard or rugged landscapes. However, the realized gains depend crucially on appropriate schedule tuning, synchronization across system components, and, in supervised adversarial LLM tasks, the semantics of evaluation criteria.

A plausible implication is that further advances may be achieved by developing adaptive or learning-based annealing schedules, expanding T-GCG into transfer or black-box optimization settings, and integrating landscape-smoothing adversarial defenses inspired by annealing's global search properties. As quantum and classical annealing-inspired methods mature, T-GCG is likely to remain a central paradigm for high-dimensional stochastic search and physically inspired computation.