Rainbow Quantum-Inspired Algorithm

• Rainbow Quantum-Inspired Algorithm is a hybrid deep reinforcement learning method that combines a ring-topology variational quantum circuit with the Rainbow DQN framework to enrich feature representation.
• It retains Rainbow’s key techniques such as distributional Q-learning, dueling decomposition, Double DQN, prioritized replay, noisy exploration, and n-step returns while leveraging quantum superposition and entanglement.
• The algorithm improves approximation in NP-hard human resource allocation problems, demonstrating up to 26.8% normalized makespan reduction on benchmark instances.

Rainbow Quantum-Inspired Algorithm denotes, in the formulation introduced by the Variational Quantum Rainbow DQN (VQR-DQN), a hybrid deep reinforcement-learning method that integrates a ring-topology variational quantum circuit into Rainbow Deep Q-Network for combinatorial resource allocation. The algorithm retains Rainbow’s distributional Q-learning, dueling decomposition, Double DQN, prioritized experience replay, noisy exploration, and n-step returns, while replacing part of the classical function approximator with a variational quantum ansatz that produces superposed and entangled feature representations. In the reported instantiation, the method is applied to the human resource allocation problem (HRAP), modeled as a Markov decision process with masked combinatorial actions and evaluated on four HRAP benchmarks (Nguyen et al., 5 Dec 2025).

1. Conceptual basis and scope

The defining idea is a hybrid architecture in which a classical preprocessing pipeline transforms the high-dimensional HRAP state vector through noisy dense layers and maps the result to gate angles for a ring-topology variational quantum circuit. The VQC then returns quantum expectation features that feed a dueling, distributional Rainbow head. In this sense, the algorithm is “quantum-inspired” because the representational class used by the value function is driven by a variational quantum ansatz capable of generating superposed and entangled feature representations that can, in principle, span richer function families than shallow classical networks, thereby improving approximation of value distributions over high-dimensional combinatorial state-action spaces. In the reported design, the VQC is used as a feature extractor between noisy dense layers and the dueling distributional head, and the paper attributes better sample efficiency and final performance on HRAP benchmarks to this placement (Nguyen et al., 5 Dec 2025).

This formulation preserves the algorithmic structure of Rainbow rather than replacing it. Distributional Q-learning remains the value-learning objective; dueling decomposition separates value and advantage streams; Double DQN is retained for target formation; prioritized experience replay controls sample selection; noisy linear layers replace $\epsilon$-greedy exploration; and n-step returns are used in target construction. A plausible implication is that the quantum component is not presented as a standalone control policy, but as a function-approximation module inserted into an otherwise standard, value-based Rainbow pipeline.

The scope of the method is explicitly combinatorial. Resource allocation is treated as NP-hard due to combinatorial complexity, and the motivating claim is that classical function approximators limit the representational power of deep reinforcement learning in such settings. The contribution is therefore not a generic quantum control architecture, but a specific Rainbow-based design for structured decision problems in which state variables couple capability, timing, and transition information.

2. HRAP as a masked combinatorial MDP

The human resource allocation problem is cast as an MDP

$M = (S, A, P, R, \gamma).$

The state vector $s$ is the concatenation of officer capability matrices, event occurrence times, and a transition matrix. Concretely, for officers $o = 1,\dots,O$, capability matrices satisfy $C_o \in \mathbb{Z}^{E \times T}$ with $C_{o,e,t}$ denoting the execution time for officer $o$ to complete task $t$ in event $e$; event occurrence times are $\Omega \in \mathbb{Z}^E$, sorted; and the transition matrix is $M = (S, A, P, R, \gamma).$0, symmetric with zero diagonal, where $M = (S, A, P, R, \gamma).$1 is travel time between events $M = (S, A, P, R, \gamma).$2 and $M = (S, A, P, R, \gamma).$3. The flattened state is

$M = (S, A, P, R, \gamma).$4

with

$M = (S, A, P, R, \gamma).$5

Actions are combinatorial assignments

$M = (S, A, P, R, \gamma).$6

with $M = (S, A, P, R, \gamma).$7, $M = (S, A, P, R, \gamma).$8, and $M = (S, A, P, R, \gamma).$9. The size of the joint assignment space is

$s$0

and the experiments instantiate this as $s$1, $s$2, $s$3, and $s$4 depending on the benchmark. Transitions update ongoing schedules, officer locations and availability, event progress, and time accounting (Nguyen et al., 5 Dec 2025).

Reward shaping is defined toward minimizing the maximum completion time, expressed as normalized makespan. The normalization constant is

$s$5

and the reward is

$s$6

Once normalized, rewards lie in a bounded negative range. Bellman optimality is written as

$s$7

A practically important aspect is feasibility control through masking. Infeasible assignments, such as those involving an officer already occupied at time $s$8 or assignments that violate event timing constraints, are removed before the $s$9 step. On the reported benchmarks, $o = 1,\dots,O$0 is small enough to score all actions, and masks restrict the logits of invalid actions to $o = 1,\dots,O$1, ensuring feasibility. For larger action spaces, the paper describes factorization into sequential per-$o = 1,\dots,O$2 selections, hierarchical event-task-officer decomposition, and top-$o = 1,\dots,O$3 candidate sampling followed by refinement as practical extensions.

3. Hybrid VQC–Rainbow architecture

The variational quantum component uses a ring-topology VQC with $o = 1,\dots,O$4 qubits and $o = 1,\dots,O$5 layers. In each layer, the circuit applies Hadamard gates to all qubits, followed by parameterized single-qubit rotations and ring entanglement:

$o = 1,\dots,O$6

followed by

$o = 1,\dots,O$7

for all $o = 1,\dots,O$8. The cumulative unitary is

$o = 1,\dots,O$9

with

$C_o \in \mathbb{Z}^{E \times T}$0

Classical state encoding is performed by a dense layer that maps $C_o \in \mathbb{Z}^{E \times T}$1 to $C_o \in \mathbb{Z}^{E \times T}$2 angles, which parameterize the $C_o \in \mathbb{Z}^{E \times T}$3 and $C_o \in \mathbb{Z}^{E \times T}$4 gates. This is explicitly a data re-uploading style encoding via gate angles rather than amplitude encoding (Nguyen et al., 5 Dec 2025).

Measurement returns Pauli-$C_o \in \mathbb{Z}^{E \times T}$5 expectations,

$C_o \in \mathbb{Z}^{E \times T}$6

yielding a feature vector $C_o \in \mathbb{Z}^{E \times T}$7 that feeds the dueling distributional head. The paper notes that one could instead define action-observables $C_o \in \mathbb{Z}^{E \times T}$8 and compute

$C_o \in \mathbb{Z}^{E \times T}$9

or generate atom logits from measured observables; in the reported implementation, however, the VQC acts as a feature extractor feeding classical heads.

The end-to-end pipeline is

$C_{o,e,t}$0

Gradients through the quantum block are computed by parameter-shift:

$C_{o,e,t}$1

which the paper describes as allowing unbiased gradient computation on simulators or hardware with finite shots.

4. Rainbow learning mechanics, expressibility, and policy quality

The Rainbow head is distributional and uses the C51 categorical distribution on fixed support $C_{o,e,t}$2 over $C_{o,e,t}$3 with $C_{o,e,t}$4 atoms, commonly $C_{o,e,t}$5. Given per-action logits $C_{o,e,t}$6, predicted probabilities are

$C_{o,e,t}$7

The loss is the cross-entropy between the projected target mass and the predicted distribution:

$C_{o,e,t}$8

Projection to fixed support uses

$C_{o,e,t}$9

and

$o$0

Dueling decomposition is retained in the form

$o$1

and is applied at the distributional level, where the value stream produces atom logits $o$2 and the advantage stream produces per-action atom logits $o$3 (Nguyen et al., 5 Dec 2025).

Double Q-learning, prioritized replay, noisy layers, and n-step returns are all kept in their Rainbow forms. The Double DQN target is

$o$4

For n-step returns,

$o$5

with $o$6 in the value-based setting. Prioritized replay uses

$o$7

and importance weights

$o$8

Noisy layers replace $o$9-greedy exploration via

$t$0

The theoretical rationale offered for the quantum module is that expressibility and entanglement correlate with policy quality. Entanglement entropy for subsystem $t$1 is

$t$2

and expressibility is characterized through the KL divergence between the circuit-induced output distribution $t$3 and the Haar-induced distribution $t$4:

$t$5

Lower values indicate better coverage of the unitary-induced state manifold. The paper reports that ring topology yields higher expressibility and entanglement, including by the Meyer–Wallach measure, than star or linear variants at comparable depth, and that this correlates with improved HRAP performance. The stated intuition is that global entanglement induced by ring CNOTs helps capture long-range dependencies across officer-task-event features and transition times.

5. Empirical results and implementation

The reported experimental program evaluates four HRAP instances of increasing combinatorial complexity, all trained for 50,000 episodes under identical conditions, with best checkpoints evaluated on 200 test episodes. Performance is reported using normalized makespan reduction and average reward (Nguyen et al., 5 Dec 2025).

Benchmark	Joint action space
3O-2T-2E	$t$6
4O-3T-2E	$t$7
4O-3T-3E	$t$8
5O-4T-4E	$t$9

The paper reports approximately $e$0 normalized makespan reduction on the 3O-2T-2E benchmark versus a random baseline, with sustained gains across all settings. Relative to Double DQN and classical Rainbow DQN, the reported improvements are approximately $e$1–$e$2 depending on configuration. In topology ablations on 3O-2T-2E, the ranking is stated as Ring $e$3 All-to-All $e$4 Linear $e$5 Star, which the paper aligns with higher expressibility and entanglement of ring circuits.

Sensitivity analysis indicates that deeper circuits improve early convergence and final performance until diminishing returns emerge; excessive depth is described as risking barren plateaus and noise sensitivity, and shot noise is said to increase gradient variance. The hybrid pipeline introduces overhead through quantum simulation or inference latency, but this is mitigated by small $e$6 and shallow circuit depth. Training stability is attributed to the Rainbow components, while the VQC is said to improve sample efficiency; variance across seeds is reported as moderate.

Implementation details are unusually explicit. The optimizer is Adam; replay uses PER with $e$7 controlling prioritization strength and $e$8 annealed to $e$9; the target network is synchronized every $\Omega \in \mathbb{Z}^E$0 steps; the discount can be set, for example, to $\Omega \in \mathbb{Z}^E$1; and the quantum hyperparameters $\Omega \in \mathbb{Z}^E$2 and $\Omega \in \mathbb{Z}^E$3 are selected by ablation. Simulation is performed with TensorFlow Quantum, hardware runs use IonQ Aria-1 QPU, and finite-shot execution is described in the range of roughly $\Omega \in \mathbb{Z}^E$4–$\Omega \in \mathbb{Z}^E$5k shots, whereas simulators can use exact expectations. The repository identified in the paper contains an HRAP environment, VQC builder, Rainbow head, YAML configuration files, and a training entry point.

6. Limitations, future directions, and terminological distinctions

The limitations stated for the VQR-DQN formulation are principally those of NISQ hardware and hybrid RL overhead. The paper notes limited qubits, restricted connectivity, and significant noise; shallow ring circuits are described as feasible, but scaling to larger HRAP instances is said to require error mitigation and smarter encodings. The gains are reported as strongest in moderate complexity regimes, and for very large action spaces the paper suggests that policy- or actor-critic variants with quantum critics may be preferable. Additional directions identified in the paper include better encodings based on data re-uploading or problem symmetries, quantum kernels for value regression, IQN or QR-DQN distributional heads for finer quantile modeling, PPO with quantum feature extractors, and adaptive topology search guided by expressibility and entanglement metrics (Nguyen et al., 5 Dec 2025).

A persistent source of ambiguity is the word “Rainbow” itself. In arXiv literature it names several unrelated objects. In cryptanalysis, “Rainbow” refers to the multivariate quadratic signature scheme attacked by Q-rMinRank, where amplitude amplification accelerates repeated kernel-finding in MinRank-based key recovery (Cho et al., 2022). In password cracking, QIris applies Grover’s algorithm to a bucketed residue stage inside a classical rainbow-table pipeline, and a later hybrid classical-quantum attack on human passwords uses structured rainbow tables with distributed exact Grover search within buckets (Quan et al., 2024, Khajeian, 19 Jul 2025). In quantum finance, “rainbow” denotes multi-asset derivatives, and Iterative Quantum Amplitude Estimation is used to price best-of and worst-of contracts (Cibrario et al., 2024). These usages are terminologically adjacent but methodologically distinct.

For that reason, the Rainbow Quantum-Inspired Algorithm discussed here is most precisely understood as the VQR-DQN class of hybrid value-based reinforcement-learning architectures: a ring-topology variational quantum circuit embedded inside Rainbow DQN, trained with C51 projection, PER, noisy nets, n-step returns, and Double Q-learning, and evaluated on masked combinatorial HRAP instances. The broader implication suggested by the reported results is that quantum-enhanced function approximation may be useful for combinatorial resource allocation when the action space remains structured enough for feasible masking and value scoring.