Regret Circuits: Neural and Computational Mechanisms

• Regret circuits are modular architectures found in both computational learning and neuroscience, combining local regret minimizers and neural pathways to guide adaptive behavior.
• In computational settings, they employ operations like Cartesian products, affine images, and convex hulls to decompose losses and ensure sublinear cumulative regret.
• In neural contexts, regret circuits involve prefrontal pathways projecting to the lateral habenula, modulating dopamine and serotonin levels to drive avoidance learning.

Regret circuits are modular architectures, appearing in both computational learning theory and systems neuroscience, that encode regret signals to guide action adaptation and avoidance of suboptimal or harmful decisions. In computational contexts, regret circuits are directed acyclic graphs composed of black-box regret minimizers over convex sets, supporting compositional design of regret-minimizing algorithms for structured domains. In systems neuroscience, “regret circuits” refer to anatomically and functionally identified pathways—specifically prefrontal projections to the lateral habenula (LHb)—that transmit signals related to evaluation of aversive, suboptimal, or regrettable outcomes, modulating neuromodulatory systems and shaping future policy. Both interpretations share the formal property that the magnitude or effect of regret is dynamically composed from local sources and influences global choice updating, either through synaptic or algorithmic means (Vadovičová, 2014, Farina et al., 2018).

1. Formalism of Computational Regret Circuits

In computational learning, a regret circuit is defined as a directed acyclic graph whose nodes are (black-box) regret minimizers over compact convex sets, and wires that carry sequences of decision vectors and loss functions. At each round $t$, the regret circuit distributes global losses to local minimizers and aggregates their decisions to produce an overall decision for the composite domain. The formal guarantee is sublinear cumulative regret for the composed device: $$R^T_{(\mathcal C, \mathcal L)} = \sum_{t=1}^T \ell^t(x^t) - \min_{\hat x \in \mathcal C} \sum_{t=1}^T \ell^t(\hat x) = o(T)$$ where $\mathcal C$ is a composite convex set constructed by operations such as product, affine map, Minkowski sum, or convex hull from simpler sets (Farina et al., 2018).

Circuit Operations

• Cartesian Product: For $\mathcal C = \mathcal A \times \mathcal B$, local minimizers over $\mathcal A$ and $\mathcal B$ are wired in parallel; regret decomposes additively: $R^T_{A \times B} = R^T_A + R^T_B$.
• Affine Image: For $\mathcal C = T(\mathcal A)$, decisions are pulled back and losses pushed forward; regret is inherited.
• Minkowski Sum: For $\mathcal C = \mathcal A + \mathcal B$, equivalent to product followed by sum.
• Convex Hull: For $\mathcal C = \mathrm{conv}(\mathcal A, \mathcal B)$, decisions are mixed via a simplex minimizer, and regret is bounded by the sum of simplex and component regrets.
• Projection (Curtailing): To enforce convex constraints, circuits incorporate nodes that project decisions via Bregman divergence and distribute subgradient-based loss to child minimizers.

These operations enable scalable, modular construction of regret-minimizing algorithms for composite structured domains, notably extensive-form games via treeplexes (Farina et al., 2018).

2. Regret Circuits in Reinforcement Learning and Game Theory

Extensive-form games and strategic settings demand the coordinated minimization of regret across hierarchically structured spaces. Regret circuits provide a recursive framework (as in CFR—Counterfactual Regret Minimization) by which local simplex regret minimizers at individual information sets or actions are composed using convex hull and product nodes corresponding to the treeplex structure of strategies. Time-averaged strategies produced by such circuits are guaranteed to approximate Nash equilibria at a rate inversely proportional to the square root of iterations, matching the best known rates for local minimizers (Farina et al., 2018).

Counterfactual Regret Interpretation

In CFR, the circuit-level loss dispatched to each simplex node is the instantaneous counterfactual regret vector for its associated information set. The global regret bound of the composed circuit is thus obtained by summing and maximizing over local regrets, with the modular structure enabling tractable scaling to large extensive-form domains.

3. Anatomical and Functional Bases of Neural Regret Circuits

Vadovičová’s work establishes direct anatomical projections from both affective and cognitive prefrontal cortex to the lateral habenula (LHb) in humans, mapping a “neural regret circuit” that integrates counterfactual and value-based evaluations to drive avoidance learning (Vadovičová, 2014). Diffusion tractography reveals robust corticohabenular pathways from:

• Dorsal anterior cingulate cortex (dACC; BA 24/32)
• Pregenual ACC (pgACC; BA 24/32 anterior to the genu)
• Anterior insula (AI) and adjacent caudolateral/lateral orbitofrontal cortex (OFC; BA 47)
• Lateral prefrontal cortex (BA 8, 9, 10, 44/45, 46)

Notably, no direct projections from medial OFC or ventral ACC to LHb are observed, supporting a segregation between circuits encoding value of regrettable events and reward (Vadovičová, 2014).

The LHb, excited by predicted or received aversive outcomes, imposes net inhibition on midbrain dopamine and serotonin neurons, directly and via the rostromedial tegmental nucleus (RMTg). This suppression drives avoidance learning and behavioral inhibition by modulating the balance of D1- and D2-pathway plasticity in striatum.

4. Computational Models of Regret Signal Integration

Vadovičová et al. propose a computational model in which each cortical input $i$ to the LHb is assigned a weight $w_i$ (proportional to tract density and mean fractional anisotropy). The overall dip in neuromodulator output, and thus the magnitude of regret updating, is formalized as: $$\Delta V = -\sum_{i} w_{i} \cdot \text{Outcome}_{i}^{\text{bad}}$$ where $\text{Outcome}_{i}^{\text{bad}}$ captures the dimension-specific prediction error (e.g., loss, social rejection), and the negative sign encodes devaluation. The suppression of dopamine ($\Delta \mathrm{DA}$) and serotonin ($\Delta 5\text{-HT}$) is modeled as: $$\Delta\mathrm{DA} = -\lambda_{\mathrm{DA}} \sum_{i} w_{i} \cdot \text{Outcome}_{i}^{\text{bad}}$$

$$\Delta 5\text{-HT} = -\lambda_{5\text{-HT}} \sum_{i} w_{i} \cdot \text{Outcome}_{i}^{\text{bad}}$$

where the $\lambda$ parameters scale the LHb-to-midbrain effect. Thus, stronger prefrontal-LHb connectivity yields stronger avoidance learning (Vadovičová, 2014).

5. Distinction Between Regret Circuits and Generic Punishment Systems

An essential feature distinguishing neural regret circuits from generic punishment pathways is the inclusion of pregenual ACC and cognitive PFC projections (not just affective evaluators). These regions encode counterfactuals—signals about “what might have been” and internal models demarcating suboptimal strategies. Thus, the regret circuit does not simply condition aversion to external punishers; it sculpts choice policies and strategic representations to minimize future regret by de-selecting both harmful actions and ineffective internal strategies, models, or plans (Vadovičová, 2014).

Concomitant anatomical observations, such as the absence of LHb projections from reward-specific regions (vACC, mOFC), reinforce the selectivity of this circuit for encoding and teaching avoidance of regret rather than generic punishment or unrewarded actions.

6. Implementation and Complexity of Computational Regret Circuits

Regret circuits are assembled according to the decomposition tree of convex set constructors. Each leaf minimizer (e.g., simplex RM/Hedge) operates at $O(n_i)$ per iteration, and combinators (product, convex hull, projection) induce only affine or linear-time costs in the decision-space dimension. Projections for enforcing convex constraints can be performed via Euclidean or Bregman divergence, maintaining iteration complexity that is linear or quasilinear in total problem size. The modular nature allows black-box minimizers to be wired together with end-to-end $O(\sqrt{T})$ regret guarantees (Farina et al., 2018).

Computational Procedure for Regret Circuits

Stage	Operation	Complexity per Iteration
Forward pass	Local decision, combinatorial aggregation (product/hull/proj/affine)	$O(n)$–$O(n\log n)$
Loss feedback	Loss decomposition and backpropagation via circuit rules	Linear in # of combinators
Update	Each minimizer updates state with received loss	$O(n_i)$ per minimizer

This construction underpins state-of-the-art scalable learning in structured games and optimization.

7. Broader Implications and Integrative Perspectives

The convergence of computational and neural models of regret circuits highlights deep analogies between biological and artificial systems. In both domains, regret is not solely a function of immediate pain or loss, but encodes teacher signals for policy refinement, counterfactual evaluation, and strategic avoidance of suboptimal actions or models. Prefrontohabenular projections reflect this dual role, integrating affective and cognitive assessments into a unified circuit for adaptive decision-making. The compositionality of regret minimization in algorithmic frameworks mirrors the distributed, modular architecture observed anatomically and functionally in the human brain, suggesting a general principle for action selection in uncertain and structured environments (Vadovičová, 2014, Farina et al., 2018).