JUDGE Action: Balancing Judgment and Data

• JUDGE Action is a formal decision protocol that integrates prior judgment with data-driven adjustments using confidence intervals.
• It employs hypothesis testing to modulate deviations from an initial judgment, ensuring that changes are statistically justified.
• The framework applies to fields like finance and policy by balancing intuition with quantifiable risk aversion through the confidence level α.

A JUDGE Action refers, in the context of “Deciding with Judgment” (Manganelli, 2019), to a formalized decision protocol that begins with an a priori judgmental decision and modulates deviations from it by reference to a statistical confidence interval. Rather than seeking the empirically optimal action regardless of prior beliefs, the JUDGE Action systematically incorporates the decision maker’s risk aversion and limits the probability of underperforming relative to the original judgment. The result is a statistically admissible rule that can be tuned to balance personal, intuitive decisions against the evidence provided by data, with direct implications for areas such as finance, policy, and empirical sciences.

1. Judgmental Decision and Formalization

The procedure is initiated by defining a “judgment” as a pair $A = \{\tilde{a}, \alpha\}$, where $\tilde{a}$ is the judgmental (prior, unoptimized) decision, and $\alpha \in (0, 1)$ is the chosen confidence level. In practical terms, $\tilde{a}$ typically reflects the action arrived at by domain knowledge or gut intuition—e.g., an investor retaining all wealth in cash.

The paper does not model the origins of $\tilde{a}$, instead taking it as a fixed input to the subsequent statistical protocol. The crucial insight is that $\alpha$ encodes statistical risk aversion, thereby quantifying the decision maker’s willingness to override prior judgment in light of new data.

2. Statistical Decision Rule: Hypothesis Testing Relative to Judgment

Given a loss function $L(\theta, a)$—for instance, $L(\theta, a) = -a\theta + \frac{1}{2}a^2$ in portfolio choice with mean $\theta$—the JUDGE Action tests whether the gradient at $\tilde{a}$ is statistically justified for change.

Specifically, the process works as follows:

• Compute the sample gradient (replacing unknown parameter $\theta$ with estimate $X$), yielding $-X + \tilde{a}$.
• Conditional on the sign of the observed gradient:
  • If $-X + \tilde{a} \leq 0$, test $H_0: -\theta + \tilde{a} \geq 0$ versus $H_1: -\theta + \tilde{a} < 0$.
  • If $-X + \tilde{a} > 0$, test $H_0: -\theta + \tilde{a} \leq 0$ versus $H_1: -\theta + \tilde{a} > 0$.
• If the null is not rejected, $\tilde{a}$ is retained.
• If the null is rejected, increment $\tilde{a}$ to the nearest boundary point of the $(1-\alpha)$ confidence interval—i.e., move just sufficiently far for the null to remain un-rejected.

Formally, letting $c_{(\alpha/2)}$ denote the standard normal quantile, the action is

$$\delta^A(X) = \mathbb{I}(-X + \tilde{a} \leq 0)\cdot \delta_-^A(X) + \mathbb{I}(-X + \tilde{a} > 0)\cdot \delta_+^A(X)$$

where, e.g.,

$$\delta_-^A(X) = \tilde{a}[1 - \psi_-^A(X)] + (X + c_{(\alpha/2)})\psi_-^A(X)$$

and $\psi_-^A(X)$ is the test function indicating rejection of $H_0$. Thus, the new action is $\hat{a} = \tilde{a} + \hat{\Delta}$ with $\hat{\Delta}$ set to the appropriate quantile-based increment.

3. Confidence Interval, Statistical Risk Aversion, and Safety Bound

A defining property is that the procedure constrains the probability that the revised action performs worse than the prior judgment. Explicitly,

$$P_\theta\left(L(\theta, \delta^A(X)) > L(\theta, \tilde{a})\right) \leq \alpha$$

The confidence level $\alpha$ plays the role of a “coefficient of statistical risk aversion.” As $\alpha \to 0$, almost all decisions remain at $\tilde{a}$; as $\alpha \to 1$, the rule approaches the data-driven maximum likelihood action. This calibration allows fine-grained control between unsafe statistical optimization and excessive conservatism.

Practically, the JUDGE Action rules out large, unjustified movements away from the prior, thereby ensuring a statistically bounded downside risk.

4. Elicitation via Laboratory (Urn) Experiments

To practically determine an individual's risk aversion coefficient $\alpha$, the paper proposes laboratory analogs in the style of Ellsberg urns. Participants are shown two urns representing the null (worst-case) and alternative (favorable) scenarios—e.g., urns with differently weighted likelihoods of “loss” (black balls) or “win” (red balls).

Subjects select among a grid of bets corresponding to different adverse event probabilities. Risk aversion is thus revealed behaviorally: bets favoring high certainty (low $\alpha$) indicate high risk aversion. This aligns the abstract statistical parameter directly with observable decision-making under uncertainty.

5. Application to Asset Allocation

The JUDGE Action methodology is concretely instantiated for asset allocation, considering an investor whose prior action $\tilde{a}$ is total cash holding. Given sample returns, the econometrician evaluates the gradient of loss and advises adjustment only if hypothesis testing, per the JUDGE Action process, statistically supports the deviation.

Hence, the investor’s allocation shifts to the nearest boundary of the confidence interval, i.e., $\hat{a} = X + c$, where $c$ is the appropriate quantile, if the data provide strong evidence. The investor thus guarantees with probability at least $1-\alpha$ that her new allocation will not fare worse than her original judgment.

The empirical benefit is strict downside protection: compared to ML or Bayesian strategies that can exhibit high out-of-sample volatility, the JUDGE Action rule mitigates catastrophic underperformance by restricting changes to those justified by robust statistical evidence.

6. Decision Protocol and Policy Implications

The JUDGE Action represents a structured approach to integrating expert judgment and statistical inference. Rather than disregarding prior beliefs or subsuming all risk to the data, deviations from the starting point are allowed only when justified at the chosen risk tolerance level $\alpha$.

Policymakers or practitioners can use this protocol to blend empirical evidence and subjective caution. For instance, a conservative administrator might select $\alpha$ near zero, resulting in data-driven deviations only when evidence is overwhelming, whereas a low-aversion administrator might select higher $\alpha$ and be more responsive to signals in the data.

This dual anchoring in subjectively meaningful actions and transparent statistical safeguards provides a blueprint for robust decision making in domains where controlling downside risk is paramount.

---

In summary, the JUDGE Action protocol as delineated in (Manganelli, 2019) formalizes the intelligent hedging of prior judgments with evidence from statistical analysis. Movements away from the starting action are not made in pursuit of empirical optimality alone, but are conditioned by the explicit confidence with which the available information contradicts the original choice. The decision maker’s risk aversion, codified by $\alpha$, is central in determining both the willingness to adapt and the exposure to potential losses, yielding a flexible, theoretically grounded, and operationally transparent decision rule.