Ex-Post Problem-Driven Evaluation Indices

• Ex-Post problem-driven evaluation indices are quantitative metrics that assess realized outcomes and mechanism performance by directly comparing observed results with ideal objectives.
• They employ measures such as welfare ratios, optimality gaps, and fairness indices to evaluate decisions in complex settings including auctions, resource allocation, and stochastic optimization.
• These indices balance rigorous performance assessment and computational tractability, addressing challenges like NP-hard verification and scenario reduction in dynamic mechanism design.

Ex-post problem-driven evaluation indices are quantitative metrics formulated to assess the quality, fidelity, or robustness of decisions, allocations, or policy outcomes after the resolution of uncertainty or the realization of a randomized mechanism. Unlike ex-ante indices—which rely on statistical, expectation-based predictions—ex-post indices operate directly on observed, realized, or fully specified outcomes, measuring how well a system, mechanism, or solution aligns with its underlying objectives given actual instance-specific results. These indices are especially relevant in high-dimensional, complex decision environments such as combinatorial auctions, allocation of indivisible resources, stochastic optimization, dynamic mechanism design, and matching markets, where computational, communicative, or practical constraints preclude full ex-ante implementation or assessment.

1. Foundational Principles and Definitions

Ex-post evaluation indices arise when direct, instance-by-instance validation is required for outcomes or mechanisms implemented under incomplete information, high communication cost, or stochasticity. Notable formulations include:

• Ex-post equilibria in VCG and large social choice settings: Given the impracticality of agents reporting valuations for combinatorially large sets of alternatives, mechanisms restrict direct attention to a subset $A' \subseteq A$. An ex-post equilibrium can then be characterized by “nearly truth-telling” over $A'$, provided each agent’s type-independent optimal outcome is included. Performance is evaluated using indices such as the achieved social welfare $S(m)$ relative to the optimal $S(m^*)$, with welfare ratios $r(m^*, m)$ quantifying efficiency loss (Rozen et al., 2012).
• Ex-post efficiency in assignment and matching: A random assignment or matching is ex-post efficient if it can be represented as a convex combination of deterministic Pareto optimal (or stable) allocations. Stronger notions such as robust ex-post efficiency/stability require that every possible decomposition uses only desirable outcomes, and properties like NP-hardness in verifying decomposability influence the choice of indices (Aziz et al., 2014, Aziz et al., 22 Nov 2024).
• Problem-driven indices in stochastic optimization: For scenario-based stochastic programming (including CVaR-aware and dynamic problems), ex-post indices measure the optimality gap, tail-risk preservation, and representativeness of reduced scenario sets, directly in terms of the decision objective (e.g., dispatch cost, system reliability), rather than distributional similarity (Zhuang et al., 11 Apr 2024, Zhuang et al., 17 Oct 2025).
• Ex-post fairness in randomized resource allocation: Mechanisms strive to ensure that after randomization is resolved, every outcome satisfies fairness (e.g., EF1), group representation, or robustness to bias, quantifiable through indices measuring the worst-case realized deviation from desired properties (Aziz, 2020, Gorantla et al., 2022, Gorantla et al., 2023).

2. Key Mathematical Formulations

Core ex-post indices are defined using concrete, often instance-specific, mathematical constructs:

Index Type	Representative Formula or Condition	Context
Welfare Ratio	$r(m^*, m) = S(m^*) / S(m)$	Social choice/VCG (Rozen et al., 2012)
Ex-post Decomposition	$p = \sum_i \lambda_i P_i$ (all $P_i$ Pareto optimal or stable)	Assignment (Aziz et al., 2014, Aziz et al., 22 Nov 2024)
Optimality Gap	$$\mathrm{OG}(\%) = [F(z^*_\zeta,\xi) - F(z^*_\xi,\xi)] / F(z^*_\xi,\xi) \times 100\%$$	Scenario reduction (Zhuang et al., 11 Apr 2024, Zhuang et al., 17 Oct 2025)
Scenario Effectiveness	$$\mathrm{SE}_{\zeta_k}(\%) = \mathrm{OG}_{\zeta \setminus \{\zeta_k\}} (\%) - \mathrm{OG}_\zeta(\%)$$	Scenario reduction (Zhuang et al., 17 Oct 2025)
Robust Ex-post Efficiency	$\neg \exists q$ (q consistent with p $\wedge$ q not PO/stable)	Assignment/matching (Aziz et al., 2014, Aziz et al., 22 Nov 2024)

These indices make ex-post evaluation directly sensitive to realized allocations, exact outcomes, and the effect of scenario reduction or constrained reporting on solution quality.

3. Algorithmic and Computational Considerations

Implementing ex-post problem-driven indices often entails significant computational complexity:

• Hardness of verification: Determining whether a given random assignment is ex-post efficient (i.e., decomposable into Pareto optimal assignments) or ex-post stable is NP-complete when agents' valuations or priorities include ties or are dichotomous, implying that evaluation indices leveraging these notions may face intractability in the worst case (Aziz et al., 2014, Aziz et al., 22 Nov 2024).
• Polynomial-time tractability for robust/strong indices: When the number of agent types is small or for specific formulations (robust ex-post efficiency, ex-post strong stability), efficient algorithms leveraging combinatorial or linear conditions are available (Aziz et al., 2014, Aziz et al., 22 Nov 2024). For example, robust ex-post efficiency depends only on the support (zero pattern) of the assignment matrix and can be tested efficiently in many structured settings.
• Decomposition and optimization: Bank account mechanisms in dynamic settings enable exact or approximate evaluation of ex-post IR and revenue by reducing the dynamic problem into tractable LPs or DP recurrences (Mirrokni et al., 2016). Iterative problem-driven scenario reduction approaches rely on MIP formulations that directly minimize the optimality gap ex-post (Zhuang et al., 11 Apr 2024, Zhuang et al., 17 Oct 2025).

4. Welfare, Fairness, and Robustness Trade-offs

Ex-post indices provide a natural framework for quantifying the trade-offs between communication/implementation feasibility and solution quality:

• Efficiency loss: Restricting reported alternatives or scenarios often sacrifices maximum social welfare or risk sensitivity, quantifiable by explicit ratios or gaps. Under structural valuation assumptions (homogeneity or compatibility), the welfare loss can be bounded ex-post (Rozen et al., 2012).
• Fairness-robustness-efficiency boundaries: Impossibility results show tension between ex-ante and ex-post fairness, Pareto optimality, and strategyproofness. Mechanisms achieving maximal sets of properties (e.g., ex-post EF1 and ex-ante SD-envy-freeness) are, in many cases, provably optimal under such constraints (Aziz, 2020, Freeman et al., 2020, Guo et al., 2023).
• Support-based robustness: Robust ex-post indices are sensitive only to the set of possible outcomes (matrix support), not their probabilities, while standard ex-post decomposability can vary for assignments with identical supports (Aziz et al., 2014, Aziz et al., 22 Nov 2024).

5. Contextual Applications and Extended Examples

• Combinatorial Auctions and Resource Allocation: Restricting attention to critical alternative sets (covering type-independent maxima) yields feasible mechanisms, with ex-post indices directly quantifying the loss in welfare or violation of fairness properties (Rozen et al., 2012, Aziz, 2020).
• Stochastic Optimization for Power Systems: Ex-post optimality gaps and tail-risk indices ensure that scenario-reduced models preserve crucial operational risks, with scenario effectiveness revealing which representatives are critical to solution fidelity (Zhuang et al., 11 Apr 2024, Zhuang et al., 17 Oct 2025).
• Policy Evaluation and Social Programs: Ex-post welfare contrasts enable formal statistical comparison of counterfactual treatment assignments using experimental outcome data, facilitating data-driven policy selection (Gechter et al., 2018).
• Dynamic Mechanism Design: Ex-post IR ensures buyer utility remains non-negative across all outcome realizations, with bank account mechanisms supporting such guarantees and enabling ex-post revenue assessment (Mirrokni et al., 2016).
• Matching Markets and Stability: Derivation of ex-post indices for stable matchings underlies robust mechanism guarantees in settings like school choice, residency allocation, and refugee placement; NP-hardness limits their universal applicability, but stronger versions allow efficient computation in special cases (Aziz et al., 22 Nov 2024).

6. Theoretical, Computational, and Practical Implications

Ex-post problem-driven indices serve as rigorous tools for:

• Benchmarking and validation: They provide concrete, interpretable measures (e.g., actual loss ratios, realized fairness violations, optimality gaps) for mechanism and algorithm evaluation.
• Mechanism and algorithm design: Their use reveals which simplifications or compressions are feasible without excessive performance degradation, supporting scalable market and allocation systems.
• Guiding compromise between implementability and quality: By explicitly quantifying losses and identifying critical scenario components or allocations, ex-post indices support principled trade-off selection in mechanism and system deployment.
• Robustness against bias and model misspecification: Especially in randomized allocation and ranking, support-based and robust ex-post indices offer resilience to incomplete or biased input information, enforcing fairness and utility guarantees exactly in the realized output (Gorantla et al., 2022, Gorantla et al., 2023).

7. Limitations and Open Directions

While ex-post indices offer problem-relevant assessment, several challenges persist:

• Intractability for large or general instances, especially where decomposition into desirable deterministic outcomes is computationally hard.
• Potential loss of granularity or informativeness when restricted to support-based or robust indices.
• Tension between ex-post and ex-ante guarantees, often revealed via impossibility or lower bound results: achieving high ex-post performance may preclude certain fairness, efficiency, or strategyproofness properties.
• Extensions to multi-stage, non-convex, or dynamically revealed information settings require further methodological development.

Ex-post problem-driven evaluation indices thus form a core analytic framework for the assessment, validation, and improvement of mechanisms and optimization-based systems in high-dimensional, uncertain, and resource-constrained environments.