Predictive Concordance for Parameter Optimisation and Mixture Synthesis

Abstract: We discuss probabilistic measures of concordance between two probability distributions based on the expected misclassification rate (EMR). The focus is on comparing a given reference distribution with other distributions in a parametrised class, and optimising concordance by identifying parameter values maximising EMR or a regularised variant. EMR is a practical and decision-theoretically meaningful measure, and its optimisation has direct interpretation as a Bayesian decision analysis with a bounded utility function. We explore theoretical properties of EMR, discuss relationships with other measures including Küllback-Leibler divergence, and recognise that its optimisation has a synthetic Bayesian emulation interpretation that aids understanding and specification of regularisation penalties. A main area of methodology is in mixture synthesis where the parametrised family is a discrete mixture of given distributions. A detailed example comes from scenario forecasting in macroeconomic policy settings, a key applied area motivating the new methodology. Theoretical developments underlie efficient numerical optimisation and analysis is easily implemented using direct Monte Carlo simulation.

• The paper introduces Expected Misclassification Rate (EMR) to quantitatively assess concordance between candidate and reference distributions, emphasizing symmetry and boundedness.
• It employs a Bayesian decision-theoretic framework with synthetic likelihood and Dirichlet regularization to optimize mixture weights for scenario forecasts.
• Empirical results in macroeconomic forecasting demonstrate that EMR-based mixtures yield robust, high-concordance approximations of the reference distribution.

Predictive Concordance via Expected Misclassification Rate for Parameter Optimisation and Mixture Synthesis

Conceptual Framework and Motivation

The paper proposes the Expected Misclassification Rate (EMR) as a principled measure for assessing concordance between a parametrised family of distributions and a reference distribution, especially in settings where discrete mixtures—such as those arising from scenario forecasts in macroeconomics—are of primary interest (2606.14382). The approach is grounded in statistical decision theory, leveraging EMR as an interpretable and bounded utility function for comparing predictive distributions in the absence of observed data. The EMR quantifies the confusion probability between a candidate distribution and a reference, interpreting closeness as low discriminability.

A notable feature is that EMR-based optimisation is cast as a Bayesian decision problem, where the parameter vector (e.g., mixture weights) is selected to maximise EMR, potentially under regularisation penalties reflecting prior beliefs. The methodology is particularly relevant for mixture synthesis, where the weights assigned to different scenario forecasts are optimised to approximate an empirically obtained reference distribution.

EMR: Statistical Properties and Comparisons

EMR is defined as the probability that a random draw from the reference distribution is misclassified as coming from the candidate distribution (and vice versa), assuming equal prior probabilities for each hypothesis. The EMR has the following key properties:

• Symmetry: EMR is symmetric in the arguments, yielding identical values for (candidate, reference) and (reference, candidate).
• Boundedness: EMR $\leq 0.5$, with equality only when the densities are identical.
• Interpretability: EMR is equivalent to $1-\text{accuracy}$ of the optimal Bayesian classifier, thus directly interpretable on an absolute probability scale.

Comparisons to alternative concordance measures reveal that EMR is less sensitive to the tails than Kullback-Leibler (KL) divergence and remains well-defined when KL may not exist—a common issue in practical settings with heavy-tailed distributions. The paper demonstrates that under regularity conditions (e.g., scale mixtures of normals), EMR admits an approximate lower bound via the inverse logistic of KL, and in regimes where the candidate and reference distributions are close, optimising EMR and minimising KL yields nearly equivalent results.

EMR Optimisation: Synthetic Likelihood and Regularisation

The optimisation problem is structured as a maximisation of EMR with respect to parameters, notably mixture weights in the simplex. The synthetic likelihood interpretation emerges by viewing EMR as the marginal probability of an unobserved Bernoulli outcome, providing a probabilistic rationale for EMR maximisation as the synthetic likelihood maximisation for parameter estimation. Regularisation is implemented through synthetic priors—typically Dirichlet for mixture weights—facilitating robust solutions and preventing pathological sparsity. The objective function thus combines the log EMR and a log prior penalty, yielding a unique mode via strictly convex optimisation.

The procedure is efficiently implemented using Monte Carlo integration, especially when the reference distribution is available as simulated samples—a common scenario in model emulation and forecasting contexts.

Mixture Synthesis and Macroeconomic Scenario Forecasting

The main applied domain is scenario forecasting in macroeconomic policy, where decision-makers evaluate multiple scenario-specific predictive distributions against an empirically calibrated reference (e.g., statistical “Outlook-at-Risk” model). The optimisation framework selects mixture weights to synthesise scenario forecasts that best approximate the reference, considering both raw EMR maximisation and regularised variants. The paper shows that the unconstrained EMR maximiser often yields sparse mixtures, but regularisation (via a minimally informative Dirichlet prior) produces more stable and robust allocations.

Empirical analysis using macroeconomic forecasts for US GDP growth demonstrates that EMR-based synthesis often favours non-baseline scenarios when their predictive densities align more closely with the reference. The methodology accommodates constraints—such as privileging baseline scenarios—either directly or through informative priors. Numerical results indicate that EMR-based mixtures yield high concordance with the reference and that practical discrimination between the synthesised mixture and the reference is virtually impossible based on single observations.

Theoretical and Practical Implications

The paper establishes EMR as a theoretically sound alternative to traditional divergence measures for distributional concordance, offering symmetry, boundedness, and robustness to tail behaviour. The synthetic likelihood framework provides a natural basis for regularisation and interpretability, enabling direct integration of contextual constraints and prior information. Convexity guarantees unique solutions for mixture optimisation, scalable to high-dimensional cases and amenable to efficient Monte Carlo methods.

Practically, the approach advances scenario synthesis in macroeconomic and financial forecasting, risk management, and business analytics, where discrete mixtures and scenario-based reasoning predominate. The methodology can guide scenario selection, inform policy deliberations, and improve robust forecast aggregation.

Future Directions

Potential extensions include:

• Deeper theoretical analysis of relationships between EMR and other divergence measures, exploiting EMR’s connection to probabilistic discrimination.
• Incorporation of soft and informed priors to encode contextual preferences and expert knowledge.
• Application to higher-dimensional outcomes, richer mixture classes, and dynamic forecast synthesis.
• Utilising EMR-based synthesis for guiding the creation of new scenarios and probing disagreements in committee settings.
• Broader deployment in financial risk assessment, stress testing, and model calibration for complex simulation-based reference distributions.

Conclusion

EMR provides a mathematically principled and practically interpretable means for parameter optimisation and mixture synthesis in predictive modelling, especially for scenarios where the reference distribution is available only through simulation. Its symmetry, boundedness, and insensitivity to tail behaviour make it preferable to standard divergence measures for many applied contexts. The synthetic likelihood perspective unifies EMR optimisation with Bayesian decision-theoretic regularisation, yielding stable, robust mixture syntheses. The methodology is directly applicable to forecasting, risk assessment, and model emulation with promising avenues for both theoretical extension and practical adoption.