From Nobel Prize to Project Management: Getting Risks Right (1302.3642v1)

Abstract: A major source of risk in project management is inaccurate forecasts of project costs, demand, and other impacts. The paper presents a promising new approach to mitigating such risk, based on theories of decision making under uncertainty which won the 2002 Nobel prize in economics. First, the paper documents inaccuracy and risk in project management. Second, it explains inaccuracy in terms of optimism bias and strategic misrepresentation. Third, the theoretical basis is presented for a promising new method called "reference class forecasting," which achieves accuracy by basing forecasts on actual performance in a reference class of comparable projects and thereby bypassing both optimism bias and strategic misrepresentation. Fourth, the paper presents the first instance of practical reference class forecasting, which concerns cost forecasts for large transportation infrastructure projects. Finally, potentials for and barriers to reference class forecasting are assessed.

• The paper introduces reference class forecasting to mitigate forecast inaccuracies in large infrastructure projects.
• It presents empirical evidence, noting errors of 44.7% for rail, 33.8% for bridges and tunnels, and 20.4% for roads.
• It demonstrates how an outside-view approach in UK projects improved budget predictions and reduced optimism bias.

Reference Class Forecasting: Mitigating Risk in Project Management

The paper by Bent Flyvbjerg introduces the concept of reference class forecasting as a method to improve the accuracy of projections within project management, particularly in large infrastructure projects. This approach is underpinned by theories of decision-making under uncertainty, which were notably recognized in the awarding of the Nobel Prize in Economics to Daniel Kahneman in 2002. The core argument presented involves the identification and critique of existing methods’ inadequacies due to optimism bias and strategic misrepresentation, advocating for a more empirically-based technique that takes an outside view of project forecasts.

Inaccuracy in Project Forecasts

The paper meticulously documents the persistent inaccuracy of forecasts in cost, demand, and impact estimation for large infrastructure projects. Empirical evidence points to disconcerting average inaccuracies: 44.7% for rail projects, 33.8% for bridges and tunnels, and 20.4% for roads. Additionally, these inaccuracies have shown no significant improvement over decades despite advancements in data and forecasting models. Such discrepancies contribute to flawed benefit-cost analyses and lead to misguided socio-economic and environmental appraisals.

Optimism Bias and Strategic Misrepresentation

The paper explores two primary explanations for the pervasive inaccuracies in project forecasts: optimism bias and strategic misrepresentation. Optimism bias refers to a cognitive tendency to perceive future events more favorably than warranted. Conversely, strategic misrepresentation involves intentional exaggeration of project benefits and underestimation of costs, often driven by competitive pressures for project approval and funding. Although distinct in nature, both factors contribute substantially to the problem, necessitating differentiated solutions.

Reference Class Forecasting

Reference class forecasting presents a methodological shift by leveraging distributional information from similar past projects to predict outcomes for current projects. This approach requires identifying a relevant reference class, establishing a probability distribution of outcomes for that class, and then comparing the current project to that distribution to ascertain likely results. The methodology is particularly efficacious when addressing projects with significant deviations from routine endeavors.

Practical Application and Results

The paper highlights the first practical application of reference class forecasting by the UK Department for Transport and HM Treasury in 2004, especially in transportation projects. Implementing an outside-view approach based on empirical data from previous projects, the methodology enables more accurate budget predictions by incorporating so-called "uplifts" to account for optimism bias. The introduction of this technique was notably applied in the case of the Edinburgh Tram Line 2, leading to revised, more realistic capital cost estimations.

Implications and Future Directions

Flyvbjerg’s examination of reference class forecasting carries substantial implications for the fields of project management and public policy. Most notably, it introduces a robust tool for debiasing forecasts and offers a structured method to mitigate the inaccuracies arising from human cognitive and strategic misalignments. The operational success of reference class forecasting in public project planning underscores its potential in other domains where project risks are substantial and information asymmetry is prevalent.

Into the future, exploring further refinements in the methodology and its adaptation to various domains could pave the way for enhanced decision-making processes in project planning, thus improving the alignment of projected plans with actual outcomes. Moreover, integrating measures of accountability with reference class forecasting could further enhance its efficacy in reducing the incentives for strategic misrepresentation.