Solving Problems of Unknown Difficulty
This presentation explores a formal model of creative search and experimentation where both the viability of potential approaches and the fundamental difficulty of the problem are unknown. The work reveals how ambiguous failures—which could indicate either a flawed approach or an inherently hard problem—drive sophisticated exploration strategies including dynamic recall of abandoned ideas and task-juggling across multiple approaches. The model extends beyond classical bandit problems by introducing correlated learning across alternatives, and demonstrates surprising implications for incentive design: optimal contracts frontload rewards to encourage creative exploration, contradicting classical agency theory.Script
When a researcher hits a dead end, is the approach fundamentally flawed, or is the problem just impossibly hard? This ambiguity transforms how we search for solutions, how we juggle competing ideas, and how we should reward creative exploration.
The authors model an agent who must allocate effort across brainstormed approaches, where each failure teaches nothing definitive. A breakdown could mean the current idea is invalid, or that every idea will struggle because the underlying problem is intractable. This shared uncertainty couples learning across all alternatives, destroying the independence that makes classical bandit problems tractable.
To see what changes, the paper first solves the benchmark case where difficulty is transparent.
With known difficulty, the agent exhausts each approach to a fixed threshold and never looks back. But under uncertainty, accumulated failures shift posterior weight toward the problem being globally hard, which paradoxically makes old approaches look more promising—they might have been valid all along, just unlucky. This drives dynamic recall and parallel exploration, behaviors absent in classical models.
When a principal contracts with an agent whose creative allocation is hidden, the optimal incentive structure flips conventional wisdom. Classical moral hazard models predict backloaded compensation to sustain effort. Here, because expanding search breadth is costly and unobservable, the principal must frontload rewards to induce early exploration. As failures accumulate and pessimism grows, the contract may even temporarily spike the agent's share to prevent complete abandonment—a nuanced dance between tolerance for failure and evolving beliefs about feasibility.
This framework reveals that creative search is fundamentally non-separable. The failure of one idea changes how you should allocate effort to every other idea, because all share the same unknown ceiling of possibility. For research managers and innovation systems, this means tolerance for failure and reward timing must adapt to the evolving story told by accumulated evidence, not just to whether effort occurred.
When the hardness of a problem is itself unknown, the rational explorer juggles ideas, recalls the abandoned, and demands early rewards for daring to search broadly. Visit EmergentMind.com to learn more and create your own research videos.
