Exploring the hierarchical structure of human plans via program generation (2311.18644v2)

Abstract: Human behavior is often assumed to be hierarchically structured, made up of abstract actions that can be decomposed into concrete actions. However, behavior is typically measured as a sequence of actions, which makes it difficult to infer its hierarchical structure. In this paper, we explore how people form hierarchically structured plans, using an experimental paradigm with observable hierarchical representations: participants create programs that produce sequences of actions in a language with explicit hierarchical structure. This task lets us test two well-established principles of human behavior: utility maximization (i.e. using fewer actions) and minimum description length (MDL; i.e. having a shorter program). We find that humans are sensitive to both metrics, but that both accounts fail to predict a qualitative feature of human-created programs, namely that people prefer programs with reuse over and above the predictions of MDL. We formalize this preference for reuse by extending the MDL account into a generative model over programs, modeling hierarchy choice as the induction of a grammar over actions. Our account can explain the preference for reuse and provides better predictions of human behavior, going beyond simple accounts of compressibility to highlight a principle that guides hierarchical planning.

• The paper demonstrates that human planning involves hierarchical decomposition, with participants favoring the reuse of subroutines over solely minimizing actions.
• The proposed grammar induction model, extending MDL principles, better predicts program creation by biasing towards frequently reused subroutines.
• Experimental results using the Lightbot task validate that structural reuse is a key factor in efficient human planning beyond conventional utility maximization.

Overview of the Hierarchical Structure of Human Planning in Program Generation

The paper examines the hierarchical organization of human plans by utilizing a controlled experimental setting known as Lightbot, a programming task that exposes the latent structure of human planning through the explicit generation of hierarchically-structured programs. Human behavior, deeply rooted in hierarchical decomposition, is typically observed through action sequences that obscure their inherent hierarchical structures. This research investigates two established principles of human planning: utility maximization, where solutions involve fewer actions, and minimum description length (MDL), where plans are represented using shorter programs. Findings reveal that, while these principles do guide human planning to some extent, they do not fully account for the observed preference for program reuse among participants, which is a key qualitative feature in the experimental data.

The authors model this preference for reuse by extending the MDL approach to a grammar induction model that generates programs. This model predicts human program creation better than existing models focused solely on compressibility. This suggests a more nuanced understanding of human hierarchical action planning that emphasizes the reuse of previously successful structures.

Key Findings and Models

1. Hierarchical Planning and Human Behavior
   • Humans create hierarchical plans by decomposing abstract actions into concrete steps, a behavior insightfully revealed through the Lightbot programming task.
   • Participants tend to favor plans that are not only efficient in terms of actions and length but also prefer reusing subroutines in a way that existing models like utility maximization and MDL fail to predict.
2. Generative Model for Hierarchical Planning
   • The authors propose a grammar induction approach, inspired by linguistic grammar models, that accounts for the reuse behavior in human hierarchical planning.
   • Using the concept of adaptor grammars and the Chinese Restaurant Process (CRP), the model biases the formation of subroutines towards those frequently used, embodying a 'rich-get-richer' dynamic that resonates with observed behavior.
3. Experimental Validation
   • Through the Lightbot task, participants' programming behavior was scrutinized, revealing a preference for simpler, reusable subroutines, despite incentives that might encourage minimal program length.
   • The grammar induction model explained participant behavior more accurately than simpler models, suggesting a fundamental aspect of human planning strategy underestimated by previous models.
4. Theoretical and Practical Implications
   • The preference for reuse posited by the grammar induction model could stem from an efficiency in cognitive processing and reflect analogous paradigms in reinforcement learning and Bayesian inference.
   • This insight enriches our theoretical understanding of human cognitive architecture related to planning and action sequence learning.
   • Practically, insights from this paper could inform the design of more intuitive programming environments and AI systems that leverage hierarchical planning.

Future Directions

The paper highlights the benefit of using process-tracing experiments like Lightbot to clarify the hierarchical organization inherent in human planning. Future work might explore how varying complexities in task environments influence hierarchical depth, further refinements in grammar induction models, or broader applications in artificial intelligence to enhance machine efficiency in planning tasks.

Conclusion

This paper advances our understanding of the hierarchical structure of human plans, emphasizing the importance of structural reuse in human strategy. Through innovative empirical and modeling approaches, the authors provide substantive evidence for deploying grammar induction models over simpler accounts of plan efficiency. Such advancements in understanding human hierarchical planning hold significant value for developing AI systems that aim to emulate human-like planning strategies.