Automated Meta Prompt Engineering for Alignment with the Theory of Mind (2505.09024v1)

Abstract: We introduce a method of meta-prompting that jointly produces fluent text for complex tasks while optimizing the similarity of neural states between a human's mental expectation and a LLM's (LLM) neural processing. A technique of agentic reinforcement learning is applied, in which an LLM as a Judge (LLMaaJ) teaches another LLM, through in-context learning, how to produce content by interpreting the intended and unintended generated text traits. To measure human mental beliefs around content production, users modify long form AI-generated text articles before publication at the US Open 2024 tennis Grand Slam. Now, an LLMaaJ can solve the Theory of Mind (ToM) alignment problem by anticipating and including human edits within the creation of text from an LLM. Throughout experimentation and by interpreting the results of a live production system, the expectations of human content reviewers had 100% of alignment with AI 53.8% of the time with an average iteration count of 4.38. The geometric interpretation of content traits such as factualness, novelty, repetitiveness, and relevancy over a Hilbert vector space combines spatial volume (all trait importance) with vertices alignment (individual trait relevance) enabled the LLMaaJ to optimize on Human ToM. This resulted in an increase in content quality by extending the coverage of tennis action. Our work that was deployed at the US Open 2024 has been used across other live events within sports and entertainment.

Automated Meta Prompt Engineering for Alignment with Theory of Mind

The paper presents a robust methodology that combines meta-prompting and Theory of Mind (ToM) alignment to optimize content generation workflows. The authors propose leveraging LLMs for automated refinements in content production, focusing on aligning AI-generated text with human mental models. This research is set within the context of live events, specifically the 2024 US Open tennis Grand Slam, where it aims to streamline the editorial process in handling sports content.

The central innovation of this paper revolves around using an LLM-as-a-Judge (LLMaaJ) to evaluate content on multiple dimensions, including novelty, factualness, repetitiveness, and relevancy. By comparing AI-generated content against human-intended and unintended modifications, the LLMaaJ effectively evolves through in-context learning reinforced by agentic reinforcement learning techniques. These processes align the outputs from another LLM with human expectations, demonstrating a 53.8% alignment rate in real-time implementations with an average of 4.38 iterations required for convergence.

Notable in this methodology is the reliance on Hilbert vector spaces to geometrically interpret content traits. This approach not only enhances the quality of AI-generated text by considering individual and collective relevance across defined parameters but also adds a layer of spatial complexity to ToM modeling. Uniquely, it combines prescriptive few-shot learning with zero-shot creative context learning to create large-scale workflows.

The practical implications of this paper are vast, especially in the sports and entertainment sectors where rapid content generation and minimal human oversight are critical. The deployment of this methodology at the US Open 2024 illustrates its potential for wide application in live sports events, enhancing the scope and quality of content coverage.

The theoretical contributions lie in advancing ToM applications within AI, particularly in improving the empathetic reasoning capacity of LLMs, thus bridging the gap between AI outputs and human expectations. Future research directions hinted at include enabling bespoke editing dimensions and integrating jury-based evaluations to further refine ToM alignment.

Overall, this work presents substantial evidence of combinative success between meta-prompting and ToM alignment. It paves the way for developing AI systems capable of more closely mimicking human cognitive and editorial processes, offering significant advancements in content generation technology.