GPT is becoming a Turing machine: Here are some ways to program it

Abstract: We demonstrate that, through appropriate prompting, GPT-3 family of models can be triggered to perform iterative behaviours necessary to execute (rather than just write or recall) programs that involve loops, including several popular algorithms found in computer science curricula or software developer interviews. We trigger execution and description of Iterations by Regimenting Self-Attention (IRSA) in one (or a combination) of three ways: 1) Using strong repetitive structure in an example of an execution path of a target program for one particular input, 2) Prompting with fragments of execution paths, and 3) Explicitly forbidding (skipping) self-attention to parts of the generated text. On a dynamic program execution, IRSA leads to larger accuracy gains than replacing the model with the much more powerful GPT-4. IRSA has promising applications in education, as the prompts and responses resemble student assignments in data structures and algorithms classes. Our findings hold implications for evaluating LLMs, which typically target the in-context learning: We show that prompts that may not even cover one full task example can trigger algorithmic behaviour, allowing solving problems previously thought of as hard for LLMs, such as logical puzzles. Consequently, prompt design plays an even more critical role in LLM performance than previously recognized.

• The paper introduces IRSA, a novel method that leverages self-attention to enable iterative reasoning and simulate code execution in GPT-3.
• The approach uses structured prompts with fragmented execution paths to achieve high accuracy on tasks like Bubble Sort and logical deduction compared to baseline models.
• The study outlines significant implications for AI evaluation metrics and potential adversarial risks while suggesting future directions for LLM design in computational tasks.

Overview of "GPT is becoming a Turing machine: Here are some ways to program it"

The paper, authored by Ana Jojic, Zhen Wang, and Nebojsa Jojic, explores the emerging capabilities of the GPT-3 family models, positing that through carefully designed prompts, these models can be triggered to execute iterative behaviors characteristic of program execution rather than mere text generation or recall. The authors introduce and elaborate on Iteration by Regimenting Self Attention (IRSA), a method by which GPT-3 can be prompted to perform iterative computations internally without external mechanisms.

Key Contributions

1. Iteration by Regimenting Self Attention (IRSA): The paper introduces IRSA, a technique that leverages the self-attention mechanisms of GPT-3 to perform iterative reasoning, akin to executing code. The methodology involves:
   • Creating prompts with strong repetitive structural examples that guide the model through the execution path of target programs.
   • Using fragmented execution paths to illustrate patterns without detailing any single execution path comprehensively.
   • Implementing "skip attention" to force the model to only attend to the currently relevant state, enhancing efficiency and accuracy.
2. Applications and Performance: The authors demonstrate the effectiveness of IRSA in several computational and logical tasks:
   • Bubble Sort and Logical Deduction: By structuring prompts with a rigid format, the models exhibited improved performance over baseline prompting strategies.
   • Balanced Parentheses and Longest Common Subsequence (LCS) problems were also addressed, showing significant accuracies with IRSA compared to standard few-shot learning approaches.
3. Experimentation and Results: The paper provides a thorough experimental analysis using various datasets and tasks:
   • For Bubble Sort, IRSA achieved accuracy up to 100%, significantly outperforming baseline models.
   • In reasoning tasks like Logical Deduction, IRSA demonstrated notable improvements, reaching accuracy levels comparable to state-of-the-art systems that utilize external reasoning modules.
4. Theoretical and Practical Implications: The research posits consequential theoretical and practical implications including:
   • Adversarial Use Risks: The capacity of GPT to execute algorithmic tasks may expose vulnerabilities that could be exploited in negative contexts.
   • Evaluation Metrics in AI: The ability of LLMs to execute program-like tasks challenges existing benchmarks and evaluation methods, suggesting a reevaluation of how AI models should be assessed, particularly concerning in-context learning capabilities.
5. Challenges in Control and Execution: The research highlights the challenges in achieving proper execution control within LLMs, emphasizing the role of prompt design in minimizing errors related to inherent model biases and the overlapping influence of learned patterns.

Speculations for Future AI Development

• Expanded Role in Software Development: The ability to simulate execution paths can potentially transform code generation and debugging processes within software engineering.
• Implications for Educational Tools: The findings suggest new avenues for using LLMs as educational tools, potentially facilitating better understanding and teaching of programming concepts.
• Improving LLM Design: Future developments in LLM architectures might incorporate native support for self-regulation of attention spans, enhancing their ability to execute complex logical sequences autonomously.

Conclusion

The paper presents compelling evidence that GPT-3 models, through IRSA and appropriate prompting, can simulate the execution of complex algorithms. This capability hints at a closer alignment of these models with Turing machine architectures, encouraging further exploration and refinement of techniques that leverage self-attention for procedural reasoning in natural language processing tasks. The findings push the boundaries of what is considered achievable with LLMs, indicating future directions for both model evaluation and practical applications in AI-driven computation.