Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Rethinking ChatGPT's Success: Usability and Cognitive Behaviors Enabled by Auto-regressive LLMs' Prompting (2405.10474v1)

Published 17 May 2024 in cs.CL

Abstract: Over the last decade, a wide range of training and deployment strategies for LLMs have emerged. Among these, the prompting paradigms of Auto-regressive LLMs (AR-LLMs) have catalyzed a significant surge in AI. This paper aims to emphasize the significance of utilizing free-form modalities (forms of input and output) and verbal free-form contexts as user-directed channels (methods for transforming modalities) for downstream deployment. Specifically, we analyze the structure of modalities within both two types of LLMs and six task-specific channels during deployment. From the perspective of users, our analysis introduces and applies the analytical metrics of task customizability, transparency, and complexity to gauge their usability, highlighting the superior nature of AR-LLMs' prompting paradigms. Moreover, we examine the stimulation of diverse cognitive behaviors in LLMs through the adoption of free-form text and verbal contexts, mirroring human linguistic expressions of such behaviors. We then detail four common cognitive behaviors to underscore how AR-LLMs' prompting successfully imitate human-like behaviors using this free-form modality and channel. Lastly, the potential for improving LLM deployment, both as autonomous agents and within multi-agent systems, is identified via cognitive behavior concepts and principles.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Human-robot interaction: An introduction. Cambridge University Press, 2020.
  2. A neural probabilistic language model. JMLR, 3:1137–1155, 2003.
  3. Language models are few-shot learners. In Advances in NIPS, volume 33, pages 1877–1901, 2020.
  4. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, 2019.
  5. Towards interpreting and mitigating shortcut learning behavior of NLU models. In NAACL-HLT, pages 915–929, 2021.
  6. Robust transfer learning with pretrained language models through adapters. In ACL-IJCNLP, pages 854–861, 2021.
  7. Reasoning with language model is planning with world model. In EMNLP, pages 8154–8173, 2023.
  8. Parameter-efficient transfer learning for nlp. In ICML, pages 2790–2799, 2019.
  9. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In ICML, pages 9118–9147, 2022.
  10. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In AAAI, volume 34, pages 8018–8025, 2020.
  11. SpanBERT: Improving pre-training by representing and predicting spans. TACL, 8:64–77, 2020.
  12. D. Kahneman. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.
  13. Large language models are zero-shot reasoners. In Advances in NIPS, 2022.
  14. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In ACL, pages 7871–7880, 2020.
  15. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, ACL-IJCNLP, 2021.
  16. Make text unlearnable: Exploiting effective patterns to protect personal data. In TrustNLP, pages 249–259, 2023.
  17. A survey on out-of-distribution evaluation of neural nlp models. In IJCAI-23, 7 2023.
  18. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692, 2019.
  19. Logicot: Logical chain-of-thought instruction tuning. In EMNLP, 2023.
  20. Self-refine: Iterative refinement with self-feedback. In Advances in NIPS, 2023.
  21. Rethinking the role of demonstrations: What makes in-context learning work? In EMNLP, pages 11048–11064, 2022.
  22. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021.
  23. Training language models to follow instructions with human feedback. In Advances in NIPS, 2022.
  24. Improving language understanding by generative pre-training. 2018.
  25. Language models are unsupervised multitask learners. 2019.
  26. Model-tuning via prompts makes NLP models adversarially robust. In EMNLP, pages 9266–9286, 2023.
  27. Artificial intelligence a modern approach. London, 2010.
  28. Exploiting cloze-questions for few-shot text classification and natural language inference. In EACL, pages 255–269, 2021.
  29. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In EMNLP, pages 4222–4235, 2020.
  30. Reflexion: language agents with verbal reinforcement learning. In Advances in NIPS, 2023.
  31. Prompting GPT-3 to be reliable. In ICLR, 2023.
  32. Axiomatic attribution for deep networks. In ICML, pages 3319–3328, 2017.
  33. Universal adversarial triggers for attacking and analyzing nlp. In EMNLP-IJCNLP, pages 2153–2162, 2019.
  34. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR. OpenReview.net, 2019.
  35. Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks. In EMNLP, pages 5085–5109, 2022.
  36. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In ACL, pages 2609–2634, 2023.
  37. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2023.
  38. Describe, explain, plan and select: Interactive planning with LLMs enables open-world multi-task agents. In Advances in NIPS, 2023.
  39. Emergent abilities of large language models. TMLR, 2022.
  40. Chain of thought prompting elicits reasoning in large language models. In Advances in NIPS, 2022.
  41. On robust prefix-tuning for text classification. In ICLR, 2022.
  42. Tree of thoughts: Deliberate problem solving with large language models. In Advances in NIPS, 2023.
  43. React: Synergizing reasoning and acting in language models. In ICLR, 2023.
  44. Robustness of demonstration-based learning under limited data scenario. In EMNLP, pages 1769–1782, 2022.
  45. Automatic chain of thought prompting in large language models. In ICLR, 2023.
  46. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiv:2304.06364, 2023.
  47. Barry J Zimmerman. Attaining self-regulation: A social cognitive perspective. In Handbook of self-regulation, pages 13–39. 2000.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Xinzhe Li (14 papers)
  2. Ming Liu (421 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets