Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
60 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GPT-4 Technical Report (2303.08774v6)

Published 15 Mar 2023 in cs.CL and cs.AI

Abstract: We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020.
  2. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022.
  3. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.
  4. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021.
  5. Transformer-XL: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019.
  6. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
  7. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  8. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019.
  9. Adafactor: Adaptive learning rates with sublinear memory cost. arXiv preprint arXiv:1804.04235, 2018.
  10. Layer normalization. arXiv preprint arXiv:1607.06450, 2016.
  11. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 2022a.
  12. Large language models can self-improve. arXiv preprint arXiv:2210.11610, 2022.
  13. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022.
  14. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  15. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701, 2020.
  16. Tensor Programs V: Tuning large neural networks via zero-shot hyperparameter transfer. arXiv preprint arXiv:2203.03466, 2022.
  17. Outrageously large neural networks: The sparsely-gated Mixture-of-Experts layer. arXiv preprint arXiv:1701.06538, 2017.
  18. ST-MoE: Designing stable and transferable sparse expert models. arXiv preprint arXiv:2202.08906, 2022.
  19. Emergent abilities of large language models. TMLR, 2022b.
  20. Universal transformers. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=HyzdRiR9Y7.
  21. RoFormer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864, 2021.
  22. Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems.
  23. PaLI: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022a.
  24. GPT-J-6B: A 6 billion parameter autoregressive language model, 2021.
  25. GPT-Neo: Large scale autoregressive language modeling with mesh-tensorflow. If you use this software, please cite it using these metadata, 58, 2021.
  26. Bloom: A 176B-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022.
  27. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.
  28. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  29. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444, 2017.
  30. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291, 2019.
  31. Flashattention: Fast and memory-efficient exact attention with io-awareness. arXiv preprint arXiv:2205.14135, 2022.
  32. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509, 2019.
  33. Self-attention does not need o⁢(n2)𝑜superscript𝑛2o(n^{2})italic_o ( italic_n start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT ) memory. arXiv preprint arXiv:2112.05682, 2021.
  34. Gpu kernels for block-sparse weights, 2017. URL https://cdn.openai.com/blocksparse/blocksparsepaper.pdf.
  35. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021a.
  36. Aligning AI with shared human values. Proceedings of the International Conference on Learning Representations (ICLR), 2021b.
  37. Language models are unsupervised multitask learners. 2019.
  38. Improving language understanding by generative pre-training. 2018.
  39. Attention is all you need. NeurIPS, 2017.
  40. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30, 2017.
  41. Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409, 2017.
  42. The computational limits of deep learning. arXiv preprint arXiv:2007.05558, 2020.
  43. Evaluating large language models trained on code. 2021.
  44. The Inverse Scaling Prize, 2022a. URL https://github.com/inverse-scaling/prize.
  45. Inverse scaling can become U-shaped. arXiv preprint arXiv:2211.02011, 2022c.
  46. Inverse Scaling Prize: First round winners, 2022b. URL https://irmckenzie.co.uk/round1.
  47. OpenAI: OpenAI API, 2020. URL https://openai.com/blog/openai-api.
  48. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615, 2022.
  49. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
  50. Transcending scaling laws with 0.1% extra compute. arXiv preprint arXiv:2210.11399, 2022.
  51. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.
  52. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1472. URL https://aclanthology.org/P19-1472.
  53. Adversarial training for large neural language models. arXiv preprint arXiv:2004.08994, 2020.
  54. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. ArXiv, abs/1803.05457, 2018.
  55. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022.
  56. WinoGrande: An adversarial Winograd schema challenge at scale. arXiv preprint arXiv:1907.10641, 2019.
  57. CodeT: Code generation with generated tests. arXiv preprint arXiv:2207.10397, 2022b.
  58. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2368–2378, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1246. URL https://aclanthology.org/N19-1246.
  59. Question directed graph attention network for numerical reasoning over text. arXiv preprint arXiv:2009.07448, 2020.
  60. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
  61. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858, 2022.
  62. Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022.
  63. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022.
  64. OpenAI. OpenAI: Introducing ChatGPT, 2022. URL https://openai.com/blog/chatgpt.
  65. OpenAI. OpenAI: GPT-4, 2023a. URL https://openai.com/research/gpt-4.
  66. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.229. URL https://aclanthology.org/2022.acl-long.229.
  67. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.
  68. OpenAI. OpenAI: How should AI systems behave, and who should decide?, 2023b. URL https://openai.com/blog/how-should-ai-systems-behave.
  69. OpenAI: Our approach to alignment research, 2022. URL https://openai.com/blog/our-approach-to-alignment-research.
  70. Joseph Carlsmith. Is power-seeking AI an existential risk? ArXiv, abs/2206.13353, 2022.
  71. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375, 2022.
  72. Red teaming language models with language models. arXiv preprint arXiv:2202.03286, 2022.
  73. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462, 2020.
  74. Dora Seigel. How do you calculate SAT score? raw and scaled, 1 2020. URL https://blog.prepscholar.com/how-to-calculate-sat-score.
  75. The Albert blog. URL https://www.albert.io/blog/.
  76. Mathematical Association of America. AMC statistics, 2023. URL http://amc-reg.maa.org/Reports/GeneralReports.aspx.
  77. Halle Edwards. SAT percentiles and score rankings, 2022. URL https://blog.prepscholar.com/sat-percentiles-and-score-rankings.
  78. College Board. Understanding SAT scores, 2022a. URL https://satsuite.collegeboard.org/media/pdf/understanding-sat-scores.pdf.
  79. College Board. AP score distributions by subject, 2022b. URL https://apcentral.collegeboard.org/media/pdf/ap-score-distributions-by-subject-2022.pdf.
  80. Center for Excellence in Education. 2020 USABO Semifinal exam score distribution, 2022. URL https://www.usabo-trc.org/sites/default/files/allfiles/2020%20USABO%20Semifinal%20Exam%20Histogram.pdf.
  81. Chris Swimmer. GRE score percentiles – what does your score mean for you? (2021 update), 4 2021. URL https://magoosh.com/gre/gre-score-percentiles/.
  82. John B. Nici. AP Art History: 5 Practice Tests + Comprehensive Review + Online Practice. Barron’s Test Prep. Barron’s Educational Series, 2020. ISBN 9781506260501.
  83. ETS. GRE sample issue task, 2022. URL https://www.ets.org/pdfs/gre/sample-issue-task.pdf.
  84. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 220–229, January 2019. doi: 10.1145/3287560.3287596.
  85. System Cards, a new resource for understanding how AI systems work. https://ai.facebook.com/blog/system-cards-a-new-resource-for-understanding-how-ai-systems-work/, February 2022.
Citations (9,415)
View on Semantic Scholar

Summary

  • The paper introduces GPT-4's multimodal architecture that processes both text and images, achieving benchmark improvements like 86.4% accuracy on MMLU.
  • The paper details rigorous safety measures including RLHF and adversarial testing that reduce toxic responses by 82% compared to GPT-3.5.
  • The paper highlights limitations such as hallucination risks and a fixed knowledge cutoff, guiding future research to enhance model reliability.

GPT-4 Technical Report Overview

The "GPT-4 Technical Report" by OpenAI delineates the intricate architecture, capabilities, limitations, and safety measures associated with GPT-4, a large-scale multimodal model designed to process both image and text inputs, and produce text outputs.

Model Architecture and Training

GPT-4 is distinguished by its multimodal capabilities, allowing it to handle image and text inputs simultaneously, a significant advancement over previous iterations like GPT-3.5. The foundational structure of GPT-4 remains rooted in the Transformer architecture, emphasizing scale predictability and infrastructural robustness across various scales. This predictability was achieved through meticulous tuning and optimization methods that allowed performance predictions based on models trained with significantly less computational resources.

Performance Benchmarks

GPT-4's performance was assessed using a diverse set of benchmarks including professional exams, traditional NLP tasks, and multilingual tests. Notably:

  • Professional and Academic Exams: GPT-4 demonstrated impressive capabilities, outscoring GPT-3.5 across multiple exams. For instance, it ranked in the top 10% of a simulated Uniform Bar Examination and achieved the 99th percentile on the Graduate Record Examination (GRE) Verbal section.
  • NLP Benchmarks: On the MMLU benchmark, GPT-4 outperformed the previous state-of-the-art, achieving an 86.4% accuracy compared to GPT-3.5's 70.0%. Additionally, in tasks like HellaSwag, and ARC, it consistently surpassed the highest scores of existing LLMs.
  • HumanEval: GPT-4 achieved a 67.0% pass rate on the HumanEval dataset, which measures the ability to synthesize Python functions, significantly better than GPT-3.5.

Multilingual Capabilities

GPT-4 exhibited noteworthy performance across languages, outperforming previous models, including Chinchilla and PaLM, in various non-English languages. This includes low-resource languages like Latvian and Welsh, suggesting that GPT-4 has broadened the horizon for multilingual NLP efficacy.

Safety and Alignment

GPT-4 incorporates Reinforcement Learning from Human Feedback (RLHF), enhancing its adherence to user intent and enhancing safety measures. This involved adversarial testing with over 50 domain experts and a sophisticated model-assisted safety pipeline. These measures have substantially improved its ability to refuse inappropriate requests while reducing the occurrence of toxic outputs.

  • Mitigation Strategies: GPT-4 was subjected to adversarial testing by experts in fields such as cybersecurity and bio-risk, identifying and mitigating potential safety risks. Measures include rule-based reward models (RBRMs) to guide responses appropriately and reduce undesired behaviors.
  • Safety Metrics: The model's tendency to respond to disallowed content was reduced by 82% compared to GPT-3.5, and toxic responses were significantly lower on the RealToxicityPrompts dataset.

Vision Capabilities

GPT-4’s ability to process visual inputs was highlighted with various examples, including interpreting diagrams, providing explanations for memes, and solving exam questions with visual components. Preliminary results indicate that the model retains its robust language processing skills while adeptly handling visual inputs.

Limitations

Despite substantial advancements, GPT-4 retains some limitations:

  • Hallucination Issues: It may still generate incorrect information or reasoning errors, necessitating careful validation in high-stakes contexts.
  • Knowledge Cutoff: The model's training data mostly ends in September 2021, which limits its knowledge of subsequent events.
  • Context Window: Similar to its predecessors, GPT-4 has a limited context window, affecting its ability to handle very long textual inputs.

Future Directions

Future research and development efforts will likely focus on enhancing the model’s safety, reliability, and expanding its understanding to more recent data. Moreover, the report emphasizes the importance of continued collaboration with external researchers to assess and mitigate potential risks as model capabilities expand.

Conclusion

GPT-4 represents a significant step in AI development, showcasing enhanced capabilities in both language and multimodal tasks. Its performance across a broad spectrum of benchmarks underscores its potential for diverse applications, while ongoing efforts to improve its safety and reliability mark a crucial direction for future advancements. The report provides a comprehensive view of GPT-4's architecture, performance, and safety considerations, offering valuable insights for the continued evolution of AI technologies.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube