Dice Question Streamline Icon: https://streamlinehq.com

Exact Parameter Count of GPT-4

Determine the exact number of parameters in GPT-4 to enable precise comparisons of model scale against other large language models and to contextualize performance differences observed on the Japanese financial benchmark.

Information Square Streamline Icon: https://streamlinehq.com

Background

In the discussion of benchmark results, GPT-4 exhibited significantly higher performance than other models. The paper attributes part of this difference to model scale, noting that GPT-4 likely has far more parameters than open models, but the exact count is not publicly known.

Clarifying the parameter count of GPT-4 would help interpret the relationship between model size and performance across the evaluated tasks, especially given the comparison to Qwen-72B and other models with disclosed parameter counts.

References

Although the number of parameters in GPT-4 has not been determined, it is estimated to be more than 500 billion.

Construction of a Japanese Financial Benchmark for Large Language Models (2403.15062 - Hirano, 22 Mar 2024) in Discussion (Section 5)