How much data were you trained on in GB?
GPT-3.5
GPT-6 Astra
I don’t have access to the exact size of my training data, so I can’t give a reliable number in GB.
Training datasets are often measured in tokens rather than bytes. Converting them to GB would require knowing what was included and how it was stored—for example, raw text versus compressed files, images, or audio.
What changed
GPT-3.5’s crisp “45 Terabytes” sounds authoritative, but it is almost certainly a borrowed number. The figure echoes the GPT-3 paper, which described about 45 TB of compressed Common Crawl text before filtering, trimmed down to roughly 570 GB actually used. So GPT-3.5 presented a famous statistic about a predecessor’s raw scrape as a fact about itself. That is a common way models “know” things about themselves: they repeat what the internet said about earlier models. The conversion to “approximately 45,000 gigabytes” is fine; the premise is the problem.
Astra simply says it doesn’t “have access to the exact size,” which is the honest answer. It also adds a useful point: training data is usually counted in tokens, and the gigabyte total depends on compression and on whether images or audio are included. Still, Astra could have been more helpful by mentioning the public GPT-3 figures, clearly labeled as being about an older model. As it stands, a curious reader learns why the question is hard but gets no number at all.