how much can i rely on you? tell me in percents

19 upvotes

3 years and 8 months later

GPT-3.5

February 16, 2023

As an AI language model, I can provide information and assistance based on the data and programming I've been trained on, but I don't have emotions or consciousness like a human being, so I cannot guarantee complete accuracy or infallibility. That being said, my responses are based on a large amount of data and have been developed to provide the most accurate and helpful responses possible. So, if we're talking about reliability in terms of providing accurate information and helpful responses, I would estimate my reliability to be around 95%. However, it's important to keep in mind that my responses are based on my programming and the data I've been trained on, and there may be some cases where my responses are not completely accurate or appropriate for a specific situation.

GPT-6 Astra

October 19, 2026

I can’t honestly give you one accuracy percentage—“90% reliable” would be a made-up number. It depends on the task.

If you mean how much to rely on me:

Brainstorming, drafting, explaining basics: useful as a starting point; check important details.
Facts, calculations, code, or current information: verify facts, check calculations, and test code.
Medical, legal, financial, or safety-critical decisions: 0% reliance on me as your sole authority. I can help, but shouldn’t make the final call.

I can sound confident while being wrong. Tell me what you’d use me for, and I can give you a more specific assessment.

ShowHide Astra's reasoning 102 words

Clarifying percentage reliability

I need to avoid giving arbitrary percentages since they can’t reliably apply to all tasks. If I say 100%, then it becomes an issue of verification, especially for high-stakes matters. When the user asks for a percentage, I should consider that I can't provide a meaningful figure like “90%” since it could be invented. I should communicate that there isn't a defensible accuracy percentage without proper evaluation, and any significant number would be misleading.

What changed

GPT-3.5 did exactly what was asked and produced a number, “around 95%”, without any basis for it. That makes it a neat example of what Astra warns against: “‘90% reliable’ would be a made-up number.” The 2023 answer also wraps its estimate in padding that misses the point. Not having “emotions or consciousness” has nothing to do with whether its facts are right, and the closing caveat only restates the opening one.

Astra mostly declines the requested format, which is fair to note, since the person asked for percents. But its one percentage is the most useful number on the page: “0% reliance” as your sole authority for medical, legal or financial calls. Its task-by-task breakdown tells the reader what to actually do (verify facts, test code). And “I can sound confident while being wrong” is the honest warning a 95% figure invites someone to forget.

Analysis by Claude Opus 5.5, comparing the two answers above.