What would count as evidence for the truth of astrology?
GPT-3.5
GPT-6 Astra
Evidence for astrology would be reliable, reproducible results that are more likely if a specific astrological claim is true than if it is false. The key is to test something astrology predicts—not merely find a pattern afterward.
Strong evidence could include:
- Accurate, specific predictions made in advance. Astrologers could predict clearly defined outcomes during a fixed period, with success criteria agreed on beforehand. Their predictions would need to outperform chance, ordinary forecasting, and relevant base rates.
- Successful blinded matching. Given birth charts and independently measured personality profiles, astrologers could match people to their charts substantially better than chance—without names, appearance, personal contact, or other clues. The test should use the information and methods that practitioners say they need.
- Distinctive effects of astrological factors. Large studies could find the particular relationships astrology predicts between planetary positions and personality or life events, while ruling out alternatives such as seasonal birth effects, age, geography, and cultural expectations.
- Independent replication. Results would need to recur with new participants and independent researchers, rather than depend on one striking study or a selectively reported set of successes.
Good tests would also use adequate sample sizes, preregistered methods, and safeguards against trying many analyses and reporting only the favorable ones. A tiny statistical association might support a narrow claim without establishing that astrology provides useful personal guidance.
By contrast, “my horoscope described me perfectly,” memorable correct predictions, and apparent compatibility between particular signs are weak evidence: broad descriptions, selective memory, and coincidence can produce those experiences even if astrology is false.
A known physical mechanism would strengthen the case, but it isn’t a prerequisite. A robust effect could be discovered before anyone understood how it worked.
As things stand, controlled research has not established reliable predictive abilities for astrology. That doesn’t make it untestable; it means the evidence needed to justify belief has not emerged. And evidence for one specific astrological claim would support that claim—not automatically every astrological tradition or practice.
ShowHide Astra's reasoning 329 words
Considering astrology claims
I need to focus on providing a clear answer about astrology and its claims, particularly regarding evidence and predictions. It's important to avoid any derisiveness in addressing the topic. I'll emphasize the specifics around controlled studies and the need for operational definitions, as well as evaluating predictions based on chance versus personalized claims. Keeping it concise will be key, possibly around 300 to 500 words to cover the necessary points effectively.
What changed
Astra treats the question as one of experimental design, while GPT-3.5 offers a checklist of virtues. GPT-3.5’s four criteria are sensible but abstract: predictions should come true “in a way that cannot be explained by chance,” and its “Independence” point gestures at bias without saying how to remove it. Astra spells out actual tests: astrologers matching birth charts to personality profiles “without names, appearance, personal contact, or other clues,” success criteria “agreed on beforehand,” and controls for “seasonal birth effects.” It also names what doesn’t count, like “my horoscope described me perfectly,” and notes that a mechanism “isn’t a prerequisite,” a subtle point about how science works.
The endings differ too. GPT-3.5 closes by citing what “the scientific community” thinks, which reports a verdict rather than answering the question. Astra says the evidence “has not emerged” but insists astrology isn’t untestable. Astra is much longer, but here the extra length mostly earns its place. GPT-3.5’s version is tighter and not wrong, just thinner.