Resolve demographic confounding in the randomized identity experiment

Construct a preregistered replication of the randomized telephone-fraud identity experiment with additional treatment arms that cross apparent age with sex and accent, thereby determining whether the observed age gradient persists independently of these identity attributes.

Background

The experiment used ten identities, but all four female identities were under thirty, leaving apparent age entangled with sex; the identities also differed in accent and other biographical characteristics. Although several robustness analyses supported the age gradient, the limited number and structure of treatment arms prevented a clean separation of these attributes.

The paper identifies a preregistered replication with more treatment arms, specifically crossing age against sex rather than confounding the two, as the natural next study. Such a design would test whether the observed targeting pattern is attributable to apparent age itself or to correlated identity features.

References

Third, the analysis was specified after collection ended rather than preregistered; we mitigate this by reporting every outcome examined (Table 2), by leading with an assumption-free randomization test, and by stating which results survive multiplicity correction, but a preregistered replication with more arms---and with age crossed against sex rather than confounded with it---is the natural next study.

— Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate  (2608.24127 - Traister et al., 25 Aug 2026) in Section 5, subsection “Limitations”