Establish whether the severity-threshold prompt genuinely improves classification

Establish whether explicitly requiring diarrhea to be active and clinically significant, rather than merely documented, produces a genuine accuracy improvement over the deployed dual-LLM prompt for identifying trofinetide-associated diarrhea.

Background

The sensitivity analysis found that prompt version V2, which explicitly excludes mild or controlled diarrhea and requires active clinical significance, had higher observed accuracy than the deployed V0 prompt. The authors caution that this apparent advantage is based on a small number of discordant cells and does not remain statistically established after multiplicity correction. Consequently, whether the severity-threshold wording truly improves classification remains unresolved.

References

The difference is nonetheless not established, as it rests on 9 cells won against 1 lost, which is $p=0.021$ before correction and $p=0.15$ after Holm correction across the seven comparisons in this family.

— Documenting Trofinetide Treatment Phenotypes in Rett Syndrome with a Dual-LLM Consensus Pipeline  (2610.22062 - Palacios et al., 15 Oct 2026) in Supplementary Note S7, subsection “Prompt wording”