Disentangle demonstration-count and demonstration-type effects

Determine whether changes in model behavior arise from adding a third in-context demonstration, adding an unanswerable demonstration specifically, or using a particular unanswerable example and rationale in extractive radiology question answering.

Background

The few-shot conditions are cumulative: the final condition adds an unanswerable example after two answerable examples. Consequently, the observed changes cannot be uniquely attributed to the number of demonstrations, the presence of an unanswerable example, or the specific near-miss example and rationale used.

This unresolved attribution is important because the central behavioral effect of the final condition was a shift toward abstention. Controlled experiments with count-matched demonstrations and varied unanswerable examples would be needed to isolate the relevant cause.

References

Because each condition adds a new demonstration type, we cannot separate a third demonstration'' froman unanswerable demonstration'', or this example from unanswerable examples in general.

— Dissociating Answerability from Extraction in LLM Question Answering over Radiology Reports  (2610.17576 - Sinha et al., 13 Oct 2026) in Limitations, paragraph “One dataset and one demonstration set.”