Specificity of tail performance to creativity

Determine whether performance on rare input-output regions, as measured by q@Tail, provides a signal specific to creativity or whether it is inseparable from general model capability.

Background

The paper finds that higher-tail subsets often produce model rankings that align more closely with the LM Arena creative-writing ranking. However, the creative-writing ranking is itself strongly correlated with overall model capability and with capability-specific rankings such as instruction following, mathematics, and coding.

Consequently, the observed association between tail-conditioned performance and human creativity judgments may reflect general language, reasoning, instruction-following, or domain capabilities rather than a creativity-specific ability. The paper treats q@Tail as a diagnostic approximation and leaves the disentanglement of creativity from general capability unresolved.

References

At the same time, the current experiments do not establish that this signal is specific to creativity or separable from general capability.

— Evaluating LLM Creativity through Long-Tail Performance  (2610.12998 - Wang et al., 9 Oct 2026) in Section B.2, “Creativity and General Model Capability”