Identification without discrimination in generatively trained natural-language autoencoders

Establish whether an objective that only reconstructs an activation from its natural-language explanation can yield perfect identification of the activation while remaining uninformative for discriminating true accounts of that activation from fluent false accounts that differ by only a small claim, including the proposed existence, Lipschitz-bound, information-theoretic, and genericity formulations.

Background

Natural Language Autoencoder training is formalized as a generative referential game: the Activation Verbaliser produces an explanation of an activation and the Activation Reconstructor attempts to reconstruct that activation. The training objective compares the reconstruction with the target activation but never requires the explanation to exclude competing accounts of the same activation.

The paper distinguishes provenance identification—recognizing which activation an explanation came from—from truth discrimination—ranking a true account above a fluent rival that falsifies one or more claims. Experiments on one released checkpoint found perfect identification but only barely-above-chance discrimination, motivating a conjecture that generative reconstruction objectives systematically reward activation-specific information rather than truth-relevant distinctions. The authors explicitly state several increasingly strong unresolved formulations, ranging from an existence construction to a genericity claim about minimizers of the generative objective.