Non-aliased control for mechanistic geometry

Construct a non-aliased control in which the competing rules are separable, and measure the corresponding optimization geometry to determine whether the geometry found for aliased rules is absent in the non-aliased case.

Background

The paper constructs a synthetic language in which RECENCY and RARITY select the same answer on every training document. Because the two rules are coextensive, the training objective is indifferent between mechanisms implementing them, and the authors observe substantial variation in the intervention-based readout across seeds and training schedules.

The authors attempted to compare the geometry of the aliased readout with directions associated with objective-constrained readouts, but that measurement was uninformative. Every cell in the reported construction is aliased, so the paper does not contain a matched non-aliased condition in which the same geometric quantities could be evaluated when the objective distinguishes the competing rules.