Generalization of ERA’s linear adapter across domains

Establish whether the linear query-side retrieval adapter W used by Efficient Retrieval Adapter (ERA)—which maps query embeddings from a query encoder to a document encoder space—avoids overfitting to the training domains and thereby maintains generalization to out-of-domain retrieval tasks when trained via ERA’s two-stage procedure of self-supervised alignment followed by supervised adaptation.

Background

The paper evaluates ERA’s robustness when trained on all domains except a target domain and observes that performance on the held-out domain is comparable to zero-shot while improving other domains. To explain this observation, the authors hypothesize that the simplicity of the linear adapter helps prevent overfitting to the training domains.

Verifying this conjecture would clarify why ERA appears to retain out-of-domain generalization despite domain-specific training and would help determine whether the linear adapter’s simplicity is indeed the cause of the observed robustness.