Developing a NESS Variant with Limited Access to Previous Inputs

Develop a variant of NESS (Null-space Estimated from Small Singular values) that constructs the stability subspace without requiring full access to previous task inputs, or with strictly limited access, while preserving the output perturbation bound for prior tasks and maintaining effective performance on new tasks.

Background

In NESS, constructing the stability subspace requires collecting inputs from previous tasks at each layer to form the covariance or singular value decomposition, which may be impractical in real-world deployment scenarios due to privacy, memory, or access constraints.

Earlier in the paper, the stability constraint is framed per previous input, and the authors note that only a forward pass is used to construct the subspace. Nonetheless, reducing or eliminating reliance on previous inputs remains challenging and would broaden the applicability of NESS.