Dice Question Streamline Icon: https://streamlinehq.com

Theoretical understanding of input-dependent LTV SSM dynamics

Develop a theoretical characterization of linear time-varying state-space models whose dynamic matrices depend on the current input, as used in selective SSM architectures such as S6/Mamba and RG-LRU, clarifying their fundamental properties and implications for sequence modeling.

Information Square Streamline Icon: https://streamlinehq.com

Background

Selective SSMs (e.g., Mamba/S6, RG-LRU) parameterize the system matrices as explicit functions of the current input, introducing input-dependent LTV dynamics that are more expressive but also more complex to analyze.

The authors emphasize that the theoretical properties arising from these input-dependent dynamics remain poorly understood in the SSM literature, highlighting a need for formal analysis to support explainability, principled design, and performance guarantees.

References

However, in the SSM literature the theoretical properties arising from these dynamics remains poorly understood.

State Space Models as Foundation Models: A Control Theoretic Overview (2403.16899 - Alonso et al., 25 Mar 2024) in Section 5, Conclusion and Future Opportunities