2000 character limit reached
Ensemble Mask Networks
Published 12 Sep 2023 in cs.LG and cs.AI | (2309.06382v2)
Abstract: Can an $\mathbb{R}n\rightarrow \mathbb{R}n$ feedforward network learn matrix-vector multiplication? This study introduces two mechanisms - flexible masking to take matrix inputs, and a unique network pruning to respect the mask's dependency structure. Networks can approximate fixed operations such as matrix-vector multiplication $\phi(A,x) \rightarrow Ax$, motivating the mechanisms introduced with applications towards litmus-testing dependencies or interaction order in graph-based models.
- Deep neural network ensembles for time series classification. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–6, 2019. doi: 10.1109/IJCNN.2019.8852316. URL https://ieeexplore.ieee.org/document/8852316/.
- Neural graphical models, 2023. URL https://arxiv.org/abs/2210.00453.
- An operator theoretic view on pruning deep neural networks, 2022. URL https://arxiv.org/abs/2111.09839.
- Training neural networks with fixed sparse masks, 2021. URL https://arxiv.org/abs/2111.09839.
- Neural arithmetic logic units, 2018. URL https://arxiv.org/abs/2001.05016.
- Sebastian Urban and Patrick van der Smagt. A neural transfer function for a smooth and differentiable transition between additive and multiplicative interactions, 2016. URL https://arxiv.org/abs/1503.05724.
- What neural networks memorize and why: Discovering the long tail via influence estimation. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546. URL https://dl.acm.org/doi/abs/10.5555/3495724.3495966.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.