Learned or principled adaptive patch-size allocation
Determine a principled or learnable policy for assigning patch sizes that replaces the current hand-crafted heuristic and aligns with downstream task preferences, while preserving APT’s speed and accuracy benefits.
References
Although APT provides significant speedups, it still relies on a hand-crafted heuristic to determine patch sizes, which may not always align with downstream users' preferences and could likely be improved.
— Accelerating Vision Transformers with Adaptive Patch Sizes
(Choudhury et al., 20 Oct 2025) in Conclusion, Limitations