ThinkingViT: Elastic Inference Vision Transformer
- ThinkingViT is a nested Vision Transformer that enables elastic inference by adjusting computational resources based on input difficulty.
- It employs progressive stages with attention head expansion and token recycling to refine predictions and improve performance incrementally.
- On ImageNet-1K, ThinkingViT achieves up to 81.44% top-1 accuracy at reduced compute, demonstrating an effective accuracy–efficiency trade-off.
77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77max_results 77 77 77 77 OR abs:\777 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 OR abs:\777 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77^ 77 77^ 77 77 77 77 77 77 77 77 77^ 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 OR abs:\777 77 77 OR abs:\777 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77^ 77 OR abs:\777 77^ 77 77^ 77 OR abs:\777 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77max_results 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77max_results 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77max_results 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77max_results 77 77 77 77 77 77 OR abs:\777 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 OR abs:\777 77 77 77 77max_results 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77max_results 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77^( OR abs:\777 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 OR abs:\777 77^( OR abs:\777 77^ 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77 77