2000 character limit reached
Efficient Multi-disparity Transformer for Light Field Image Super-resolution
Published 22 Jul 2024 in eess.IV and cs.CV | (2407.15329v1)
Abstract: This paper presents the Multi-scale Disparity Transformer (MDT), a novel Transformer tailored for light field image super-resolution (LFSR) that addresses the issues of computational redundancy and disparity entanglement caused by the indiscriminate processing of sub-aperture images inherent in conventional methods. MDT features a multi-branch structure, with each branch utilising independent disparity self-attention (DSA) to target specific disparity ranges, effectively reducing computational complexity and disentangling disparities. Building on this architecture, we present LF-MDTNet, an efficient LFSR network. Experimental results demonstrate that LF-MDTNet outperforms existing state-of-the-art methods by 0.37 dB and 0.41 dB PSNR at the 2x and 4x scales, achieving superior performance with fewer parameters and higher speed.
- The (new) stanford light field archive. URL http://lightfield.stanford.edu/index.html. Accessed: 2024-05-02.
- Basiclfsr: Open source light field toolbox for super-resolution. https://github.com/ZhengyuLiang24/BasicLFSR, 2023. Accessed: 2023-06-10.
- Activating more pixels in image super-resolution transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22367–22377, 2023.
- Recursive Generalization Transformer for Image Super-Resolution, February 2024.
- Exploiting Spatial and Angular Correlations With Deep Efficient Transformers for Light Field Image Super-Resolution. IEEE Transactions on Multimedia, 26:1421–1435, 2024. ISSN 1520-9210, 1941-0077. 10.1109/TMM.2023.3282465.
- Paul Debevec. Experimenting with light fields. https://blog.google/products/google-ar-vr/experimenting-light-fields/, 2018. Accessed: 2024-05-02.
- An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- The Linux Foundation. Pytorch. https://pytorch.org/. Accessed: 2024-05-02.
- Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. ISBN 978-1-4673-8851-1.
- A dataset and evaluation methodology for depth estimation on 4d light fields. In Asian Conference on Computer Vision, pages 19–34. Springer, 2016.
- Texture-Enhanced Light Field Super-Resolution With Spatio-Angular Decomposition Kernels. IEEE Transactions on Instrumentation and Measurement, 71:1–16, 2022.
- Light field inpainting propagation via low rank matrix completion. IEEE Transactions on Image Processing, 27(4):1981–1993, 2018.
- Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1833–1844, 2021.
- Light field image super-resolution with transformers. IEEE Signal Processing Letters, 29:563–567, 2022.
- Learning non-local spatial-angular correlation for light field image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12376–12386, 2023.
- Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
- Improved image classification with 4D light-field and interleaved convolutional neural network. Tools and Applications, 78(20):29211–29227, October 2019. ISSN 1573-7721.
- Presentation attack detection for face recognition using light field camera. IEEE Transactions on Image Processing, 24(3):1060–1075, 2015.
- Raytrix. 3d light field camera technology. https://raytrix.de/. Accessed: 2024-05-02.
- New Light Field Image Dataset. 8th International Conference on Quality of Multimedia Experience (QoMEX), pages 1–2, 2016.
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1874–1883. IEEE, June 2016. ISBN 978-1-4673-8851-1.
- The (new) stanford light field archive. Computer Graphics Laboratory, Stanford University, 6(7):3, 2008.
- Light Field Image Super-Resolution Network via Joint Spatial-Angular and Epipolar Information. IEEE Transactions on Computational Imaging, pages 1–16, 2023. ISSN 2333-9403, 2334-0118, 2573-0436. 10.1109/TCI.2023.3261501.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Shift-invariant-subspace discretization and volume reconstruction for light field microscopy. IEEE Transactions on Computational Imaging, 8:286–301, 2022.
- Detail preserving transformer for light field image super-resolution. In Proc. AAAI Conf. Artif. Intell., 2022a.
- A 4D light-field dataset and CNN architectures for material recognition. In European Conference on Computer Vision, pages 121–138. Springer, 2016.
- Disentangling light fields for super-resolution and disparity estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022b.
- Datasets and benchmarks for densely sampled 4D light fields. In Vision, Modelling and Visualization (VMV), volume 13, pages 225–226, 2013.
- Wikipedia contributors. Lytro — Wikipedia, the free encyclopedia. https://w.wiki/7G9s, 2020. Accessed: 2024-05-02.
- Light Field Spatial Super-Resolution Using Deep Efficient Spatial-Angular Separable Convolution. IEEE Transactions on Image Processing, 28(5):2319–2330, 2019. ISSN 10577149.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.