Mapping Image Transformations Onto Pixel Processor Arrays (2403.16994v1)
Abstract: Pixel Processor Arrays (PPA) present a new vision sensor/processor architecture consisting of a SIMD array of processor elements, each capable of light capture, storage, processing and local communication. Such a device allows visual data to be efficiently stored and manipulated directly upon the focal plane, but also demands the invention of new approaches and algorithms, suitable for the massively-parallel fine-grain processor arrays. In this paper we demonstrate how various image transformations, including shearing, rotation and scaling, can be performed directly upon a PPA. The implementation details are presented using the SCAMP-5 vision chip, that contains a 256x256 pixel-parallel array. Our approaches for performing the image transformations efficiently exploit the parallel computation in a cellular processor array, minimizing the number of SIMD instructions required. These fundamental image transformations are vital building blocks for many visual tasks. This paper aims to serve as a reference for future PPA research while demonstrating the flexibility of PPA architectures.
- Michael JB Duff et al. Review of the CLIP image processing system. In Proc. National Computer Conference, pages 1055–1060. AFIPS Press Arlington, Va, 1978.
- System design for pixel-parallel image processing. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 4(1):32–41, 1996.
- A cmos vision chip with simd processing element array for 1ms image processing, 1999 dig. tech. papers of 1999 ieee int. In Solid-State Circuits Conf.(ISSCC99)(San Francisco, 1999.2. 16)/Abst, pages 206–207.
- A general-purpose cmos vision chip with a processor-per-pixel simd array. In Proceedings of the 27th European Solid-State Circuits Conference, pages 213–216. IEEE, 2001.
- MIPA4k: A 64×\times× 64 cell mixed-mode image processor array. In 2009 IEEE International Symposium on Circuits and Systems, pages 1927–1930. IEEE, 2009.
- A general-purpose vision processor with 160x80 pixel-parallel SIMD processor array. In Proceedings of the IEEE Custom Integrated Circuits Conference, 2017.
- CMOS vision sensors: embedding computer vision at imaging front-ends. IEEE Circuits and Systems Magazine, 18(2):90–107, 2018.
- A 100,000 fps vision sensor with embedded 535GOPS/W 256×\times× 256 SIMD processor array. In 2013 Symposium on VLSI Circuits, pages C182–C183. IEEE, 2013.
- A 1ms high-speed vision chip with 3d-stacked 140 GOPS column-parallel PEs for spatio-temporal image processing. In 2017 IEEE International Solid-State Circuits Conference (ISSCC), pages 82–83. IEEE, 2017.
- A 5500-frames/s 85-GOPS/W 3-d stacked BSI vision chip based on parallel in-focal-plane acquisition and processing. IEEE Journal of Solid-State Circuits, 54(4):1096–1105, 2019.
- A 1280×\times× 720 back-illuminated stacked temporal contrast event-based vision sensor with 4.86 μ𝜇\muitalic_μm pixels, 1.066 GEPS readout, programmable event-rate controller and compressive data-formatting pipeline. In 2020 IEEE International Solid-State Circuits Conference-(ISSCC), pages 112–114. IEEE, 2020.
- Feature extraction using a portable vision system, 2017.
- A camera that cnns: Towards embedded neural networks on pixel processor arrays. In The IEEE International Conference on Computer Vision (ICCV), October 2019.
- Visual odometry for pixel processor arrays. In Proceedings of the IEEE International Conference on Computer Vision, pages 4604–4612, 2017.
- Vision chips with in-pixel processors for high-performance low-power embedded vision systems. In ASR-MOV Workshop, CGO, volume 6, page 14, 2016.
- AUKE: Automatic kernel code generation for an analogue SIMD focal-plane sensor-processor array. ACM Transactions on Architecture and Code Optimization (TACO), 15(4):59, 2019.