Efficient Inference via Universal LSH Kernel

Abstract: Large machine learning models achieve unprecedented performance on various tasks and have evolved as the go-to technique. However, deploying these compute and memory hungry models on resource constraint environments poses new challenges. In this work, we propose mathematically provable Representer Sketch, a concise set of count arrays that can approximate the inference procedure with simple hashing computations and aggregations. Representer Sketch builds upon the popular Representer Theorem from kernel literature, hence the name, providing a generic fundamental alternative to the problem of efficient inference that goes beyond the popular approach such as quantization, iterative pruning and knowledge distillation. A neural network function is transformed to its weighted kernel density representation, which can be very efficiently estimated with our sketching algorithm. Empirically, we show that Representer Sketch achieves up to 114x reduction in storage requirement and 59x reduction in computation complexity without any drop in accuracy.

• The paper introduces the Representer Sketch framework, which transforms neural network inference into efficient kernel density estimation using Locality Sensitive Hashing (LSH).
• Representer Sketch significantly reduces memory footprint (up to 114x) and computational complexity (up to 59x) for neural network inference, enabling deployment on resource-constrained devices.
• Evaluated on various datasets, the method achieves accuracy comparable to baselines and offers better memory-performance trade-offs than traditional pruning or distillation.

The paper "Efficient Inference via Universal LSH Kernel" investigates the challenge of deploying large neural networks, which often require substantial computational resources, in resource-constrained environments such as mobile devices or embedded systems. The proposed solution is the Representer Sketch, a method that leverages Locality Sensitive Hashing (LSH) and kernel methods to significantly reduce memory and computational demands during the inference phase of neural networks while maintaining accuracy.

Core Concepts and Contributions:

1. Representer Sketch Framework:
   • The Representer Sketch is inspired by the Representer Theorem from functional analysis, which allows any learned function, including those expressed by neural networks, to be represented as weighted sums of kernel functions.
   • By transforming a neural network's inference function into a weighted kernel representation, the Representer Sketch uses LSH to efficiently approximate this kernel density estimation.
2. Locality Sensitive Hashing (LSH):
   • LSH functions are used to map similar data points to the same hash with high probability, enabling efficient data aggregation and reducing computational complexity.
   • The paper discusses using LSH kernels, which are a subclass of universal kernels capable of approximating any continuous bounded function over a compact domain.
3. Efficient Inference:
   • The paper achieves a substantial reduction in the memory footprint (up to 114x) and computational complexity (up to 59x) without sacrificing accuracy. This reduction is critical for enabling complex models to run on devices with limited resources.
   • Representer Sketch bypasses the traditional need for large matrix multiplications by using simple operations such as addition and subtraction, thereby achieving energy-efficient inference.
4. Comparison with Existing Methods:
   • Compared to techniques like pruning, quantization, and knowledge distillation, Representer Sketch provides a more fundamental transformation of the neural network that abstracts away from its parameterization into kernel sums.
   • It offers better accuracy across different memory budgets compared to commonly used pruning and distillation methods.
5. Theoretical Guarantees:
   • The paper provides theoretical analysis and proofs to substantiate the unbiased nature and bounded error of its kernel density estimates. It extends the RACE framework for efficient kernel density estimation to handle weighted sums.
6. Empirical Evaluation:
   • Representer Sketch is evaluated on various datasets (from the UCI repository), showing consistent performance across different domains and tasks, including classification and regression.
   • The results demonstrate that it achieves comparable accuracy to baseline neural networks and provides more favorable trade-offs between memory usage and performance.

Overall, the Representer Sketch presents a novel and theoretically grounded approach to efficient neural network inference, particularly suited for environments with stringent resource constraints. By utilizing LSH and compact kernel representations, it enables practical deployment of large-scale models without the associated high computational costs.