Verifying Properties of Binarized Deep Neural Networks (1709.06662v2)

Abstract: Understanding properties of deep neural networks is an important challenge in deep learning. In this paper, we take a step in this direction by proposing a rigorous way of verifying properties of a popular class of neural networks, Binarized Neural Networks, using the well-developed means of Boolean satisfiability. Our main contribution is a construction that creates a representation of a binarized neural network as a Boolean formula. Our encoding is the first exact Boolean representation of a deep neural network. Using this encoding, we leverage the power of modern SAT solvers along with a proposed counterexample-guided search procedure to verify various properties of these networks. A particular focus will be on the critical property of robustness to adversarial perturbations. For this property, our experimental results demonstrate that our approach scales to medium-size deep neural networks used in image classification tasks. To the best of our knowledge, this is the first work on verifying properties of deep neural networks using an exact Boolean encoding of the network.

• The paper introduces a SAT-based method to provide an exact Boolean representation of BNNs and verify their robustness and equivalence properties.
• The authors design an encoding strategy that transitions from MILP to ILP and finally to SAT, leveraging sequential counters for enhanced scalability.
• Counterexample-guided search with Craig interpolants refines the verification process, significantly improving speed and accuracy over traditional methods.

Verifying Properties of Binarized Deep Neural Networks

The paper "Verifying Properties of Binarized Deep Neural Networks" by Narodytska et al. presents a novel approach to understand and verify the properties of Binarized Neural Networks (BNNs) using Boolean satisfiability (SAT). This method offers an exact Boolean representation of BNNs and leverages modern SAT solvers alongside a counterexample-guided search procedure to verify important properties such as robustness to adversarial perturbations and network equivalence.

BNNs, known for their efficiency in resource-constrained environments due to binary parameters and activations, achieve competitive performance compared to traditional neural networks. The authors focus on examining these networks through SAT techniques, making it possible to investigate BNN properties within the SAT domain. The encoding developed is both functional and structural, benefiting from insights into BNNs' predominantly binary nature and modular structure.

Boolean Encoding of BNNs

The authors first propose a Mixed Integer Linear Programming (MILP) encoding of individual network layers, grounding their approach in conventional practices seen in prior deep network verification efforts. They transition from MILP to Integer Linear Programming (ILP) and ultimately derive a comprehensive SAT encoding. The SAT encoding utilizes sequential counters to manage cardinality constraints efficiently, enhancing the scalability of the approach.

Verification of Network Properties

The encoding supports verifying properties such as:

1. Adversarial Robustness: BNNs can be verified for robustness against small perturbations by formulating adversarial conditions as SAT problems. Robustness here is framed in terms of perturbation norms ($L_1$, $L_\infty$).
2. Network Equivalence: Verification entails ensuring consistency between network outputs across various inputs, highlighting concerns like model reduction and transformation invariance.

Counterexample-Guided Search

To tackle the potential high computational cost of large SAT encodings, the authors introduce a counterexample-guided search strategy using Craig interpolants. This efficient method leverages the network's modular structure, iteratively refining the solution space by extracting critical unsatisfiable cores and interpolants.

Experimental Evaluation

Experimental validation on datasets such as MNIST demonstrate the effectiveness of the SAT-based approach. The use of counterexample-guided search accelerates the certification of BNN properties, outperforming traditional MILP and ILP methods in both speed and solvability. The experiments confirmed robustness in several instances, with the SAT encoding enabling deeper insights into adversarial perturbations.

Implications and Future Work

The authors contribute significantly to the field by offering an exact procedure for BNN verification, which could extend to more general neural network structures. While the approach shows promise in handling network robustness and equivalence properties, enhancing scalability to manage even larger networks remains a pivotal challenge. Future work suggests optimizing formula structuring and exploring fixed-bit representation networks.

In conclusion, this research effectively bridges BNNs and SAT, setting a foundation for robust neural network verification. The innovative application of SAT solvers and counterexample refinement strategies presents a formidable direction for advancing the reliability and understanding of machine learning models in practical applications.