Graph Normalization: Fast Binarizing Dynamics for Differentiable MWIS

Abstract: We introduce Graph Normalization (GN), a principled dynamical system on graphs that serves as a differentiable approximation engine for the NP-hard Maximum Weight Independent Set (MWIS) problem. MWIS encompasses many combinatorial challenges, including optimal assignment, scheduling, set packing, and MAP inference in discrete Markov Random Fields. Unlike Belief Propagation, we prove GN always converges to a binary indicator of a Maximum Independent Set. GN realizes a fast quasi-Newton descent through an exact Majorization-Minimization step, systematically improving the MWIS relaxed primal objective. We establish an equivalence between GN and the Replicator Dynamics of a nonlinear evolutionary game, where vertices compete for inclusion in an independent set. While a non-potential game, the GN game follows Fisher's Fundamental Theorem of Natural Selection, where the average fitness equals the MWIS primal objective and strictly increases. This connection leads to a weighted extension of the Motzkin-Straus theorem, showing MISes are in bijection with the local minima of a quadratic form over a tilted simplex. For the Assignment Problem, GN acts as a variant of the Sinkhorn algorithm that naturally converges to a hard assignment while generalizing to arbitrary constraint graphs. We demonstrate GN's performance as a fast binarization engine for the state-of-the-art Bregman-Sinkhorn relaxed MWIS solver. On real-world benchmarks with up to 1M edges, GN identifies solutions within 1% of the best known results in seconds on a CPU. GN opens new avenues for deep learning architectures requiring differentiable, "hard" decisions under constraints, with applications in structured sparse attention, dynamic network pruning, and Mixture-of-Experts. Beyond core AI, the GN framework enables end-to-end learning of constrained optimization in computer vision, computational biology, and resource allocation.

• The paper introduces Graph Normalization, a principled iterative normalization operator that bridges continuous relaxations and combinatorial MWIS feasibility.
• It employs a regularized weighted variant (WRGN) that performs quasi-Newton descent on a quadratic energy function, ensuring rapid convergence to binary solutions.
• Empirical results demonstrate state-of-the-art performance on large-scale benchmarks, achieving near-optimal MWIS solutions in subsecond times.

Graph Normalization: Differentiable Dynamics for Efficient MWIS Binarization

Formal Framework and Theoretical Foundations

The paper introduces Graph Normalization (GN), a dynamical system crafted to operate as a differentiable engine for the Maximum Weight Independent Set (MWIS) problem. MWIS, known for its NP-hardness and broad relevance in combinatorial optimization, encompasses diverse applications such as resource allocation, scheduling, assignment, and MAP inference in MRFs. Standard approaches, including LP relaxations and Graph Neural Network (GNN)-based solvers, suffer from significant binarization deficits, typically producing relaxed (fractional) outputs requiring costly and heuristic rounding procedures.

GN offers a principled iterative normalization operator that generalizes Sinkhorn-style normalization to arbitrary graph topologies, effectively bridging the gap between continuous relaxations and combinatorial feasibility. Iterative Graph Normalization (IGN) updates each node's value by normalizing its weight with respect to its (closed) neighborhood, establishing a dynamic where nodes systematically compete for inclusion in an independent set.

The regularized weighted variant, WRGN, further incorporates a structural regularization parameter $\gamma$ and node weight scaling, embedding MWIS objectives directly into the fixed-point topology. Theoretical analysis shows that WRGN realizes exact Majorization-Minimization (MM) updates, constituting a fast quasi-Newton descent on a quadratic energy function, strictly decreasing the energy at each iteration and guaranteeing convergence to a unique fixed point.

Importantly, for $\gamma > 1$, WRGN undergoes a topological phase transition in its energy landscape, ensuring only binary solutions—corresponding to maximal independent sets—are stable attractors, while all fractional fixed points become strictly repulsive. This eliminates the reliance on annealing schedules or entropy regularization, providing robust binarization natively in the system dynamics.

Evolutionary Game Theory Connection and Generalized Motzkin-Straus Theorem

GN is shown to be formally equivalent to nonlinear Replicator Dynamics in an evolutionary game setting, where the population average fitness equals the MWIS primal objective and increases monotonically throughout the dynamics. This perspective situates GN as a rare globally convergent non-potential replicator system—a significant distinction from classical linear replicator dynamics which rely on potential games and exhibit more complex behavior such as chaos or limit cycles in non-potential settings.

Building on this interpretation, the paper establishes a geometric extension of the Motzkin-Straus theorem for weighted graphs: MWIS solutions are bijective to the local minima of a quadratic form on the weight-tilted simplex, with regularization $\gamma$ acting as a selectivity filter. This grants a rigorous characterization of the optimization landscape, where the structural and weighted constraints are encoded directly in the geometry of the constraint manifold, rather than via algorithmic procedures.

Algorithmic Instantiation and Integration in Differentiable Pipelines

GN is implemented efficiently and shown to act as a high-speed binarization layer for relaxed solutions produced by state-of-the-art MWIS solvers, such as the Bregman-Sinkhorn algorithm (Haller et al., 2024). In practice, GN serves both as a stand-alone solver and as a bridge converting fractional MWIS outputs to valid integer solutions, enabling end-to-end differentiable learning—a key requirement for modern neural architectures that must enforce "hard" structured decisions under constraint.

The paper describes a y-pursuit algorithm, where the regularization parameter $\gamma$ is gradually increased, tracking the global minimum trajectory and exploiting phase transitions for robust binarization. Two initialization strategies are assessed: uniform random starts and warm starts seeded by fractional solutions, the latter yielding superior start basins and convergence to near-optimal solutions.

Empirical Performance and Numerical Results

On large-scale real-world MWIS benchmarks (Amazon Vehicle Routing, Meta-Segmentation for Cell Detection), GN demonstrates unmatched efficiency—producing solutions within a 1% gap of best known results, even for graphs with up to 1 million nodes and hundreds of millions of edges. GN requires only seconds on commodity hardware to achieve these results, outperforming greedy heuristics and rounding-based post-processing, especially when warm-started from Bregman-Sinkhorn fractional solutions.

Strong numerical results include convergence to best known solutions in subsecond times for moderate graphs (AVR_024, AVR_034), and under 1% gaps for instances up to 1M edges. For hard instances, GN reliably closes the fractional-to-integral gap orders of magnitude faster than existing rounding schemes. Notably, GN consistently achieves strictly increasing MWIS objectives per iteration, confirming theoretical predictions.

Implications and Future Directions

Practically, GN enables differentiable binarization in MWIS and related combinatorial problems, supporting end-to-end learning in architectures requiring discrete control—dynamic network pruning, structured sparse attention, and Mixture-of-Experts (MoE) models. Theoretically, the integration of MM-based energy descent, nonlinear replicator dynamics, and regularized geometric optimization provides a fertile foundation for continuous formulations of other NP-hard combinatorial tasks.

The paper suggests further extension to hypergraph constraints and continuous-time dynamics, reconstructing combinatorial optimization as a continuous geometric flow suitable for deep learning integration. Such directions indicate GN's potential for catalyzing developments in structured optimization within machine learning, computer vision, computational biology, and resource allocation.

Conclusion

GN establishes a mathematically robust, differentiable mechanism for resolving the combinatorial bottleneck in MWIS and related problems, aligning fractional relaxation and discrete feasibility while ensuring convergence, scalability, and integration with gradient-based learning. The system is theoretically well-grounded in energy descent, game dynamics, and geometric optimization, and empirically validated on large-scale instances. Its differentiability and binarization guarantees make it an attractive candidate for deep architectures handling constrained combinatorial tasks (2605.05330). Future work on hypergraph generalization and continuous-time analogs may further enhance the applicability and speed of GN-based optimization in production-scale environments.