Deep Reinforcement Learning for Minimum Zero-Forcing Sets

Abstract: This paper explores the problem of finding the minimum zero-forcing set on undirected graphs and proposes an adapted machine-learning framework to solve the problem. The minimum zero-forcing set problem is a graph coloring problem where the color of an initial set of nodes propagates throughout a network. The set of nodes is zero-forcing if it forces all uncolored nodes to change color under the constraint of the color-change rule. There are several applications to this problem across different domains such as network science, network control, and designing logical circuits. Finding the minimum zero-forcing set is shown to be NP-hard. We propose a reinforcement learning framework, SD-ZFS, that adapts the S2V-DQN architecture to the ZFS problem. We train several models on this adapted framework and analyze the performance across graph datasets that have varying structures. We evaluate how the models trained on the framework generalize, scale, and transfer to different network types. The results demonstrate the effectiveness of the framework when compared against the optimal solution and greedy heuristic. We provide further insight into how the ZFS problem can be solved through machine-learning and the influence of network structure on the problem.

• The paper introduces SD-ZFS, a deep RL method leveraging S2V embeddings and DQN to address the NP-hard minimum zero-forcing set problem.
• It achieves near-optimal results on random and scale-free graphs, reducing ZFS size by up to 11.45% compared to greedy baselines.
• The approach generalizes across synthetic and real-world networks, demonstrating the potential of RL for scalable graph optimization.

Deep Reinforcement Learning for the Minimum Zero-Forcing Set

Problem Formulation and Zero-Forcing Sets

The minimum zero-forcing set (ZFS) problem addresses a central NP-hard graph optimization: given a graph $G=(V, E)$, identify a minimum cardinality subset of nodes $S \subseteq V$ such that, if each $v \in S$ is colored blue (with all others white), iterative application of the zero-forcing rule colors the entire graph. Under this color-change rule, a blue node $u$ can force color change in a unique uncolored neighbor $v$ if $v$ is the only white neighbor of $u$. This problem is closely tied to applications in minimum rank bounds in matrix theory, network controllability, electric grid monitoring, and logic circuit synthesis, with its combinatorial nature leading to a landscape of greedy, combinatorial, and ILP/SAT-based heuristics with varying scalability and optimality profiles.

Related Work and Limitations of Prior Methods

Classical methods for ZFS approximation have primarily leveraged local heuristics—greedy selection based on closure gains, iterative propagation, and SAT/ILP formulations [5,6]. Greedy approaches often lack theoretical guarantees on their approximation ratios and scale poorly on dense or structurally complex graphs, with complexity often $\mathcal{O}(n(n+m))$. Supervised GNNs, such as the GCN-based approach of Ahmad et al. [2], achieve computational gains by imitating greedy solutions, but typically optimize only local objectives and may struggle with generalization beyond training graph families.

SD-ZFS: Deep RL for Zero-Forcing

The paper introduces SD-ZFS, an adaptation of the S2V-DQN framework to the ZFS problem. This method reformulates ZFS construction as a sequential decision-making process, integrating Structure2Vec (S2V) graph embeddings with a Deep Q-Network (DQN). This hybrid RL architecture accepts as input the evolving state of the colored graph and selects actions (node additions to the ZFS) guided by learned q-value functions, with solution quality directly reflected in the reward signal.

Key architectural components:

• State encoding: Representation of the current colored set and node features via S2V message-passing, including local and global graph contexts.
• Action space: Unselected nodes; DQN assigns q-values to candidate nodes at each decision point.
• Reward structure: Sparse, step-penalized—negative reward for each selected node, no explicit imitation of greedy strategies.
• Policy: Deterministic greedy (select maximal q-value) with $\epsilon$-greedy exploration for RL training.

This approach contrasts with prior GCN methods by not hardwiring short-term greedy behavior, but instead optimizing the global, cumulative reward—leading, in principle, to novel, structure-sensitive heuristics.

Training and Evaluation Protocol

Training datasets: The framework is trained on Erdős–Rényi and Barabási–Albert random graphs (synthetic), as well as real-world graphs (Facebook ego-nets, COLLAB, IMDB-BINARY, REDDIT-BINARY).

Optimization: Double DQN with n-step returns and the Huber loss, with target and policy networks periodically synchronized. Replay memory enables off-policy value learning over randomly sampled mini-batches of transitions.

Key metrics: Mean difference (and deviation) of SD-ZFS over greedy and optimal solutions, and the relative approximation ratio.

Empirical Results and Analysis

Random Networks

SD-ZFS trained on small ER graphs (Small_ER) achieved a mean improvement of 4.47 nodes and an 11.45% reduction in ZFS size over the greedy baseline, with an approximation ratio of 1.048—closely matching optimal values even on larger ER graphs. This pattern persists as the framework scales to graphs of 1000 nodes, demonstrating robust generalization to size and density variations without performance degradation.

Scale-Free Networks

On Barabási–Albert (BA) scale-free graphs, SD-ZFS also outperforms or matches greedy approaches, both in small (mean gain 0.14%, near-optimal) and large (mean gain 1.18%) sizes. The framework exploits the structural regularity of power-law degree distributions, efficiently identifying nodes critical for rapid forcing.

Real-World Networks

Evaluation on real-world networks shows nuanced outcomes:

• On COLLAB networks (high clustering, scale-free tendencies), SD-ZFS provides minor gains in ZFS size.
• On IMDB-BINARY (high density and clustering), both SD-ZFS and greedy/GCN approaches converge to near-optimal solutions, reflecting the triviality of ZFS for dense, clique-like subgraphs.
• On REDDIT (hub-and-spoke structure, high leaf fraction), all models marginally underperform greedy—analyses reveal that in extreme hub-and-spoke topologies, the optimal solution converges to $|V|-2$, trivially obtainable by randomness.

Generalization, Transfer, and Network Properties

A defining outcome is that SD-ZFS generalizes well when training and test graphs share topological properties (degree distribution, clustering, assortativity). When confronted with out-of-distribution structures (e.g., hub-and-spoke in REDDIT), its learned heuristics provide no advantage over naive strategies. This aligns with theoretical observations that structural graph properties (e.g., density, clustering coefficient, degree gini) fundamentally affect ZFS complexity and reward landscape.

Performance Against GCN Architectures

Comparative trials show SD-ZFS achieves small but consistent improvements in ZFS sizes over GCNs trained to imitate greedy heuristics, particularly when faced with low-density, structurally diverse random graphs—demonstrating the advantage of direct reward-driven RL in discovering structure-exploiting policies beyond shallow heuristics.

Implications and Future Directions

The results demonstrate that deep RL architectures, provided with effective message-passing embeddings, can outperform classical greedy and GCN-based heuristics on zero-forcing set construction—particularly when network structure is complex but repetitive across samples. However, in certain pathological or trivial classes (e.g., complete graphs, hub-and-spoke topologies), further architectural or objective function innovations are needed to induce adequate generalization.

Theoretical implications: The demonstrated transferability and approximate optimality highlight the suitability of RL for graph-theoretic problem domains where reward is aligned with structural properties not easily captured by local heuristics. The instability in performance across graph families suggests opportunities for transfer learning, meta-learning, or adaptive curriculum strategies.

Practical outlook: As network control and monitoring tasks in infrastructure and science increasingly rely on rapid heuristics for large graphs, RL-based ZFS policies offer a compelling path to scalable, high-quality approximations—provided domain-specific graph properties are reflected in training distributions.

Possible research directions:

• Incorporating explicit structural features or augmenting the reward function to recognize and adapt to trivial subgraph classes (e.g., trees, clique/complete, hub-centric structures).
• Meta-RL or continual learning to enable rapid adaptation to unseen graph distributions.
• Expanding to stochastic or time-evolving networks, where the ZFS must account for dynamic topologies.

Conclusion

SD-ZFS, an RL-augmented S2V-DQN framework for minimum zero-forcing set computation, reliably achieves or improves over conventional heuristics across synthetic and real-world graphs, provided structural properties are adequately represented in the training data. Its performance illustrates both the strengths of deep RL in combinatorial optimization and the persistent challenge of achieving true dataset-agnostic generalization in graph algorithms. Further progress will require finer integration of structural signal, adaptive objectives, and cross-family transfer mechanisms.