Verifying In-Network Computing Systems for Design Risks

Abstract: The emergence of programmable switches has brought in-network computing (INC) into the spotlight in recent years. By offloading computation directly onto the data transmission process, INC improves network utilization, reduces latency to sub-RTT levels, saves link bandwidth, and maintains throughput. However, INC disrupts the transparency of traditional networks, forcing developers to consider network exceptions like packet loss and out-of-order. If not properly handled, these exceptions can lead to violations of application properties, such as cache consistency and lock exclusion. Usual testing cannot exhaustively cover these exceptions, raising doubts about the correctness of INC systems and hindering their deployment in the industry. This paper presents INCGuard, the first general-purpose tool for verifying INC systems. INCGuard provides a high-level specification language and saves developers 67.2% lines of code on average. To help better understand the behavior of the system, INCGuard offers configurable network environments. INCGuard enables developers to express INC-specific correctness properties. INCGuard translates developer-specified systems into state transition representations, performs model checking to detect potential design risks, and reports violation traces to developers. We propose optimizations for INC-specific scenarios to address the challenge of state space explosion. We modeled seven INC systems and identified their risks with INCGuard in seconds. We further reproduce them in real systems to confirm the validity of our verification result.

• The paper introduces INCGuard, the first tool that uses a domain-specific specification language and model checking to verify design risks in in-network computing systems.
• It leverages input space and symmetry reduction techniques to mitigate state space explosion, reducing verification time by up to 88.1% in practical scenarios.
• Empirical evaluations on systems like SwitchML and NetCache demonstrate that formal verification can effectively uncover rare safety and liveness violations before production deployment.

Formal Verification of In-Network Computing Systems with INCGuard

Introduction

The rapid adoption of programmable switches has driven the proliferation of in-network computing (INC), enabling fundamental changes to how networked systems achieve low-latency computation and improved resource utilization. However, the deployment of INC systems introduces nontrivial correctness challenges, particularly due to network unreliability (e.g., packet loss, duplication, out-of-order delivery) and limited switch programmability. This paper introduces INCGuard (2604.10186), the first general-purpose tool for verifying INC systems with respect to design risks, targeting both communication and computation correctness.

INCGuard aims to bridge the gap between the demands of INC application developers—who require rigorous correctness guarantees—and the inadequacy of both traditional network verification and ad-hoc testing approaches. By providing a domain-specific specification language, configurable network modeling, and scalable model checking, INCGuard detects subtle protocol design flaws missed by conventional techniques and aids iterative protocol refinement.

Figure 1: The workflow of INCGuard spans high-level protocol modeling (nodes, links) and the underlying formal state-space analysis through model checking.

INCGuard: Architecture and Specification Model

INCGuard abstracts an INC system as a set of nodes (machines) with multiple concurrent threads communicating by passing packets over a network composed of configurable links. Threads synchronize using local state and interleave according to all possible event orders, reflecting system concurrency and real-world network nondeterminism.

The INCGuard specification language supports:

• Configuration: Users define network environment characteristics (reliable/lossy/out-of-order), network nondeterminism constraints (e.g., packet loss limits), and protocol constants.
• Topology: Explicit modeling of node types, instantiations, interconnections, and customizable routing tables.
• Protocol: State variables and thread routines parameterized by node type. The language introduces high-level constructs for packet operations, synchronization, and breakpoint management to control model checking granularity.
• Properties: Correctness properties are expressed as assertions or CTL formulas, encompassing both state (“safety”) and path (“liveness”) invariants.

The modeling approach privileges system design verification (protocol-level reasoning) over implementation verification, enabling early error detection before committing to P4 or C/C++ code.

Figure 2: The workflow of INCGuard closely mirrors the logical structure of INC protocols—from network components down to state transitions during verification.

Verification Engine and Model Checking Optimizations

INCGuard compiles protocol specifications into TLA+-based state transition representations, leveraging mature model checking infrastructure to exhaustively explore state spaces with strong fairness constraints.

To address state space explosion, which is exacerbated in data-intensive and concurrent INC workloads, INCGuard implements:

• Input space reduction: Exploits per-object data independence to restrict analysis scope to a small number of operations per item, ensuring coverage without explicit cross-object enumeration.
• Symmetry reduction: Identifies and eliminates redundant exploration by recognizing permutations of nodes of the same type, e.g., clients, thereby dramatically compressing state space.

Figure 3: Symmetry reduction yields significant reduction in both verification time and unique states, as demonstrated on systems like SwitchML.

• Network nondeterminism constraints: Users can cap the degree of loss, out-of-order, or duplication per trace, enabling practical analysis without loss of bug-finding fidelity.

Figure 4: The impact of network nondeterminism—packet loss, out-of-order, and duplication—on state space size and analysis time is empirically characterized.

Evaluation: Coverage, Efficiency, and Property Violations

INCGuard is evaluated by modeling and verifying seven representative INC systems covering tensor aggregation (SwitchML, ATP, NetReduce), key-value caches (NetCache, FarReach), and lock managers (NetLock, FissLock). The tool is shown to reduce specification lines of code by an average of 67.2% compared to baseline TLA+, demonstrating concise modeling expressivity.

Strong numeric results include:

• Detection time for first property violation is consistently low—most protocols reveal design flaws in seconds to minutes with reasonable system parameters.
• Symmetry reduction decreases verification time by up to 88.1% and the number of distinct states by 95.5%.
• Model checking faithfully reproduces complex failure modes in practical code bases when guided by violation traces.

Figure 5: Verification time, state space diameter, and size on NetCache illustrate the practical feasibility of exhaustive analysis under INCGuard.

Confirmed property violations include:

• In SwitchML and ATP, out-of-order delivery leads to computation and memory leak failures due to untracked request state, violating correctness and liveness guarantees.
• In NetCache and FarReach, specific loss/interleaving patterns cause cache consistency violations not captured by the request/response programming idioms.
• NetLock and FissLock suffer exclusion violations via retransmission or delayed grant/release, arising from limited switch queue manipulation primitives and agent migration races.

  Figure 6: Execution trace of cache consistency violation in NetCache (all operations on a single item). Violations arise under carefully constructed packet loss patterns.

  

  Figure 7: Execution trace of cache consistency violation in FarReach; subtle event interleaving disrupts consistency guarantees despite a reliable network.

  

  Figure 8: Lock exclusion violation in FissLock—agent migration and grant delay create states where exclusive access is lost.

All of these violation patterns have been induced in real INC system implementations in containerized testbeds, confirming the model's predictive power and the practical relevance of design-time verification.

Implications and Future Research Directions

Practically, the results underscore the necessity of exhaustive, formal analysis for INC protocols prior to production deployment. The observed error cases, although rare, can result in severe safety/liveness property violations and are difficult or infeasible to uncover through conventional workload testing or simulation.

Theoretically, the work points to open problems in reasoning about correctness in constrained, asynchronous, and highly concurrent data planes. Current approaches for performance-oriented switch programming insufficiently address compositionality under faults and concurrency. There is an identified need for richer hardware- and implementation-aware abstractions, and for integrating protocol synthesis with formal design verification loops.

Several directions for future research are evident:

• Development of custom model checkers tailored for INC systems with further optimization beyond off-the-shelf tools.
• Automated extraction/translation from implementation code to specification models, closing the gap between design and deployment.
• Hardware-specific modeling of data plane execution semantics (e.g., Tofino’s pipeline model) to ensure abstraction fidelity.
• Incorporation of counterexample-guided refinement for protocol repair.
• Broader application of INCGuard’s principles to other domains where network-embedded computation or stateful data planes are critical.

Conclusion

INCGuard provides a rigorous, configurable, and practical framework for verifying design risks in INC systems, capturing nuanced property violations arising from concurrency, network unreliability, and hardware constraints. Its abstractions, optimizations, and empirical effectiveness make it a critical tool for ensuring the safe adoption of in-network computing architectures and protocols. The work implies that formal verification must be a standard step in modern network protocol development and deployment pipelines.