Anonymized Network Sensing Graph Challenge

• The challenge defines an end-to-end pipeline that transforms raw packet captures into anonymized source-destination traffic matrices, preserving essential network structure and privacy.
• It leverages sparse linear algebra and hypersparse matrix representations to analyze network metrics, ensuring reproducibility and scalable performance across systems.
• Benchmark evaluations include processing billions of packets with metrics on execution time, throughput, and system scalability while emphasizing legal and technical privacy safeguards.

The Anonymized Network Sensing Graph Challenge is a MIT/IEEE/Amazon GraphChallenge problem that defines a privacy-preserving, scalable pipeline for converting packet captures into anonymized source–destination traffic matrices and then analyzing those matrices with sparse linear algebra. Its stated goal is to enable large, open, community-based approaches to protecting networks by sharing broad observatory data with “the highest regard for privacy,” while preserving enough structure to compute meaningful network properties at Internet scale. The challenge uses over 100 billion network packets derived from the CAIDA UCSD Network Telescope, provides a GraphBLAS reference implementation without requiring GraphBLAS, and emphasizes correctness, performance, anonymization, and reproducibility (Jananthan et al., 2024, Voloshchuk et al., 2 Jan 2026).

1. Institutional setting and challenge scope

The challenge sits within the broader GraphChallenge program, which frames graph and sparse-data problems as reproducible benchmarks spanning algorithms, systems, software, and hardware. Within that program, the Anonymized Network Sensing Graph Challenge is distinguished by its end-to-end character: it begins with raw packet data, continues through IP extraction and anonymization, constructs sparse traffic matrices, stores them, and finally computes network quantities over the resulting matrices. The intended data product is an anonymized source-to-destination traffic matrix that can be shared under standard data sharing agreements while still supporting network analysis (Jananthan et al., 2024).

The reference benchmark uses PCAP files of size $N_p = 2^{30}$ packets. For the real dataset, the packet captures are derived from the CAIDA UCSD Network Telescope, with selections around noon and midnight in the first quarter of 2022. The challenge also includes a freely available randomized $2^{30}$-packet PCAP generated with $2 \times 2^{30}$ calls to the PCG32 PRNG, so that correctness and performance can be evaluated independently of restricted-access data (Jananthan et al., 2024).

The challenge is defined as a sequence of timed steps. These include reading or streaming a $2^{30}$-packet PCAP file, extracting source and destination IP addresses, anonymizing those addresses consistently across files, constructing sequential traffic matrices with $N_V = 2^{17}$ valid packets per matrix, saving those matrices, reading them back, summing them into a single aggregate matrix $A_t$, and computing the specified aggregate properties. Success is evaluated in terms of correctness and performance, with metrics such as total packets processed, execution time, rate in packets per second, and processor configuration (Jananthan et al., 2024).

2. Data model, anonymization, and matrix semantics

The canonical traffic representation is a sparse adjacency matrix over anonymized IP space. For a time bin $t$, the challenge defines a traffic matrix $T(t)$ with entries

$$T_{ij}(t) \;=\; \sum_{p \in \mathcal{P}(t)} w(p)\,\mathbf{1}[src(p)=i,\,dst(p)=j],$$

where $w(p)$ is typically $2^{30}$0 for packet counts, though byte-weighted variants are also admissible when explicitly declared (Jananthan et al., 2024). In the reference implementation’s aggregation stage, the stored matrices are summed as

$2^{30}$1

This reflects the challenge’s file organization: a $2^{30}$2-packet PCAP is partitioned into $2^{30}$3 matrices, each built from $2^{30}$4 packets, with $2^{30}$5 matrices per output file and $2^{30}$6 output files per PCAP (Jananthan et al., 2024, Voloshchuk et al., 2 Jan 2026).

Anonymization in the reference path is performed on IPv4 source and destination addresses, with CryptoPAN used as the stated method. CryptoPAN is prefix-preserving and deterministic, which allows data to be aggregated across files while retaining subnet-level structure. The challenge materials also note that matrix operations used in downstream analysis are invariant to row and column permutations, so reindexing the anonymized address space does not change derived quantities such as row sums, column sums, or spectra. At the same time, the challenge documentation explicitly notes that prefix preservation can leak structural information such as subnet membership, and treats legal agreements and operational controls as part of the privacy posture (Jananthan et al., 2024).

A further structural choice is the use of constant-packet, variable-time windows. Rather than fixing wall-clock intervals, the reference pipeline fixes $2^{30}$7 valid packets per matrix. The accompanying analysis literature argues that this reduces statistical fluctuations under heavy-tailed arrival processes and improves the stability of multi-scale aggregation and downstream detection (Jananthan et al., 2024, Kepner et al., 2023).

The sparse-matrix view yields direct GraphBLAS formulations for the challenge’s network quantities. Representative examples include

$2^{30}$8

for valid packets,

$2^{30}$9

for unique links,

$2 \times 2^{30}$0

for packets from each source, and

$2 \times 2^{30}$1

for source fan-out. Analogous formulas apply on the destination side through $2 \times 2^{30}$2 and $2 \times 2^{30}$3 (Jananthan et al., 2024).

3. Workflow, GraphBLAS formulation, and reference analysis

The reference implementation is organized around hypersparse matrices. This is motivated by the fact that traffic matrices derived from telescope data are extremely sparse: the logical address space is very large, but only a small fraction of source–destination pairs are observed in a short packet window. Hypersparse representations therefore store only non-empty vectors and the nonzeros within them, which supports “no down-sampling of packets,” high performance through sparse linear algebra, and portability through GraphBLAS (Voloshchuk et al., 2 Jan 2026).

At the algebraic level, the reference path uses element-wise addition for aggregation, reductions for scalar and vector summaries, and masks for subrange selection. Summation is carried out with the additive monoid

$2 \times 2^{30}$4

and subrange extraction is expressed as

$2 \times 2^{30}$5

where $2 \times 2^{30}$6 is a diagonal selector matrix (Voloshchuk et al., 2 Jan 2026). The challenge documentation also describes binary temporal aggregation, in which small packet-window matrices are recursively summed into coarser time scales. This reduces recomputation and exploits the fact that link overlap causes nonzero growth to remain below the naive worst case (Jananthan et al., 2024).

The step most heavily treated in follow-on work is the final “read, sum, analyze” phase. The refactored reference implementation described in 2026 isolates this as step (6): it reads stored matrices, sums them into one aggregate matrix $2 \times 2^{30}$7, and computes the challenge’s reported properties. The paper presents a simplified pseudocode loop that initializes $2 \times 2^{30}$8, reads batches of $2 \times 2^{30}$9 matrices, accumulates them with GraphBLAS element-wise addition, and then analyzes the final aggregate. The refactoring reduces the original Python code base from approximately 1000 lines across three files to approximately 325 lines across two modules while maintaining full functionality (Voloshchuk et al., 2 Jan 2026).

The challenge has also encouraged alternative interpretations of the same algebra. A 2025 GPU-oriented data-science implementation explicitly maps the GraphBLAS expressions onto dataframe operations: for example, $2^{30}$0 becomes a sum over packet counts; $2^{30}$1 becomes a count of distinct $2^{30}$2 pairs; and source fan-out becomes a grouped count over distinct destinations per source. This does not alter the mathematical specification, but it broadens the implementation space beyond conventional sparse-matrix APIs (Samsi et al., 3 Sep 2025).

4. Systems implementations and performance envelope

The challenge has rapidly become a systems benchmark as much as an analytic one. Follow-on papers cover CPU refactoring, SmartNIC packet extraction, dense-GPU analysis, commodity-GPU dataframe pipelines, and distributed collaborative deployments. The common thread is that each system targets a different stage of the same packet-to-matrix-to-analytics workflow.

Approach	Scope	Reported result
Refactored CPU reference implementation	Step (6): read, sum, analyze	67% code-size reduction; nearly linear scaling across cores and nodes (Voloshchuk et al., 2 Jan 2026)
FPGA SmartNIC with P4 and HLS	Packet parsing and source/destination extraction	Approximately 95 Gbps; able to keep up with 100 Gbps traffic (Han et al., 2024)
C++26 std::execution on GPUs	Analysis of anonymized matrices	Up to 55x analysis-time improvement on 8 A100 GPUs versus a serial GraphBLAS baseline (Mandulak et al., 15 Oct 2025)
RAPIDS cuDF/CuPy pipeline	Dataframe realization of challenge analytics	147x–509x on A100, 243x–1269x on H100, and 332x–2185x on H200 versus single-core Pandas (Samsi et al., 3 Sep 2025)
DBOS collaborative sensing	Web-service integrated local/global aggregation	$2^{30}$3 web requests per second per instance; linear scaling up to 64 nodes for Python-GraphBLAS and 32 nodes for OneSparse PostgreSQL (Lockton et al., 11 Sep 2025)

The FPGA line of work addresses the front of the pipeline. The SmartNIC implementation parses Ethernet, IPv4, IPv4 options, TCP, TCP options, and UDP on an AMD Xilinx Alveo U280, extracts source and destination IPv4 addresses, batches $2^{30}$4 address pairs into a custom information packet, and forwards those packets to the host via QDMA and DPDK. The reported throughput reaches approximately 95 Gbps and keeps pace with 100 Gbps ingress traffic, but the paper is explicit that on-FPGA anonymization and matrix construction are not yet implemented; those steps remain on the host side and are identified as future work (Han et al., 2024).

The CPU and distributed-array line of work concentrates on simplifying the reference code path and exposing its parallel structure. The refactored implementation uses pMatlab and pPython maps to distribute file indices across processes, yielding a communication-free inner loop because files and time windows are independent. The reported behavior is nearly linear scaling across cores and nodes, with summation dominating analysis because it has the larger memory footprint and more data movement (Voloshchuk et al., 2 Jan 2026).

GPU work divides into two styles. One style treats GPUs as first-class sparse-analysis devices: the C++26 Senders/Receivers implementation evaluates challenge properties such as total valid packets, unique links, unique sources, max source fan-out, unique destinations, and max destination fan-in over sparse matrices $2^{30}$5, and reports a best analysis time of 1.17 seconds on 8 A100 GPUs versus 64.23 seconds for the serial GraphBLAS baseline (Mandulak et al., 15 Oct 2025). The other style treats the challenge as an ETL-and-dataframe workload: RAPIDS cuDF and CuPy implement the same GraphBLAS formulas through groupby, value_counts, unique, and aggregation, and show that storage format is decisive—Parquet is much more GPU-friendly than PCAP or the fine-grained tar hierarchy (Samsi et al., 3 Sep 2025).

A further extension integrates the challenge into a service runtime. DBOS Network Sensing embeds GraphBLAS hypersparse traffic matrices into web services, aggregates local matrices into a global view, and uses either Python-GraphBLAS or OneSparse PostgreSQL to compute the challenge analytics. The reported results indicate negligible overhead on DBOS request processing and linear scaling to dozens of nodes, which makes the challenge relevant not only to batch analysis but also to collaborative situational awareness (Lockton et al., 11 Sep 2025).

5. Privacy model, residual risks, and de-anonymization concerns

The challenge’s privacy model is operational rather than absolute. It combines identifier transformation, data reduction to source–destination counts, and controlled access under sharing agreements. In the reference materials, anonymization is primarily attached to consistent CryptoPAN mapping of IPv4 endpoints; payloads are not required for matrix construction; and the benchmark notes that matrix-derived quantities are permutation-invariant. This creates a practical privacy-preserving data product, but it does not claim formal differential privacy or immunity to auxiliary-information attacks (Jananthan et al., 2024).

Research on graph de-anonymization is therefore directly relevant. In social-network settings, topology-only anonymization has been shown to be insufficient: one study reported that “a third of the users who can be verified to have accounts on both Twitter and Flickr can be re-identified in the anonymous Twitter graph with only a 12% error rate,” using a method based purely on network topology and requiring no large-scale sybil injection (0903.3276). A plausible implication is that anonymized network sensing graphs cannot be evaluated purely as identifier-suppressed objects; overlap with auxiliary graphs, operator logs, or cross-platform views may matter even when attributes are absent.

Work on sparse-graph privacy reinforces the same tension from a different direction. For sparse graphs, “any” differentially private mechanism that maintains a reasonable similarity with the initial dataset is described as having a very weak privacy guarantee, and the same paper proposes smooth-$2^{30}$6-anonymity as an alternative for sparse full-graph release (Epasto et al., 2022). This suggests a structural trade-off for network sensing: high-fidelity release of an entire sparse graph is difficult to reconcile with strong formal privacy, especially when downstream tasks depend on preserving detailed topology.

The challenge documentation itself acknowledges residual risk. Prefix preservation may reveal subnet structure; sophisticated adversaries with auxiliary information may attempt de-anonymization; and the mitigations listed are legal agreements, operational controls, and, where appropriate, stronger anonymization research beyond the reference path (Jananthan et al., 2024). In practice, this has pushed related work toward aggregation, selective masking, heavy-tail summary statistics, and collaborative analytics that share compressed matrices or derived quantities rather than raw packet streams (Kepner et al., 2023, Lockton et al., 11 Sep 2025).

6. Scientific uses, derived analytics, and continuing directions

Although the challenge is defined procedurally, it has also become a substrate for new analytic models. One line of work uses GraphBLAS hypersparse matrices for sensor focusing and calibration on a 246 GB gateway PCAP containing $2^{30}$7 packets. That work partitions the IPv4 space into subranges, fits Zipf–Mandelbrot heavy-tail models to link counts, and develops a detection model with explicit probability of detection $2^{30}$8 and probability of false alarm $2^{30}$9. It reports single-core, single-thread processing at approximately three times real time for a 6.8 Gbps gateway trace while achieving compression below one byte per packet for the stored anonymized matrices and below $N_V = 2^{17}$0 bits per packet for derived aggregates (Kepner et al., 2023).

Another line of work studies temporal source recurrence in challenge-scale datasets. Using ANSGC data from CAIDA and GreyNoise, the “cyber orbit” model posits an inverse-square relationship

$N_V = 2^{17}$1

with observed best-fit behavior

$N_V = 2^{17}$2

and hence

$N_V = 2^{17}$3

The paper reports a roughly 200-fold gap between benign and malicious orbit radii in Cartesian log plots, and frames this as an aid to understanding anonymized network behavior at scale (Kepner et al., 23 Aug 2025).

The challenge has also become a reference point for collaborative deployments. DBOS-style global aggregation, pPython-based file parallelism, and dense-GPU analysis all indicate that the canonical ANSGC workflow can be embedded in wider operational systems without abandoning its matrix formalism (Lockton et al., 11 Sep 2025, Voloshchuk et al., 2 Jan 2026, Mandulak et al., 15 Oct 2025). This broadens the challenge from a benchmark into a common intermediate representation for network sensing, one that can be realized as GraphBLAS kernels, dataframes, FPGA extraction pipelines, or database-integrated matrix services.

Several open directions are explicit in the literature. IPv6 handling is “encouraged but not mandated” in the challenge definition, which leaves address-space scale, anonymization, and sparse-storage choices as active implementation questions (Jananthan et al., 2024). Hardware papers identify host-side anonymization and on-device aggregation as unfinished stages (Han et al., 2024). Privacy papers suggest that stronger anonymization for sparse graph release remains unresolved (Epasto et al., 2022). System papers identify I/O, memory movement, and hierarchical aggregation as continuing bottlenecks even when analysis kernels themselves scale nearly linearly (Voloshchuk et al., 2 Jan 2026, Samsi et al., 3 Sep 2025, Mandulak et al., 15 Oct 2025).

Taken together, these developments place the Anonymized Network Sensing Graph Challenge at the intersection of graph theory, network measurement, sparse linear algebra, privacy engineering, and high-performance systems. Its central abstraction—the anonymized traffic matrix—has proven flexible enough to support packet extraction hardware, GraphBLAS reference pipelines, GPU and database implementations, collaborative awareness platforms, and new statistical models of Internet-scale behavior, while keeping the benchmark anchored to concrete workflow stages and measurable outputs (Jananthan et al., 2024).