TeraNoC: Hybrid Mesh–Crossbar NoC

• TeraNoC is a hybrid mesh–crossbar network-on-chip architecture designed for many-core clusters, enabling low-latency, word-granular access to a globally shared scratchpad memory.
• It employs a hierarchical design combining single-cycle local crossbars, intra-Group crossbars, and a 4×4 inter-Group mesh to balance minimal latency with scalable wiring and bandwidth.
• Innovative router remappers and parallel 32-bit channels optimize traffic distribution and improve performance, validated through both simulation and implementation studies.

Searching arXiv for the most relevant TeraNoC papers and related context. I’ll retrieve the provided arXiv records to ground the article in the latest papers. TeraNoC denotes, in current many-core architecture literature, an open-source hybrid mesh–crossbar network-on-chip designed to scale a single tightly coupled shared-L1-memory cluster beyond 1000 cores while preserving low-latency, word-granular access to a globally shared scratchpad memory. In the 1024-core testbed, it combines single-cycle local crossbars, 16×16 intra-Group crossbars, and a fine-grained multi-channel 4×4 2D mesh across Groups, with separate request and response networks and a router remapper for load balancing (Zhang et al., 4 Aug 2025). The same name appeared earlier in a distinct context for a graphene-enabled terahertz wireless network-on-chip, where on-chip cores communicate via THz graphene nanoantennas under a channel model that combines dielectric propagation loss and molecular absorption attenuation (Vien et al., 2017). A later study uses the many-core TeraNoC testbed to validate a fast end-to-end event-driven simulator, GVSoC, showing end-to-end error below 7% against a cycle-accurate RTL golden model and up to 115× faster simulation (Li et al., 8 May 2026).

1. Design problem and architectural position

TeraNoC is motivated by a specific scale-up problem in tightly coupled many-core clusters: pure crossbars preserve very low latency but incur routing complexity and physical routing resources that grow quadratically with the number of endpoints, whereas pure 2D meshes scale wiring more regularly but impose hop-dependent latency that is too high for latency-sensitive core-to-L1-memory access. In the 1024-core regime discussed for TeraNoC, a state-of-the-art hierarchical crossbar-only design, TeraPool, allocates up to 40.7% of die area to routing channels to reach 1024 cores and 4096 banks, while a flat 16×16 tile mesh reaches zero-load round-trip latency of 127 cycles at maximum distance and 45.7 cycles on average for small-word requests (Zhang et al., 4 Aug 2025).

The design response is a hierarchy that places fully combinational logarithmic crossbars where low latency is most critical and a multi-channel narrow-link 2D mesh where wiring regularity and bandwidth scaling are most important. The key traffic assumption is that L1 is accessed at word granularity, specifically 32-bit, so widening mesh links would complicate network interfaces and add wires that are wasted for fine-grained L1 traffic; TeraNoC instead scales bandwidth through many parallel 32-bit channels (Zhang et al., 4 Aug 2025).

A common misconception is to read TeraNoC as a conventional shared-memory cache hierarchy. The many-core TeraNoC is instead a scratchpad-centric system: the L1 memory is software-managed and globally shared, there is no cache coherence, and programmers explicitly manage data movement and synchronization (Li et al., 8 May 2026). This suggests that its architectural novelty lies not only in topology selection but also in matching interconnect structure to explicit shared-SPM execution.

2. Cluster organization, processing elements, and memory semantics

In the 1024-core testbed, the cluster is partitioned hierarchically into 16 Groups connected by a 4×4 2D mesh; each Group contains 16 Tiles; and each Tile contains 4 processing elements and 16 L1 SPM banks of 1 KiB each. The resulting configuration comprises 1024 Snitch RV32IA cores, 4096 banks, and a globally shared 4 MiB L1 scratchpad memory (Zhang et al., 4 Aug 2025).

Each PE is a single-issue, single-stage 32-bit RV32IA Snitch core. In the simulation-oriented description, the PE is extended with an IPU for int32/16b, an FPU for fp32/16b, and a load-store unit allowing multiple outstanding transactions to tolerate access latency (Li et al., 8 May 2026). In the implementation-oriented description, the Snitch LSU uses an 8-entry outstanding transaction table for tolerance of NUMA latencies (Zhang et al., 4 Aug 2025). The memory system is a shared unified address space implemented as many small SRAM banks with word-level address interleaving across banks.

The memory model is explicitly scratchpad-centric. Loads and stores are memory-mapped operations, and correctness depends on explicit data placement, tiling, and synchronization rather than on a coherence directory. Synchronization uses software barriers and atomics, and the latency term for a memory operation can be decomposed as

$$L_{\text{total}} = L_{\text{SPM}} + L_{\text{NoC}} + L_{\text{sync}},$$

where $L_{\text{SPM}}$ includes bank access time and bank-conflict-induced waiting, $L_{\text{NoC}}$ captures end-to-end interconnect delay for remote accesses, and $L_{\text{sync}}$ covers software barriers, atomics, and progress skew across PEs (Li et al., 8 May 2026).

This organization makes contention multi-level rather than monolithic. Bank conflicts arise first at the banked SPM, then at the intra-Tile crossbar among 4 PEs, then at the intra-Group crossbar among 16 Tiles, and finally at inter-Group mesh links. A plausible implication is that locality and address mapping are architectural concerns rather than merely compiler conveniences.

3. Interconnect hierarchy, routing regime, and quantitative characteristics

TeraNoC uses a hybrid mesh–crossbar interconnect with three regimes: an intra-Tile crossbar from PEs to local banks, an intra-Group multi-level crossbar for traffic among Tiles inside a Group, and an inter-Group 2D mesh connecting the 16 Groups. Requests and responses are separated physically and logically, and all crossbars and mesh routers operate at 32-bit granularity, matching the word-width access pattern of the shared L1 (Zhang et al., 4 Aug 2025).

At the Tile level, a fully combinational logarithmic crossbar connects 4 cores to 16 banks with single-cycle latency, round-robin arbitration inside multiplexers and demultiplexers, and valid–ready handshake. At the Group level, 16 Tiles are connected by a 16×16 logarithmic crossbar, with spill registers at tile boundaries to break long paths for timing. For inter-Group traffic, 16 Groups form a 4×4 mesh. In the testbed, each Tile has $K=2$ request ports and $K=2$ response ports to $K$ routers for inter-Group traffic, and the baseline mapping uses two mesh routers per Tile at the Group level (Zhang et al., 4 Aug 2025). In the simulation model, the cluster-level instantiation is described as 32 parallel request meshes plus 32 parallel response meshes, with channels that are 32 bits wide, or 4 bytes per link per cycle (Li et al., 8 May 2026).

The mesh router microarchitecture is specified as a 5×5-port router with dimension-ordered XY routing, input and output FIFOs, valid–ready backpressure, and two-depth FIFO per routing direction. The per-hop router latency is $L_{\text{hop}} = 2$ cycles, and no virtual channels are used; instead, physical separation of request and response channels avoids message-level deadlocks (Zhang et al., 4 Aug 2025). In the simulator, virtual channels and some pipeline flip-flops are abstracted and folded into calibrated base latency and queueing behavior rather than modeled explicitly (Li et al., 8 May 2026).

The latency regime is one of the defining quantitative features of TeraNoC. In the implementation description, intra-Tile accesses complete in 1 cycle round-trip, intra-Group accesses in 3 cycles round-trip, inter-Group neighbor accesses in 7 cycles, the farthest 7-hop access in 31 cycles, and the average inter-Group latency across the 4×4 mesh in 13.7 cycles, inclusive of mesh and crossbars (Zhang et al., 4 Aug 2025). The simulator presents the same inter-Group figures, while describing local and intra-Group accesses as “on the order of a few cycles” (Li et al., 8 May 2026).

Bandwidth is scaled through channel multiplicity rather than width. The generic relation is

$B = W \cdot f \cdot C,$

with word width $W$, frequency $L_{\text{SPM}}$0, and channel count $L_{\text{SPM}}$1 (Zhang et al., 4 Aug 2025). For the 1024-core implementation, the mesh link bandwidth uses $L_{\text{SPM}}$2 B, $L_{\text{SPM}}$3 MHz, and $L_{\text{SPM}}$4 parallel response channels per direction per link, giving approximately $L_{\text{SPM}}$5 GB/s per direction. The peak L1 bandwidth is 4 KiB/cycle, equivalent to 3.74 TiB/s at 936 MHz, and the mesh bisection bandwidth is 0.5 KiB/cycle, approximately 0.47 TiB/s (Zhang et al., 4 Aug 2025).

Aspect	Value	Notes
Cluster hierarchy	16 Groups × 16 Tiles/Group × 4 cores/Tile	1024 cores total
Shared L1 SPM	4096 banks × 1 KiB	4 MiB globally shared
Local latency	1 cycle round-trip	Tile crossbar
Intra-Group latency	3 cycles round-trip	16×16 crossbar with spill regs
Inter-Group latency	7 cycles neighbor; 31 cycles farthest; 13.7 cycles average	4×4 mesh
Mesh data path	32-bit channels	4 bytes/cycle per channel
Operating point	936 MHz, TT/0.80 V/25°C	GF12nm LP+ FinFET

This hierarchy is neither a local-crossbar cluster with an attached mesh nor a mesh with local shortcuts. Rather, the crossbars and the mesh define separate contention regimes with different timing sensitivities, which is precisely the property the later GVSoC modeling work treats as essential.

4. Router remapping and traffic balancing

A central TeraNoC mechanism is the router remapper, introduced to address the inefficiency of fixed mapping from tile ports to mesh routers. With a fixed mapping, one router can become congested while others remain underutilized when traffic is directional or imbalanced. TeraNoC addresses this by decomposing a global remapper into multiple small crossbar-based remappers. In the 1024-core testbed, each remapper connects one of the $L_{\text{SPM}}$6 ports from each of $L_{\text{SPM}}$7 tiles and produces a $L_{\text{SPM}}$8-to-$L_{\text{SPM}}$9 remap to the routers; the reported testbed uses $L_{\text{NoC}}$0 tiles per remapper and $L_{\text{NoC}}$1 remappers per tile set. A shift register seeded with a programmable value generates pseudorandom permutations, and a stride-based offset on tile IDs increases spatial diversity. The mapping changes which sending port goes to which router without changing packet headers (Zhang et al., 4 Aug 2025).

The direct quantitative effect is reported for f32 GEMM inner-loop traces. Average congestion, measured as ChannelStalls/Cycle, falls 80% from 0.40 to 0.08, peak congestion falls 63% from 0.83 to 0.31, and observed global L1 access bandwidth improves 2.7× from 405.3 GiB/s to 1081.4 GiB/s (Zhang et al., 4 Aug 2025). In the later simulation-based design-space exploration, a conceptually related remapper is inserted between Tiles and parallel mesh routers and updated every cycle via a deterministic pseudo-random schedule. A full 32-to-32 remapper is treated as impractical, so the design is decomposed into smaller $L_{\text{NoC}}$2-port remappers with $L_{\text{NoC}}$3; the eight-port remapper performs best, the four-port variant is close, and traffic-aware partitioning outperforms uniform partitioning (Li et al., 8 May 2026).

The workload-level outcome in the design-space exploration is an overall approximately 10% improvement on representative workloads, with kernel-wise gains reported as BN 11.5%, Cholesky 12.2%, LN 47.1%, GEMM 12.3%, MMSE 9.5%, and Softmax 33.8% (Li et al., 8 May 2026). The simulation paper further notes that spatial imbalance across parallel meshes and temporal imbalance as hotspots shift over time both contribute to PE stalls even when average link utilization is modest.

A useful derived interpretation appears in the simulation study: balance can be summarized by Jain’s fairness index,

$L_{\text{NoC}}$4

where $L_{\text{NoC}}$5 is per-link utilization and $L_{\text{NoC}}$6 is the number of links across the bisection; remapping raises $L_{\text{NoC}}$7 toward 1 by smoothing both spatial and temporal skew (Li et al., 8 May 2026). This suggests that TeraNoC’s remapper is not a mere throughput enhancement device but a mechanism for restoring effective parallelism in a multi-channel narrow-link mesh.

5. Implementation results, efficiency, and workload behavior

The published implementation uses GlobalFoundries 12 nm LP+ FinFET and operates at TT/0.80 V/25°C with a cluster frequency of 936 MHz. The core-to-L1 interconnect occupies 10.9% of the logic area, reduces total cluster die area by 37.8% relative to a hierarchical crossbar-only cluster, and increases frequency by 13.3% versus a baseline crossbar-only cluster operating at 850 MHz. The interconnect is reported to be off the timing critical path at this operating point (Zhang et al., 4 Aug 2025).

Performance is characterized on GenAI and data-parallel kernels. For local-access-dominated kernels such as axpy, dotp, and gemv, IPC reaches up to 0.85; dotp and gemv incur small extra WFI cycles for reduction synchronization. For conv2d, which has mixed local and global traffic, IPC is 0.82 and only 1% LSU stalls are reported due to localized weights and short-distance fetches. For global-access-dominated matmul, IPC is approximately 0.70 for both fp32 and fp16, and the loss is attributed to FU RAW stalls waiting for long-distance data rather than to interconnect conflicts; the paper attributes the residual latency-hiding limit to 32 architectural registers (Zhang et al., 4 Aug 2025).

Energy and area efficiency are reported in workload-dependent form. The interconnect consumes 7.6% of total cluster power in kernels dominated by crossbar accesses and 22.7% in kernels with high 2D-mesh traffic. Throughput reaches up to 0.77 TFLOP/s in fp32 and 1.3 TFLOP/s in fp16. Area efficiency improves by up to 98.7% in GFLOP/s/mm² relative to the hierarchical crossbar-only baseline (Zhang et al., 4 Aug 2025).

These measurements clarify a common misunderstanding about narrow links. The 32-bit link width is not presented as a minimalist compromise; it is a deliberate match to word-granular L1 traffic, and the bandwidth target is recovered by channel multiplicity, router placement near tiles, and bundling of directional channels along straight shortest paths. This suggests that TeraNoC’s efficiency gains derive from alignment between transaction granularity and physical interconnect structure rather than from a single isolated optimization.

6. End-to-end modeling, profiling, and co-optimization

The 2026 simulation study uses TeraNoC as a latency-sensitive, scratchpad-centric many-core testbed for validating GVSoC, an event-driven simulator with clock engines and timestamped event queues. Components interact via request calls with explicit target timestamps, preserving causality while abstracting non-essential microarchitectural details. The model precisely captures latency-sensitive SPM paths, including word-level accesses, bank conflicts, and arbitration, while folding spill registers into adjacent routers and representing virtual channels and pipeline flip-flops through calibrated base latency and queueing behavior rather than explicit micro-state simulation (Li et al., 8 May 2026).

The simulator is selective about asynchronous and synchronous modeling. Intra-Group request paths are asynchronous to capture fine-grained contention and arbitration; intra-Group response paths are synchronous with lightweight congestion handling because lower contention was observed; and both request and response paths of the inter-Group mesh are asynchronous because multi-hop contention is significant in both directions. End-to-end NoC latency is expressed as

$L_{\text{NoC}}$8

with

$L_{\text{NoC}}$9

and a back-of-the-envelope queueing approximation

$L_{\text{sync}}$0

under a simple M/M/1 assumption (Li et al., 8 May 2026).

Validation is reported across linear algebra, signal processing, and ML kernels. The model tracks a cycle-accurate RTL golden model with relative error consistently below 7% end-to-end and delivers up to 115× speedup versus RTL, with many workloads exceeding two orders of magnitude faster. Residual error is attributed primarily to folding simple hardware elements into router latency, synchronous modeling of some lower-contention response paths, and abstracted microarchitectural details such as the absence of explicit virtual-channel modeling (Li et al., 8 May 2026).

A major reason the simulator is useful beyond validation is its instrumentation. The profiling API provides PE-level instruction traces, per-PE state, instruction/load/store counters, and stall-cycle breakdown, together with interconnect-level transfer event traces, link and port busy cycles, transfer counts, queue occupancy, port conflicts, blocking, and memory-level access logs and bank-conflict statistics (Li et al., 8 May 2026). These outputs are used directly in two case studies.

In FlashAttention-2, profiling shows low PE utilization, substantial interconnect stalls, and high synchronization overhead caused by progress divergence, traced to concentrated conflicts at remote request input ports of a subset of Tiles, especially at the start of inner tiling loops. The reported optimization has two parts: reorganize workload distribution so that query accesses in score computation and all accesses in online softmax stay within the local Group, and introduce staggered start offsets for PEs to desynchronize bursty accesses. The result is a significant drop in interconnect-induced stalls and synchronization overhead (Li et al., 8 May 2026).

In the NoC remapping case study, profiling indicates that PE stalls due to mesh congestion can persist despite modest average link utilization because traffic is inefficiently distributed across the parallel meshes. The simulation framework is then used for design-space exploration of remappers with different port counts and partitioning strategies, leading to the kernel-dependent gains summarized earlier (Li et al., 8 May 2026). A plausible implication is that TeraNoC is well suited to iterative software/hardware co-design precisely because local stalls, bank conflicts, and mesh imbalance are all observable within one timing-consistent model.

7. Earlier terahertz wireless interpretation

Before its current association with the shared-L1 many-core interconnect, the name TeraNoC was used for a terahertz wireless network-on-chip fabric enabled by graphene-based nanoantennas. In that context, each core integrates a graphene-based plasmonic nanoantenna that transmits and receives in the THz band, and the principal research problem is not hierarchical wired interconnect scaling but environment-aware intra-package wireless propagation (Vien et al., 2017).

The core channel model combines two mechanisms. The first is two-ray dielectric propagation loss, expressed as

$L_{\text{sync}}$1

The second is molecular absorption attenuation, modeled through Beer–Lambert’s law,

$L_{\text{sync}}$2

with total path loss

$L_{\text{sync}}$3

The absorption coefficient $L_{\text{sync}}$4 is the sum of isotopologue-specific terms derived from HITRAN 2012 spectroscopic parameters and a Van Vleck–Weisskopf asymmetric line shape (Vien et al., 2017).

This earlier TeraNoC reports strong frequency selectivity from molecular absorption attenuation, with pronounced loss peaks around 1.21 THz, 1.28 THz, and 1.45 THz in the studied environment. The paper states that, across the 1–2 THz range, capacity is at least $L_{\text{sync}}$5 bits/s lower than in a conventional model without molecular absorption attenuation. At 1 THz, capacity degradation is reported as up to 26.8% across the full temperature range and up to 25% across the pressure range; at a distance of 0.01 mm, capacity can be up to 31.8% lower than in the conventional model. For links in the 10–100 μm range at 1 THz, water-filling improves capacity by up to 200 Gb/s over flat allocation in the proposed model (Vien et al., 2017).

The wireless TeraNoC and the later hybrid wired TeraNoC therefore address different bottlenecks. The former studies THz nanocommunications inside a chip package using graphene-enabled wireless links and environment-sensitive channel modeling; the latter studies a wired, open-source, shared-L1 many-core interconnect based on narrow multi-channel meshes and crossbars. The shared name should not obscure the fact that they belong to distinct research lines with different physical assumptions, analytical models, and optimization targets.