Multi-Granularity-Aware Router

• Multi-Granularity-Aware Router is a system that dynamically selects and integrates information across different scales (e.g., session, paragraph, or wavelength) to enhance retrieval and resource use.
• It employs adaptive routing algorithms—such as softmax/entropy-based and transformer-guided methods—to balance fine-grained precision with broader contextual coverage.
• Empirical results demonstrate significant performance gains in applications like memory-augmented LLMs, retrieval-augmented generation, traffic analysis, and optical networking.

A Multi-Granularity-Aware Router is a computational or networking module that dynamically selects and integrates information, signals, or resources across multiple granularity levels—such as temporal, textual, optical, or traffic aggregation scales—to optimize accuracy, relevance, cost, and/or computational efficiency according to context-dependent objectives. The architecture, algorithms, and applications of multi-granularity-aware routing span domains including conversational LLM memory, retrieval-augmented generation, encrypted traffic analysis, and optical networking. Core to all approaches is the explicit architectural or algorithmic exposure of granularity options, a routing or selection mechanism that dynamically weights or fuses them, and (often) hierarchical or graph-based modeling that supports both coarse- and fine-grained aggregation or selection.

1. Core Concepts and Motivations

Multi-granularity-aware routers generalize the routing or retrieval task to operate not at a single fixed level of abstraction, but over a spectrum or hierarchy of granularities. In memory-augmented LLMs, these granularities correspond to dialogue session, turn, summary, and keywords (Xu et al., 26 May 2025). In retrieval-augmented generation (RAG), they are defined by snippet size (sentence, paragraph, page) or graph hop-radius (Zhong et al., 2024); for encrypted traffic analysis, by temporal patches of differing width (Pan et al., 22 Mar 2026); and in next-generation optical routers, by wavelength, waveband, or subcarrier/spectrum-slice (Wu, 2019).

The fundamental motivation is that information relevance, cost, or resource utilization is inherently scale-dependent. For example, fine-grained retrieval offers precision but risks noise and computational cost, while coarse retrieval yields context coverage but may dilute relevance. Multi-granularity routers aim to automate the granularity decision, leveraging query- or context-adaptive mechanisms to optimize the routing objective, and to support cross-scale fusion as required by the application.

2. Architectural Patterns

Multi-Granularity Memory and Data Structures

A unifying architectural principle is explicit construction or representation of multi-granularity units:

• Textual memory: Each memory chunk is represented as a tuple containing session, turn, generated summary, and set of keywords (Xu et al., 26 May 2025). Embeddings are computed for each level.
• RAG knowledge sources: Chunks are produced at multiple granularities by segmenting documents using sliding windows or graph neighborhoods (Zhong et al., 2024).
• Traffic feature maps: Time-series representations are embedded through parallel convolutional paths at distinct kernel widths, yielding patch token sequences at different temporal resolutions (Pan et al., 22 Mar 2026).
• Optical signal paths: Routing control structures span wavelength, waveband, and spectrum slice levels, with switching fabrics supporting variable aggregate units (Wu, 2019).

Table 1 summarizes the principal domains and their multi-granularity representations:

Domain	Granularity Levels	Representation/Unit
LLM Memory (Xu et al., 26 May 2025)	Session, Turn, Summary, Keyword	Memory tuple, per-granularity embedding
RAG (Zhong et al., 2024)	Sentence, Paragraph, Page, Graph-hop	Text chunk, graph-based neighborhoods
Traffic Analysis (Pan et al., 22 Mar 2026)	Patch (via CNN kernels of size k_i)	Patch tokens per temporal scale
Optical Routing (Wu, 2019)	λ (wavelength), B (waveband), δ (subcarrier)	Fiber traffic units, switch fabrics

3. Routing Mechanisms and Algorithms

Neural and Information-Theoretic Routers

• Softmax/MLP-based routers: A lightweight MLP maps encoded queries to “granularity importance scores” (weights). The router distributes retrieval weights over granularities, and the weighted sum of per-granularity relevance scores governs chunk selection (Zhong et al., 2024).
• Entropy-based routers: Shannon entropy of similarity-weighted softmax per granularity is inverted to yield adaptive routing weights. Low entropy (high confidence) granularities are attributed higher routing weights in retrieval, balancing precision (low granularity) and coverage (high granularity) (Xu et al., 26 May 2025).
• Gradient-free soft-label training: Training uses soft labels derived from semantic similarity between candidate snippets at each granularity and the reference ground truth, assigned as a soft binary distribution (e.g., 0.8/0.2/0), and learned via cross-entropy loss without backpropagation through retrieval (Zhong et al., 2024).

Hierarchical Attention and Router Tokens

• Transformer-based routers: Models such as PrismWF implement router tokens at each granularity, aggregated and fused via local and global attention. These router tokens serve as semantic anchors, enabling global reconnaissance at coarse scales and local querying at fine scales (Pan et al., 22 Mar 2026).
• Graph-based extension: In graph-augmented settings, router weights are assigned over subgraphs defined by hop distance, supporting retrieval of non-contiguous but semantically related information (Zhong et al., 2024).

Routing in Optical Networks

Multi-granularity-aware optical routers employ integer linear programming and custom heuristics for granularity selection (λ, WB, δ), with cost and bandwidth as primary drivers. The architecture dynamically chooses switching granularity per lightpath request with constraints on spectrum contiguity, waveband non-overlap, and cost minimization (Wu, 2019).

4. Integration into Broader Pipelines

Memory-Augmented LLMs

• Memory association and retrieval: Gaussian mixture models (GMMs) associate new memory chunks to historical tuples via learned relevance, augmenting an evolving association graph. For each query, the entropy-based router allocates retrieval weights to session, turn, summary, and keyword memory, and final context is filtered by an LLM-based redundancy filter (Xu et al., 26 May 2025).

Retrieval-Augmented Generation

• Document chunking and retrieval: Documents are preprocessed at multiple granularities; for a query, the router assigns weights, retrieves top-ranked candidates per level, computes a weighted sum, and injects the maximally weighted granularity as context to the LLM for answer generation (Zhong et al., 2024).
• Graph extension: Retrieval spans hop-radius neighborhoods in a sentence graph, with router assigning importance to each hop-based chunk set.

Website Fingerprinting and Traffic Analysis

• Temporal patch fusion: Convolutional feature extractors feed multi-resolution temporal data to router tokens, which fuse local (patch) and global (router-token) information via intra- and inter-granular attention, refining feature representations and supporting robust classification (Pan et al., 22 Mar 2026).

Optical Networks

• Resource allocation and routing: ILP models and heuristic schedulers select paths and assign resources at the appropriate granularity, constrained by capacity, switching, and spectrum assignment rules, enabling dramatic reductions in blocking probability, hardware cost, and spectrum utilization (Wu, 2019).

5. Mathematical Formulation

Representative mathematical models and routing algorithms are summarized below:

Memory Routing (LLM, Retrieval)

• Relevance probabilities: $$P^g_i = \frac{\exp(\mathrm{sim}(q, M_i^g)/\lambda)}{\sum_j \exp(\mathrm{sim}(q, M_j^g)/\lambda)}$$
• Entropy per granularity: $H^g = -\sum_i P^g_i \log P^g_i$
• Routing weights: $w^g = \frac{1/H^g}{\sum_{g'}1/H^{g'}}$
• Final score: $$\text{score}_i = \sum_g w^g \cdot \mathrm{sim}(q, M_i^g)$$ (Xu et al., 26 May 2025)

RAG Routing

• MLP router outputs: $$w = (w_1, ..., w_{n_{\mathrm{gra}}}) = \mathrm{MLP}(e_q)$$
• Weighted chunk score: $S(c) = \sum_g w_g t_{rs}(c, g)$; final top-$k$ selection on $S(c)$ (Zhong et al., 2024)

Optical Routing (Selected ILP Constraints)

• Path-granularity selection: $\sum_{g,p}x^g_{r,p}=1,\ \forall r$
• Link capacity: $$\sum_{r,p:\ell\in p}d^g_r x^g_{r,p} \leq \mathrm{Cap}^g_\ell$$
• Cost minimization: $H^g = -\sum_i P^g_i \log P^g_i$0 (Wu, 2019)

6. Empirical Results and Practical Impact

Consistent quantitative improvements are reported across domains:

• RAG and LLM Memory: Query-adaptive routers yield absolute accuracy gains of 1.5–5 points (e.g., Llama3: 59.0% → 66.5%) over fixed-granularity baselines, and achieve similar or better performance with lower retrieval set sizes (k_r = 3–8 vs. 16+) (Zhong et al., 2024, Xu et al., 26 May 2025).
• Graph-extended routers: Adding graph-based chunking enables further 1–2 point gains, efficiently retrieving multi-hop context (Zhong et al., 2024).
• Optical routers: Multi-granular routers reduce spectrum blocking by up to 4×, halve WSS port counts, and cut power and CapEx by 70–75% without performance reduction (Wu, 2019).
• Traffic analysis: Router tokens and multi-granular attention blocks achieve state-of-the-art classification of mixed and single-tab web traffic, demonstrating powerful fusion of local and global patterns across temporal scales (Pan et al., 22 Mar 2026).

7. Design Guidelines and Limitations

Designing a multi-granularity-aware router requires careful selection of granularity levels matched to data distribution, task, and hardware constraints. Empirical studies indicate:

• Intermediate granularities (e.g., waveband size B ≈ 4–8) often balance hardware cost and blocking/flexibility in optical networks (Wu, 2019).
• Fine-to-coarse and coarse-to-fine cross-attention, as well as separate routing for different memory types, maximize recall without redundancy or noise (Xu et al., 26 May 2025, Pan et al., 22 Mar 2026).
• Soft-label training enables practical MLP-based routers without retrieval backpropagation and with interpretable, query-dependent granularity selection (Zhong et al., 2024).
• Limiting the number of router candidates, using informative association graphs, and filtering with LLM-based redundancy prompts further improve efficiency and final retrieval quality.

A plausible implication is that excessive granularity levels or inadequate graph/attention modeling can lead to diminishing returns or computational expense, motivating ablation and parameter studies for optimal deployment.

---

Cited Papers:

• (Xu et al., 26 May 2025) Towards Multi-Granularity Memory Association and Selection for Long-Term Conversational Agents
• (Zhong et al., 2024) Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation
• (Pan et al., 22 Mar 2026) PrismWF: A Multi-Granularity Patch-Based Transformer for Robust Website Fingerprinting Attack
• (Wu, 2019) Resource Allocation in Multigranular Optical Networks