LLM-CAT Protocol: Cascade and Crypto Methods

• LLM-CAT Protocol is an advanced framework that employs cascaded LLMs to optimize inference efficiency by dynamically routing between small and large models.
• It introduces cascade-aware training objectives like CAT-Xent and CAT-Dist with rigorous threshold calibration to enhance both output quality and computational cost tradeoffs.
• The protocol extends to multi-agent communication and cryptographic steganography, enabling secure, context-rich interactions in telecom and covert messaging applications.

The LLM-CAT protocol encompasses a set of advanced methodologies leveraging LLMs in cascaded architectures, consistency-accuracy tradeoff analysis, context-rich multi-agent communication, and cryptographically secure steganography. LLM-CAT frameworks enable optimization of resource-aware model serving, nuanced benchmarking of consistency and robustness, contextualized multi-agent orchestration in domains like telecom, and covert cryptographic communications, each with rigorously defined algorithms, loss functions, metrics, and schema architectures. The protocol family is prominent across recent work in LM deployment (Wang et al., 2024), LLM evaluation (Cavalin et al., 26 Nov 2025), multi-agent model context exchange (Shah et al., 12 Nov 2025), and secure messaging (Gligoroski et al., 11 Apr 2025).

1. Cascade-Aware Training and Inference Architecture

LLM-CAT originated in the context of cascade-aware training for LMs, where serving cost and latency are mitigated through a conditional routing architecture between small and large models. In the canonical 2-model cascade (Wang et al., 2024):

• $p_S$: Small LM (e.g., PaLM-2 Gecko, $\sim$1.5B parameters)
• $p_L$: Large LM (e.g., PaLM-2 Otter, $\sim$8B parameters)
• For each input $x$, $p_S$ generates $\hat{y}_S \sim p_S(\cdot|x)$, and the token-level negative log-likelihood confidence score is

$$r(x) = -\frac{1}{N} \sum_{i=1}^N \log p_S(y_{S,i} \mid x, y_{S,<i})$$

If $r(x) < \tau$ (tunable threshold), output $\hat{y}_S$; otherwise defer to $p_L$ for output $\hat{y}_L$.

Cascade routing thus defines the overall distribution:

$$p_{cas}(y|x) = \mathbb{1}[r(x)<\tau] \cdot p_S(y|x) + \mathbb{1}[r(x)\geq\tau] \cdot p_L(y|x)$$

Expected query cost is:

$C_{cas}(x) = C_S + \mathbb{1}[r(x) \geq \tau] C_L$

where $C_S$, $C_L$ are per-token FLOP costs. This design enables dynamic tradeoff between inference efficiency and output quality.

2. Cascade-Aware Training Objectives and Algorithms

Optimizing the cascade requires training the small model $p_S$ with loss functions incorporating downstream model competence (Wang et al., 2024). For a dataset $\{(x, y)\}$, length $N$, and output position $i$:

$$\alpha_i(x, y_{<i}) = \mathbb{1}[y_i = \textrm{argmax}_{y'} p_S(y'|x, y_{<i}) \ {\rm OR}\ y_i = \textrm{argmax}_{y'} p_L(y'|x, y_{<i})]$$

Two objective variants are supported:

• Cascade-aware cross-entropy (CAT-Xent, $w=1$):

$$L_{\text{cat-xent}}(x, y) = -\sum_{i=1}^N \alpha_i \log p_S(y_i|x, y_{<i})$$

• Cascade-aware distillation (CAT-Dist, $0 \leq w \leq 1$):

$$L_{\text{cat-dist}}(x, y) = -\sum_{i=1}^N \alpha_i \Big[ w \log p_S(y_i|x, y_{<i}) + (1-w) \sum_{y'=1}^V p_L(y'|x, y_{<i}) \log p_S(y'|x, y_{<i}) \Big]$$

$\alpha_i$ masks out "hard" tokens, focusing the loss on those predictable by at least one model. The small model is fine-tuned by freezing $p_L$, iteratively computing, masking, and aggregating per-token losses over batches, updating $\theta$ via gradient descent.

3. Threshold Calibration, Quality-Cost Tradeoff, and Empirical Results

Optimal cascade thresholding is achieved via grid sweep of $\tau$ values across validation sets (Wang et al., 2024). For each $\tau$:

• Deferral rate: fraction with $r(x) \geq \tau$
• Quality metric: $M_{cas}(\tau)$, e.g., accuracy or BLEU
• Computational cost: $C_S + \text{deferral rate} \times C_L$

Empirical highlights:

Dataset	Model Segments	Baseline vs. CAT	Cascade Benefit
SuperGLUE	Classification	CAT-Xent saves 13% FLOPs @ 86% acc	$\uparrow$ +2pp accuracy @ 2B FLOPs
WMT22	Generation (BLEU)	CAT-Xent: +1–2 BLEU on $p_S$	Higher BLEU at fixed cost
FLAN2021G	Mixed tasks, zero-shot	CAT-Xent cascades outperform baseline	Enhanced low-deferral performance

CAT-Xent and CAT-Dist consistently outperform standard cross-entropy and vanilla distillation for small models and overall cascade robustness. CAT notably raises segment $A_1(\tau)$ (non-deferred small model accuracy); $A_2(\tau)$ (large model) remains unaltered.

4. Consistency-Accuracy Tradeoff: CAT as Benchmarking Protocol

The CAT protocol also embodies a framework for evaluating LLM consistency versus accuracy under controlled input variations (Cavalin et al., 26 Nov 2025):

• CAR Curve: For multiple-choice (MC) tasks, generate $M$ variants per example, compute per-example response consistency

$$RC_i = \frac{1}{M} \sum_{j=1}^M \mathbb{1}[a_i^{(j)} = o_i^*]$$

Across threshold grid $C = \{c_1, c_2, \dots, c_K\}$, define

$$MCA(c) = \frac{1}{N} \sum_{i=1}^N \mathbb{1}[RC_i \geq c]$$

The CAR curve $\{(c_k, MCA(c_k))\}$ visualizes how accuracy degrades as minimum consistency $c$ tightens.

• CORE Index: Quantifies overall tradeoff:

$CORE = AUCAR \times \textrm{normDTW}$

where $AUCAR$ is the area under the CAR curve, and normDTW is a shape similarity score using dynamic time warping.

Procedurally, CAT encompasses prompt variation, inference, parsing, consistency scoring, CAR curve construction, and CORE calculation, and is directly extensible to open-ended generation via continuous similarity functions.

5. Model Context Protocol and Multi-Agent Communication (TeleMCP)

LLM-CAT also serves as the foundation for context-rich multi-agent orchestration in applications such as telecom networks (Shah et al., 12 Nov 2025). TeleMCP extends the generic Model Context Protocol, enabling typed exchange of domain-specific context objects (KPI vectors, PCAP message records, state representations):

• Context vector: $\mathbf{c} = [c_1, \dots, c_n]^T \in \mathbb{R}^n$
• State representation: $\mathbf{s} = [s_1, \dots, s_m]^T \in \mathbb{R}^m$
• TeleMCP message schema:

$$m = \bigl(\mathrm{id}, \mathrm{sender}, \mathrm{receiver}, \mathrm{type}, \tau, \mathbf{c}, \mathbf{s}, \mathrm{payload} \bigr)$$

Payload schemas are registered; message tags propagate provenance and versioning.

Tele-LLM-Hub realizes this protocol via low-code workflow orchestration wherein agents (Gen-LLM, Val-LLM, Debug-LLM) communicate through TeleMCP nodes, context brokers, and schema-managed payloads. Integration with srsRAN and PyShark enables direct ingestion and normalization of telecom telemetry. Schema versioning, message granularity, and access controls ensure principled system scalability and auditability.

6. Cryptographic Steganography via LLM-CAT

A distinct manifestation of the LLM-CAT protocol enables cryptographically secure, covert communication over chat channels using LLM-generated humanlike text (Gligoroski et al., 11 Apr 2025):

• Parties share a key or password for symmetric AEAD or public-key cryptosystem (ECDH/ECDHE/KEM).
• Ciphertext+tag is hex-encoded, mapped to frequent English letters via $h(.)$.
• Pseudorandom embedding positions $\mathbf{b}$ are seeded by derived keys, and embedding proceeds by forcing mapped characters at designated positions during LLM generation, using top-$k$ sampling and temperature tuning.

The embedding algorithm $\mathsf{EmbedderLLM}$ and extraction algorithm $\mathsf{Extract}$ are specified with LaTeX pseudocode. Security is rigorously analyzed: for optimal ranges $T \in [0.7, 0.9]$, $k \in [40, 60]$, token-level indistinguishability probability is bounded by $2^{-\mathrm{sec}}$. Theorem 1 establishes computational indistinguishability from ordinary chat, given secret embedding positions and public model distributions.

Empirically, the method achieves $\sim 0.5$ bytes/token throughput, $99.8\%$ embedding success, and $\sim100$ B/s covert data rate on standard GPUs. The approach is LLM-agnostic and supports post-quantum AEAD and KEM schemes.

7. Practical Considerations, Limitations, and Extensions

LLM-CAT protocols introduce several practical tradeoffs and open research avenues:

• Training overhead is doubled by the requirement for dual model forward passes but can be mitigated by caching or batching [$p_L$].
• Masking non-differentiable tokens allows focused calibration, but may discard rare informative samples. Re-weighted masking is a plausible future direction.
• The protocol applies to any two-model (or deeper) cascade, to multi-agent domains via workflow-managed context objects, and to cryptographic settings provided tokenization and decoding parameters align.
• Extensions include threshold learning, reinforcement learning from human feedback (RLHF), federated/on-device scenario deployment, calibration of confidence signals, and improved synonym-driven embedding algorithms.
• Security relies on shared PRF seeds and consistent encoding; adversaries lacking these secrets are provably unable to distinguish covert LLM-CAT communications.

LLM-CAT thus serves as a foundation for resource-optimal inference, robust and interpretable benchmarking, secure context exchange in complex multi-agent environments, and the embedding of cryptographic communications indistinguishable from normal human/LLM chat. Applications range from zero-shot task evaluation to next-generation wireless network management and post-quantum encrypted messaging.