SmolRGPT: Compact GPT for RAG & Warehouse Reasoning

• SmolRGPT is a family of compact GPT systems that optimize deployment by constraining computation through task-specific design.
• One variant employs a Tiny-Critic RAG pipeline with LoRA-tuned SLM for deterministic binary routing, while the other fuses RGB and depth for spatial reasoning.
• The architecture achieves low latency and reduced cost despite minor trade-offs in performance, paving the way for further research in efficiency and robustness.

SmolRGPT denotes compact GPT-oriented systems designed for deployment under stringent compute, latency, or memory constraints. In the arXiv record, the name is used in two distinct senses: first, as a RAG-based GPT pipeline derived from the Tiny-Critic RAG framework, in which a parameter-efficient Small LLM (SLM) performs binary routing before generation or fallback (Wu et al., 1 Mar 2026); second, as a 600M-parameter vision-language architecture for warehouse spatial reasoning, in which RGB and depth region embeddings are injected into a pretrained LLM through a three-stage curriculum (Traore et al., 18 Sep 2025). This suggests that the shared name identifies an efficiency-first design orientation rather than a single unified architecture.

1. Terminological scope and research contexts

The term covers two substantially different systems. One belongs to the Retrieval-Augmented Generation literature and is concerned with factuality, hallucination mitigation, routing, and agentic fallback. The other belongs to multimodal spatial reasoning and is concerned with RGB-depth fusion, region-level reasoning, and industrial warehouse deployment.

Usage of “SmolRGPT”	Problem setting	Defining mechanism
Tiny-Critic-derived SmolRGPT	High-throughput or agentic RAG	LoRA-tuned SLM, deterministic gatekeeper, constrained single-token pass/fail routing
Warehouse SmolRGPT	Warehouse spatial reasoning	Region-level RGB+depth pooling, token insertion into SmolLM2-360M, three-stage curriculum

A common misconception is to treat SmolRGPT as a single model family. The available sources instead document two separate constructions with different modalities, training procedures, and evaluation suites. In one case, “SmolRGPT” is the name given in a technical report that operationalizes Tiny-Critic RAG as a compact Retrieval-Augmented GPT pipeline (Wu et al., 1 Mar 2026). In the other, “SmolRGPT” is the formal title of a warehouse-oriented vision-LLM (Traore et al., 18 Sep 2025).

2. SmolRGPT as a Tiny-Critic-derived RAG system

In the Tiny-Critic setting, SmolRGPT is defined as a compact, cost-effective RAG-based GPT pipeline in which the heavyweight evaluator used in reflective RAG is replaced by a parameter-efficient SLM fine-tuned with Low-Rank Adaptation. The motivating claim is that running a full-blown LLM such as GPT-4o-mini merely to decide whether retrieved evidence is acceptable introduces severe computational redundancy in high-throughput systems, while inaccurate retrieval in autonomous-agent settings can induce excessive tokens, spurious reasoning, redundant tool calls, inflated Time-to-First-Token (TTFT), and higher cost (Wu et al., 1 Mar 2026).

The adaptation mechanism is standard LoRA. Starting from a pre-trained SLM such as Qwen-1.7B, each Transformer weight matrix $W_0 \in \mathbb{R}^{d \times m}$ is frozen and augmented with a low-rank update,

$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$

with

$$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$

The reported setting uses $r=16$ as intrinsic rank and $\alpha=32$ as LoRA scaling factor, applied to the $W_q,W_k,W_v,W_o$ projection matrices in each self-attention block. For $d=m=4096$ and $r=16$, the extra parameter ratio per matrix is approximately $0.2\%$, and the report states that across all four projections and all layers the total added overhead is $\ll 1\%$ of the base model (Wu et al., 1 Mar 2026).

Training is formulated as final-token binary classification with labels “pass” and “fail,” using standard cross-entropy:

$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$0

This recasts evidence evaluation as a narrow routing problem rather than a full generative reasoning problem. The resulting SmolRGPT blueprint therefore decouples evaluation from generation: the small model decides whether retrieval is acceptable, while the GPT-style generator is invoked only when that gate is opened.

3. Deterministic routing, non-thinking inference, and system-level effects

The core routing mechanism casts the decision as $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$1, with a joint representation $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$2 extracted from the final-token embedding and scored by a sigmoid gate:

$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$3

The decision rule is deterministic, with threshold $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$4, and in practice the SLM is forced to emit exactly one token, “t_pass” or “t_fail.” Constrained decoding is implemented by masking every vocabulary item except those two tokens, so decoding halts after the first autoregressive step. The report states that this enforces latency $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$5, consisting only of the KV-cache fill step, and eliminates sampling nondeterminism (Wu et al., 1 Mar 2026).

The same framework explicitly disables chain-of-thought in the critic. Only the first autoregressive step is allowed, and all other positions are forbidden. The token count per routing invocation therefore drops from $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$6 tokens to $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$7. With

$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$8

the reported empirical comparison is

$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$9

which is described as a $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$0 reduction. The corresponding token savings per 10k queries are reported as $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$1, with implicit cost savings of approximately $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$21.20$ due to avoided <a href="https://www.emergentmind.com/topics/react" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">ReAct</a> wasted chain-of-thought (<a href="/papers/2603.00846" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Wu et al., 1 Mar 2026</a>).</p> <p>The end-to-end evaluation reported for the compact RAG setting is summarized below.</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Model</th> <th>Routing / faithfulness</th> <th>TTFT / <a href="https://www.emergentmind.com/topics/cluster-promoting-quantization-cpq" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">CPQ</a></th> </tr> </thead><tbody><tr> <td>Naive RAG</td> <td>Routing F1: N/A; Faithfulness: 0.44</td> <td>450 ms; \$0.00 Heavy-CRAG Routing F1: 0.934; Faithfulness: 0.88 1235 ms; \$3.00 Tiny-Critic Routing F1: 0.912; Faithfulness: 0.86 492 ms; \$0.06

The same report defines Faithfulness as “RAGAS hallucin. density in $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$3” and CPQ as “explicit routing Cost per 10k q’s, AWS G5 pricing.” It further states that Figures 1 and 2 show Tiny-Critic RAG almost matching the robustness of the gpt-4o-mini baseline while residing in the lower-left corner of the cost-latency plane (Wu et al., 1 Mar 2026).

The integration recipe for SmolRGPT is explicitly modular. A retriever such as BGE-M3 fetches top-$$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$4 documents $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$5; the Tiny-Critic performs a single-token pass/fail classification; if $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$6, $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$7 is forwarded to the GPT-style generator; if $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$8, a fallback tool $$A\in\mathbb{R}^{d\times r},\quad B\in\mathbb{R}^{r\times m},\quad r \ll \min(d,m).$$9 such as Bing or Specialist-QA is invoked to obtain cleaned $r=16$0, and generation proceeds on $r=16$1. The specified parameter budget uses a 1.7B-parameter base SLM with LoRA overhead of approximately $r=16$2 or about $r=16$3M parameters, a target critic latency below $r=16$4 ms per decision, and full TTFT below $r=16$5 ms. The documented trade-off is a slight F1 drop, $r=16$6 versus $r=16$7, in exchange for $r=16$8 lower explicit cost (Wu et al., 1 Mar 2026).

4. SmolRGPT as a warehouse vision-LLM

In the warehouse literature, SmolRGPT is a compact vision-language architecture for spatial reasoning with RGB and depth. The inputs are an RGB image $r=16$9 and its aligned depth map $\alpha=32$0. A shared frozen SigLIP2-base-patch16-256 backbone $\alpha=32$1 maps each modality to a $\alpha=32$2-D feature tensor,

$\alpha=32$3

Two separate connectors then project these features into the LLM embedding space using pixel-shuffle and a linear layer,

$\alpha=32$4

Two refiner modules, one per modality, use transpose-conv blocks to upsample the projected features back to a spatial resolution matching region masks, producing $\alpha=32$5 and $\alpha=32$6 (Traore et al., 18 Sep 2025).

Region-level pooling is central. Given $\alpha=32$7 region masks $\alpha=32$8, the refined RGB and depth features are pooled to form

$\alpha=32$9

These region features are inserted as special tokens, <mask_rgb> and <mask_depth>, into the token sequence of a pretrained SmolLM2-360M LLM. Fusion occurs without introducing a new cross-modal self-attention layer; instead, each pooled region embedding is treated as a language-model token. If $W_q,W_k,W_v,W_o$0 are word embeddings and $W_q,W_k,W_v,W_o$1 is the concatenated region representation, the transformer input takes the form

$W_q,W_k,W_v,W_o$2

Within each transformer layer, multi-head attention remains standard:

$W_q,W_k,W_v,W_o$3

This allows joint attention over text and region-level visual embeddings inside the pretrained LLM (Traore et al., 18 Sep 2025).

The design is therefore region-centric rather than patch-centric at the language interface. RGB and depth are kept separate through connectors and refiners, then fused at the region-token stage. This suggests that the architecture is optimized for spatial relations tied to localized objects or zones, rather than for unconstrained image captioning alone.

5. Curriculum learning, efficiency profile, and reported benchmarks

Training proceeds in three stages. Stage 1, Global Feature Alignment, uses LLaVA-CC3M-595K with 595K image-caption pairs. Only the RGB connector $W_q,W_k,W_v,W_o$4 is trainable; SigLIP2, the refiners, the depth connector, and the LLM remain frozen. The objective is language-modeling cross-entropy, with learning rate $W_q,W_k,W_v,W_o$5, $W_q,W_k,W_v,W_o$6 epochs, and batch size $W_q,W_k,W_v,W_o$7. Stage 2, Spatial Warm-Up, uses the first $W_q,W_k,W_v,W_o$8M examples of the Open Spatial Dataset. The trainable components are the depth connector $W_q,W_k,W_v,W_o$9 and both refiners, while the RGB connector is frozen. The objective combines language-modeling cross-entropy, spatial relation classification, and distance regression:

$d=m=4096$0

This stage runs for one epoch, specified as $d=m=4096$1 steps, with learning rate $d=m=4096$2 and batch size $d=m=4096$3. Stage 3, Warehouse-Specific Fine-Tuning, uses PhysicalAI-Spatial-Intelligence-Warehouse with $d=m=4096$4K+ QA pairs. All modules except SigLIP2 are trainable, and the objective becomes

$d=m=4096$5

The reported hyperparameters are learning rate $d=m=4096$6, $d=m=4096$7 epochs to convergence or $d=m=4096$8 steps, and batch size $d=m=4096$9 (Traore et al., 18 Sep 2025).

The parameter breakdown totals approximately $r=16$0M. SigLIP2-base-patch16 contributes $r=16$1M frozen parameters, the SmolLM2 LLM contributes $r=16$2M, RGB and depth connectors contribute $r=16$3M, the two refiners contribute $r=16$4M, and mask-pooling with small heads contributes $r=16$5M, for a total trainable count of approximately $r=16$6M plus $r=16$7M frozen. Training used two NVIDIA RTX 5090 GPUs. On a single 5090, end-to-end inference for a $r=16$8 input is reported as approximately $r=16$9 ms per image in FP16, with peak GPU VRAM around $0.2\%$0 GB in mixed precision. Compared to a 7B-parameter SpatialRGPT requiring approximately $0.2\%$1 GB VRAM and approximately $0.2\%$2 ms per image, SmolRGPT is reported as approximately $0.2\%$3 faster with $0.2\%$4 the parameters (Traore et al., 18 Sep 2025).

On the AI City Challenge 2025 Track 3 warehouse benchmark, the reported task metrics are: Left-Right accuracy $0.2\%$5; Count accuracy $0.2\%$6 with RMSE $0.2\%$7; Multiple-Choice accuracy $0.2\%$8; and Distance accuracy $0.2\%$9 with RMSE $\ll 1\%$0. The overall “S1” score is $\ll 1\%$1, corresponding to $\ll 1\%$2rd place among $\ll 1\%$3 teams. On SpatialRGPT-Bench over OSD for qualitative spatial relations, SmolRGPT reaches an average accuracy of $\ll 1\%$4, compared with $\ll 1\%$5 for SpatialRGPT, $\ll 1\%$6 for GPT-4V, and $\ll 1\%$7 for GPT-4. On quantitative spatial estimation within $\ll 1\%$8 of ground truth, SmolRGPT reports DirectDist $\ll 1\%$9, Width $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$00, Height $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$01, and Direction $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$02 (Traore et al., 18 Sep 2025).

The ablations identify the roles of the curriculum and modality-specific components. Removing Stage 2 reduces left/right accuracy to $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$03 and increases distance RMSE by $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$04. Freezing the depth refiner in Stage 3 reduces count accuracy by approximately $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$05. Pixel-shuffle, compared with standard flattening, reduces GPU memory by approximately $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$06. These results place the architecture’s efficiency gains in explicit correspondence with design choices rather than parameter count alone (Traore et al., 18 Sep 2025).

6. Comparative interpretation, misconceptions, and open directions

The two SmolRGPT usages solve different problems but share a common systems-level pattern: they reduce deployment cost by narrowing the expensive part of the computation to the part that is strictly necessary. In the Tiny-Critic setting, the expensive operation is evaluator inference inside a reflective or agentic RAG loop, so SmolRGPT replaces it with deterministic binary routing and single-token decoding. In the warehouse setting, the expensive operation is broad multimodal reasoning at large scale, so SmolRGPT preserves a compact language core while externalizing modality alignment to connectors, refiners, and region pooling. This suggests a broader interpretation of “small” in these works: compactness is achieved not only by parameter reduction, but by architectural restriction of the task interface (Wu et al., 1 Mar 2026).

Another misconception is that the shared name implies equivalent capability classes. The RAG-oriented SmolRGPT is a pipeline blueprint for hallucination-aware retrieval routing; the warehouse SmolRGPT is a multimodal model for region-level spatial reasoning. Their evaluation criteria therefore differ fundamentally: routing F1, TTFT, and CPQ in one case, and task accuracy, RMSE, VRAM, and latency per image in the other. Direct performance comparison between them would be category error.

The limitations are also domain-specific. In the Tiny-Critic configuration, the reported compromise is a slight routing F1 drop from $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$07 to $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$08 relative to Heavy-CRAG, even though the explicit routing cost falls from $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$093.00$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$10\$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$11 per 10k queries. The same report identifies calibration under domain shift, multi-hop gating with dynamic thresholds, multi-modal adaptation via quantized vision-language LoRA adapters, and online rank adaptation as open challenges. In the warehouse model, absolute dimension estimation remains weak, with Width at $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$12 and Height at $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$13 success within $W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$14 of ground truth; the external answer-normalization pipeline using Longformer and Qwen2.5 adds system complexity; and domain-shift robustness under changing warehouse layouts and lighting has not yet been tested. The proposed future directions include end-to-end answer extraction in the LLM output head, a small adapter for absolute size estimation, domain adaptation through unsupervised or adversarial training, extension to multi-agent scenarios, and aggressive quantization and pruning for sub-$W \;=\; W_0 \;+\;\Delta W \;=\; W_0 \;+\; A\,B,$15 ms mobile or embedded deployment (Traore et al., 18 Sep 2025).

Taken together, the SmolRGPT label currently refers not to a single canonical model but to a class of compact, deployment-oriented GPT systems whose defining property is efficient task specialization. In one lineage, that specialization is binary routing in agentic RAG; in the other, it is region-grounded RGB-depth reasoning for warehouse environments.