Adaptive Chunking in AI and Cognitive Systems

Updated 4 July 2026

Adaptive Chunking is a method where the unit of processing is dynamically determined by context, uncertainty, and resource constraints.
It optimizes segmentation in systems such as multimodal tokenization, retrieval-augmented generation, and robot control, leading to improved coherence and efficiency.
Applications span from compiler-level memory management to reinforcement learning, showcasing enhanced performance through adaptive partitioning.

Adaptive chunking denotes a family of methods in which the unit of processing is not fixed in advance but is determined from context, task signals, uncertainty, or resource constraints. In recent work, the term covers dynamic cross-modal tokenization in multimodal LLMs, query-conditioned document segmentation in retrieval-augmented generation, variable action horizons in robot control, compiler-level partitioning for memory-efficient inference, and curriculum-controlled compression in tokenization-free hierarchical models (Yu, 3 May 2025, Rastogi, 29 Apr 2026, Liang et al., 5 Apr 2026, Zhao et al., 2024, Dang et al., 28 May 2026). Across these literatures, the common move is from static partitions toward units that are variable-sized, context-sensitive, and selected to balance coherence, generalization, reactivity, or system efficiency (Yu, 3 May 2025, Sheng et al., 1 Jun 2025).

1. Definitions and conceptual scope

In cognitive and computational usage, chunking refers to grouping lower-level elements into higher-level units that are useful for memory, inference, or control. In the multimodal-cognition literature, human chunks are described as dynamic, hierarchical, and cross-modal: boundaries shift with context and task, representations compose across levels such as characters $\rightarrow$ words $\rightarrow$ phrases or pixels $\rightarrow$ objects $\rightarrow$ scenes, and visual regions can be integrated with linguistic phrases into a single unit (Yu, 3 May 2025). The same work frames static tokenization in multimodal models as static, modality-specific, and flat, and identifies this contrast as a bottleneck for human-like multimodal reasoning.

A broader theoretical formulation appears in work on cognitive evolution, where chunking is defined as a form of non-elemental learning in which combinations of elements acquire a meaning distinct from their components. On that view, adaptive chunking is not a fixed capacity but a tunable process governed by parameters such as weight-increase rates, decay rates, fixation thresholds, temporal windows for co-activation, and attentional filters; the central trade-off is between under-chunking and over-chunking, with consequences for generalization, specificity, and memory load (Lotem et al., 20 Jan 2025). This line of work treats chunking as a general organizing principle for configural discrimination, sequence learning, problem solving, and cognitive flexibility.

The term also has a distinct but related meaning in systems and control. In robot learning, a chunk is often an action sequence executed open loop before replanning; adaptive chunking then means deciding when to keep a plan and when to replan, or how many actions to execute before the next policy call (So et al., 14 Oct 2025, Liang et al., 5 Apr 2026, Feng et al., 2 Jun 2026). In compiler and systems work, chunking means splitting computation or ingestion into pieces chosen to satisfy a memory budget or edge-resource constraint, with the adaptive element lying in the selection of chunk regions, tensor dimensions, batch sizes, or chunk sizes from current resource conditions (Zhao et al., 2024, Yagiz et al., 30 Apr 2026).

2. Mechanisms and formalizations

A recurring formal pattern is the use of a score, probability, or value to decide whether to place a boundary, extend a segment, or terminate execution. The specific signal varies by domain, but the operational structure is similar: estimate local structure, compute a criterion, then choose a chunk boundary or chunk length (Yu, 3 May 2025, Rastogi, 29 Apr 2026, Liang et al., 5 Apr 2026, Zhao et al., 2024).

Domain	Adaptive signal	Decision
Multimodal tokenization	$B(x;\theta)=\sigma(f_\theta(x)-\tau)$	place soft token boundaries
Query-adaptive RAG	sentence–query cosine similarity and chunk score aggregation	choose seeds, expand windows, retain chunks
Robot control	cosine similarity, action entropy, denoising variance, or multi-chunk Q-values	replan, extend a queue, or execute a prefix
Systems optimization	activation memory, available memory, or memory pressure	split computation or ingestion under a budget

In dynamic cross-modal tokenization, the core construct is a differentiable boundary detector,

$B(x; \theta) = \sigma(f_\theta(x) - \tau),$

where $x$ is fine-grained input, $f_\theta(x)$ is a learned boundary score, and $\tau$ is a threshold (Yu, 3 May 2025). Because $B(x;\theta)\in(0,1)$ , segmentation can remain soft during training, and higher-level chunks can be formed by weighted aggregation between likely boundaries. The same framework combines bottom-up aggregation with top-down refinement,

$\rightarrow$ 0

so adaptive chunking affects both composition and decomposition across representational levels.

In query-adaptive semantic chunking for RAG, the central signals are sentence–query cosine similarities, an adaptive percentile threshold for seed selection, contextual window expansion around seeds, and a chunk-level weighted score based on proximity to the seed sentence (Rastogi, 29 Apr 2026). In ultra-long reading comprehension, dynamic chunking similarly relies on semantic similarities between adjacent sentences, but chunk retention is then controlled by a question-aware classifier trained to predict whether a chunk is answerable for a given question (Sheng et al., 1 Jun 2025).

In robotic action chunking, the adaptive variable is not a text boundary but an execution horizon. One line of work compares the first action in an already queued chunk with the first action in a newly sampled chunk and either keeps the queue or replaces it depending on cosine similarity (So et al., 14 Oct 2025). Another estimates entropy over candidate action chunks and chooses a chunk size from a “knee point” in the average-entropy curve, while enforcing a minimum action-magnitude constraint (Liang et al., 5 Apr 2026). A third uses denoising dynamics in flow-based policies: variance of clean-action estimates over the final denoising steps defines a stable low-variance prefix, and a rolling estimate of the local variance scale sets an adaptive threshold for replanning (Feng et al., 2 Jun 2026).

Compiler-level adaptive chunking formalizes a different optimization problem. For a module $\rightarrow$ 1, chunking reduces activation memory from

$\rightarrow$ 2

$\rightarrow$ 3

where $\rightarrow$ 4 is the chunk factor (Zhao et al., 2024). The challenge is then to choose chunk regions, chunk dimensions, and chunk sizes that satisfy a memory budget while minimizing speed loss.

3. Representation learning and multimodal modeling

In multimodal LLMs, adaptive chunking has been instantiated as a redefinition of what counts as a token. The Dynamic Cross-Modal Tokenization framework adds adaptive token boundaries, hierarchical token representations, and cross-modal alignment modules to a transformer-based multimodal model, and reports statistically significant improvements over state-of-the-art baselines: $\rightarrow$ 5 on Visual Question Answering and $\rightarrow$ 6 on Complex Scene Description, with test-set results of $\rightarrow$ 7 on VQA, $\rightarrow$ 8 on Complex Scene, $\rightarrow$ 9 on GQA, and $\rightarrow$ 0 on CMCE (Yu, 3 May 2025). The same study also reports a correlation of $\rightarrow$ 1 between model attention maps and human gaze, compared with $\rightarrow$ 2 for baselines, and a contextual modulation index of $\rightarrow$ 3 versus $\rightarrow$ 4, indicating more human-like boundary variability and context sensitivity.

Adaptive chunking has also been used to replace static patchification in diffusion transformers. The Dynamic Chunking Diffusion Transformer starts from patch size $\rightarrow$ 5, uses an encoder–router–decoder scaffold, selects boundary tokens through local similarity, and assigns non-boundary positions to the nearest boundary position on the grid, yielding irregular, data-dependent spatial chunks (Haridas et al., 6 Mar 2026). The router learns to compress uniform background regions into fewer tokens and detail-rich regions into more tokens, and it also adapts its compression across diffusion timesteps, using fewer tokens at noisy stages and more tokens as fine details emerge. On class-conditional ImageNet $\rightarrow$ 6, the method improves both FID and Inception Score over parameter-matched and FLOP-matched DiT baselines across $\rightarrow$ 7 and $\rightarrow$ 8 compression; for example, at XL scale and $\rightarrow$ 9 compression it reports FID $\rightarrow$ 0 and IS $\rightarrow$ 1, compared with FID $\rightarrow$ 2 and IS $\rightarrow$ 3 for the FLOP-matched DiT baseline (Haridas et al., 6 Mar 2026).

In tokenization-free hierarchical language modeling, adaptive chunking appears as compression control rather than boundary prediction alone. Adaptive Targeted Dynamic Chunking treats the target compression ratio $\rightarrow$ 4 as a function of training time, uses curriculum learning to move from $\rightarrow$ 5 to $\rightarrow$ 6, and tracks realized chunk sizes through Bytes-Per-Innermost-Chunk (BPIC) (Dang et al., 28 May 2026). On FineWeb-Edu 100B, the byte-level H-Net models equipped with ATDC improve Bits-Per-Byte over fixed-compression counterparts: the 680M model reaches $\rightarrow$ 7 BPB versus $\rightarrow$ 8, and the 1.3B model reaches $\rightarrow$ 9 versus $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 0, while also improving zero-shot average downstream accuracy from $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 1 to $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 2 at 680M and from $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 3 to $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 4 at 1.3B (Dang et al., 28 May 2026). A plausible implication is that adaptive chunking in representation learning increasingly functions as learned tokenization: the model learns not only embeddings, but also the units over which those embeddings should be computed.

4. Retrieval-augmented generation and ultra-long reading

In retrieval-augmented generation, adaptive chunking is used both to construct better retrieval units and to choose among chunking methods. Query-Adaptive Semantic Chunking makes segmentation query-dependent by selecting seed sentences with high sentence–query cosine similarity, expanding contextual windows around them, and retaining windows whose aggregated chunk scores exceed a threshold (Rastogi, 29 Apr 2026). On 100 technical documents and 200 queries, QASC reports Precision $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 5, Recall $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 6, and F1 $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 7, corresponding to an 18–27% relative improvement over fixed chunking and an 8–12% improvement over semantic and agentic alternatives; human evaluation by three annotators with Cohen kappa $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 8 also rated QASC higher on relevance, coherence, completeness, and answer quality (Rastogi, 29 Apr 2026).

A related approach for ultra-long question answering separates semantic segmentation from chunk selection. Dynamic Chunking and Selection computes semantic similarities between adjacent sentences to form variable-length chunks and then applies a question-aware classifier to select “sensitive chunks” for a compressed prompt (Sheng et al., 1 Jun 2025). On single-hop and multi-hop benchmarks, the method consistently outperforms strong baselines and maintains robustness up to 256k tokens. On LLaMA-3-8B-Instruct, it reports an average of $B(x;\theta)=\sigma(f_\theta(x)-\tau)$ 9 on six single-hop datasets and $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 0 on six multi-hop datasets, corresponding to improvements of $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 1 and $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 2 over the previous best averages, with especially large gains on long-context datasets such as Loogle_SD, Factrecall_en, and HotpotWikiQA (Sheng et al., 1 Jun 2025).

A more document-centric variant selects the chunking method rather than only the chunks. “Adaptive Chunking: Optimizing Chunking-Method Selection for RAG” proposes five intrinsic, document-based metrics—References Completeness, Intrachunk Cohesion, Document Contextual Coherence, Block Integrity, and Size Compliance—and chooses the most suitable chunking strategy for each document (Júnior et al., 26 Mar 2026). On a corpus spanning legal, technical, and social science domains, this document-aware procedure raises answer correctness to $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 3 from $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 4 and increases the number of successfully answered questions by over 30% ( $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 5 vs. $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 6) without changing models or prompts (Júnior et al., 26 Mar 2026).

These methods sit alongside related representation-level adaptations. A comparative study of late chunking and contextual retrieval concludes that contextual retrieval preserves semantic coherence more effectively but requires greater computational resources, whereas late chunking offers higher efficiency but tends to sacrifice relevance and completeness (Merola et al., 28 Apr 2025). This suggests that adaptive chunking in RAG is not a single algorithmic family but a spectrum of interventions: adaptive boundaries, query-time re-chunking, answerability-based selection, and document-level method selection.

5. Action chunking in robotics and reinforcement learning

In robot learning, chunking typically means predicting or executing a sequence of future actions rather than a single action, and adaptive chunking means modulating that sequence online. In Generative Behavior Cloning with diffusion policies, adaptive chunking is implemented by maintaining an action queue, sampling a new chunk at every timestep, comparing the first queued action with the first action of the newly sampled chunk, and replacing the queue only when cosine similarity falls below a threshold $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 7 (So et al., 14 Oct 2025). This mechanism is designed to preserve temporal consistency during smooth motion while permitting replanning during high-precision phases. Within the SGAC framework, the combined self-guidance and adaptive chunking method exceeds vanilla Diffusion Policy by $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 8 and BID by $B(x; \theta) = \sigma(f_\theta(x) - \tau),$ 9 on average in the main DDIM-30 setting, and in a real-world stochastic task with a periodically moving cup it achieves $x$ 0 success over 20 trials while running at approximately $x$ 1 Hz (So et al., 14 Oct 2025).

Adaptive Action Chunking for Vision-Language-Action models uses a different signal: action entropy estimated from multiple sampled action chunks (Liang et al., 5 Apr 2026). For each candidate chunk size $x$ 2, it computes the average entropy of translation, rotation, and gripper actions, identifies a “knee point” in the curve $x$ 3, and then enforces a minimum action-magnitude lower bound. This inference-time mechanism requires no retraining and improves several backbones. With GR00T N1.5, RoboCasa average success rises from $x$ 4 to $x$ 5, LIBERO from $x$ 6 to $x$ 7, and real-world average success across three tasks from $x$ 8 to $x$ 9 (Liang et al., 5 Apr 2026).

Denoising-Variance Adaptive Chunking derives its signal directly from the generative process of flow-based policies. It computes the variance of clean-action estimates over the final denoising steps for each future action index, executes the stable low-variance prefix, and calibrates the threshold with a rolling estimate of local variance scale (Feng et al., 2 Jun 2026). With a $f_\theta(x)$ 0-based policy, DVAC improves LIBERO success from $f_\theta(x)$ 1 to $f_\theta(x)$ 2 and reduces replanning by $f_\theta(x)$ 3; it also yields aggregate gains on RoboTwin and CALVIN and improves real-world execution efficiency (Feng et al., 2 Jun 2026). The paper also provides a bound linking endpoint error to $f_\theta(x)$ 4, giving denoising variance a formal role as a stability signal rather than only a heuristic.

In offline-to-online reinforcement learning, Adaptive Action CHunking extends the idea from execution heuristics to value learning. ACH estimates Q-values for all candidate chunk lengths in a single forward pass using a causal Transformer critic, samples a full action chunk, evaluates all prefixes, and selects the chunk length from a softmax over prefix Q-values (Shin et al., 11 May 2026). Evaluated on 34 tasks, the method consistently outperforms fixed-length baselines, indicating that chunk length is a state-dependent temporal abstraction rather than a global hyperparameter.

6. Systems optimization, continual adaptation, and theoretical trade-offs

Adaptive chunking is also a systems technique for making computation or storage fit operational constraints. AutoChunk treats activation-memory reduction as a compiler optimization problem, searches legal chunk regions in a computation graph, and selects chunk plans under a memory budget (Zhao et al., 2024). It reports over 80% activation-memory reduction while maintaining speed loss within 10%, extends maximum sequence length by $f_\theta(x)$ 5 to $f_\theta(x)$ 6, and still yields over 70% additional reduction even when memory-efficient attention kernels are already used. ChunkWise LoRA applies adaptive sequence partitioning to low-rank adaptation: a runtime scheduler estimates token difficulty, partitions the sequence into variable-length chunks, and assigns chunk-specific LoRA rank and KV-cache policies, giving up to 34% lower latency and 38% memory reduction compared to baseline LoRA while maintaining or improving BLEU, EM, and perplexity (Thakkar et al., 28 Jan 2026). In IoT edge logging, adaptive chunking controls how many log entries are processed per batch from available memory and memory pressure, enabling a Merkle-tree pipeline with throughput exceeding 130,000 logs/s for 100,000 records, per-entry verification latency of approximately 22 ms, an average proof size of 1,006 bytes, and peak memory usage below 5 MB (Yagiz et al., 30 Apr 2026).

Streaming and continual-learning settings highlight a different adaptive role for chunk size. Chunk-Adaptive Restoration changes the chunk size of block-based data-stream classifiers after concept drift detection, uses a Variance-based Stabilization Detection Method to decide when the system has stabilized, and then returns to the base chunk size (Kozal et al., 2021). The experimental study, backed up with statistical tests, reports that CAR significantly reduces restoration time; for abrupt drift, Sample Restoration can be improved by around 50% relative to the baseline for some configurations (Kozal et al., 2021). In a broader unsupervised setting, the Continual General Chunking Problem formalizes chunk discovery in non-stationary streams, and SyncMap addresses it with a self-organizing dynamic map that adapts to changing structure without an explicit loss function, surpassing or tying with the best comparator on 66% of scenarios while being the second best in the remaining 34% (Vargas et al., 2020).

Theoretical work places these engineering results in a larger trade-off structure. Adaptive fine-tuning of chunking mechanisms has been proposed as an explanation for cognitive diversity, with chunk formation shaped by ecological structure and constrained by combinatorial explosion, misleading specificity, and the tension between generalization and specificity (Lotem et al., 20 Jan 2025). Across current machine-learning applications, a comparable pattern recurs: smaller or more frequent chunks improve responsiveness and reduce restoration time, while larger or more compressed chunks improve efficiency, coherence, or long-range structure. This suggests that adaptive chunking is best understood not as a single method, but as a control principle for selecting the granularity of representation or execution under changing informational and resource conditions.