Sentence-Based Chunking Strategy

Updated 6 July 2026

Sentence-based chunking is a structural segmentation method that preserves entire sentences to maintain local semantic integrity and avoid meaning fragmentation.
It serves as a robust baseline in NLP pipelines such as retrieval-augmented generation, offering a compromise between fixed-size splitting and complex semantic segmentation.
Empirical studies show its effectiveness varies by domain, with adaptive or hierarchical methods outperforming it in low-resource, legal, or highly structured documents.

Sentence-based chunking is a structure-based text segmentation strategy in which chunk boundaries respect sentence boundaries and chunks are formed from whole sentences rather than arbitrary character or token windows. In retrieval-augmented generation (RAG), dense retrieval, and related NLP pipelines, it is commonly used as a middle ground between fixed-size splitting, which may fragment meaning, and more complex semantic or LLM-guided segmentation, which adds computational and implementation cost. Recent work shows that its behavior is strongly domain-dependent: it is a strong production baseline in some English question-answering settings, but it is not uniformly optimal in low-resource procedural text, legal corpora, or long structured academic documents, where higher-level document structure can matter more than sentence boundaries alone (Chhoun et al., 21 May 2026, Bennani et al., 20 Jan 2026, Zhou et al., 19 Feb 2026, Prior et al., 19 May 2026).

1. Definition and segmentation mechanics

Sentence-based chunking is typically defined by one constraint: no sentence is split. Chunks are produced by detecting sentence boundaries and aggregating contiguous sentences into units of a chosen size. In one representative formulation, the method first detects sentence delimiters using punctuation markers such as the Khmer full stop “។”, then groups sentences into overlapping windows of size $N=5$ . If a document contains sentences $S1$ through $S8$ , the first chunk is $[S1\text{–}S5]$ and the next overlapping chunk is $[S5\text{–}S8]$ (Chhoun et al., 21 May 2026).

Within current chunking taxonomies, sentence-based chunking is a structure-based segmentation method alongside fixed-size and paragraph-based splitting. A common experimental default is 5 sentences per chunk, implemented with a regex-based sentence splitter. The strategy is presented as a deterministic compromise between preserving enough local context for meaning and avoiding the brittleness of arbitrary token windows (Zhou et al., 19 Feb 2026).

The same segmentation principle appears in several operational variants. In some pipelines, sentence chunks are the final retrieval units; in others, sentences are only the atomic substrate from which larger or adaptive chunks are later constructed. For example, FreeChunker treats sentences as atomic units, encodes each sentence individually, and then supports arbitrary sentence combinations through a chunk pattern mask rather than committing to one fixed granularity in advance (Zhang et al., 23 Oct 2025). HiChunk likewise begins with sentence tokenization, but treats sentence segmentation as the base representation from which hierarchical chunk points are inferred rather than as the final retrieval structure (Lu et al., 15 Sep 2025).

The central rationale is semantic coherence at the local linguistic level. Because the method never cuts through a sentence, it is intended to preserve sentence-level meaning better than fixed character or token windows. This assumption is stronger in corpora with reliable punctuation and weaker in corpora where sentence segmentation is irregular or where the semantically relevant unit is larger than a sentence (Bennani et al., 20 Jan 2026, Chhoun et al., 21 May 2026).

2. Position in RAG pipelines and evaluation frameworks

Sentence-based chunking is usually embedded in a standard RAG workflow. One explicit six-stage formulation comprises data preparation, chunking, embedding, indexing, retrieval, and evaluation. In the Khmer agricultural setting, source manuals are converted from PDF to text and normalized into Markdown, segmented by one of several chunkers, embedded with the BGE-M3 multilingual model, indexed in FAISS, and queried through dense similarity search. The paper describes BGE-M3 as producing 1,024-dimensional vectors and supporting inputs up to 8,192 tokens. Retrieval uses top- $k$ vector search with L2 distance for nearest-neighbor identification, while evaluation uses cosine similarity for semantic answer alignment (Chhoun et al., 21 May 2026).

The same architectural role appears in industrial English QA. A systematic Natural Questions study evaluates sentence chunking end-to-end with SPLADE retrieval and Ministral-8B-Instruct-2410 generation, varying chunk size $S$ , overlap $O$ , and context budget $C$ . Retrieved chunks are inserted into the generator by a fill-to-budget policy rather than a fixed top- $K$ , which isolates chunking behavior from trivial chunk-count effects (Bennani et al., 20 Jan 2026).

Evaluation protocols for sentence-based chunking often separate retrieval proximity, semantic alignment, and task-specific fidelity. The Khmer study defines four metrics:

$S1$ 0

$S1$ 1

$S1$ 2

$S1$ 3

Here, lower $S1$ 4 is better; higher Answer Relevance, Khmer Coverage, and Khmer IoU indicate better semantic and script-level fidelity (Chhoun et al., 21 May 2026).

Operational cost is also part of the evaluation. In the industrial study, overlap is reported to provide no measurable benefit while increasing indexing cost, with overlap ratio $S1$ 5 inflating chunk count by

$S1$ 6

For example, $S1$ 7 yields 1.25\times more chunks, strengthening the case for sentence chunking with 0\% overlap when the corpus and retriever do not specifically benefit from redundancy (Bennani et al., 20 Jan 2026).

3. Comparative empirical behavior across retrieval settings

Empirical results do not support a universal ranking for sentence-based chunking. Some studies identify it as the most cost-effective default for text-centric RAG; others find that paragraph-level, recursive, or hierarchy-preserving strategies are stronger. The dominant pattern is task dependence (Bennani et al., 20 Jan 2026, Zhou et al., 19 Feb 2026, Kreileder et al., 2 Jul 2026, Prior et al., 19 May 2026).

Study	Setting	Main finding about sentence-based chunking
(Bennani et al., 20 Jan 2026)	Natural Questions, industrial RAG	Most cost-effective; statistically tied with semantic chunking up to $S1$ 8k tokens
(Zhou et al., 19 Feb 2026)	BEIR and GutenQA	Strong baseline in in-corpus retrieval; weaker in in-document retrieval
(Kreileder et al., 2 Jul 2026)	Academic theses	Sentence-cluster semantic chunking did not outperform simpler strategies
(Prior et al., 19 May 2026)	German Civil Code	Strong fine-grained baseline, but section and subsection retrieval were better

In Natural Questions, the reported ordering is sentence $S1$ 9 semantic $S8$ 0 token $S8$ 1 code. Sentence and semantic chunking are described as statistically tied up to $S8$ 2k context tokens, with semantic chunking showing only a slight edge when $S8$ 3k. The same study identifies a context cliff beyond about 2.5k tokens: for sentence chunking with $S8$ 4 and $S8$ 5, BERTScore is stable from 500 to 2.5k tokens and then declines by about 4–5\% relative at 10k. The paper’s recommended production defaults are Sentence chunking, $S8$ 6– $S8$ 7, $S8$ 8, and $S8$ 9 for QA (Bennani et al., 20 Jan 2026).

A broader reproduction across BEIR and GutenQA reaches a different conclusion. Sentence-based chunking is again a reasonable structural baseline, but it is not usually the best structure-based method. In one Jina-v3 BEIR condition, sentence-based chunking averaged 0.4723, compared with 0.4948 for paragraph-based and 0.4849 for fixed-size chunking. For in-document retrieval, sentence-based chunking is clearly weaker than LumberChunker and often behind paragraph-based chunking. Under contextualized chunking, sentence-based retrieval can improve modestly for in-corpus retrieval but degrades in-document retrieval; for GutenQA with Jina-v2, the score falls from 0.3261 to 0.3085 (Zhou et al., 19 Feb 2026).

Evidence from long structured academic texts is also unfavorable to sentence-level semantic grouping in the tested configuration. A study on 13 theses and 10 associated queries reports that cluster-based semantic chunking, implemented over sentences using regex sentence splitting and all-MiniLM-L6-v2, did not improve retrieval or answer quality. For free questions, fixed-size and recursive chunking reached a median Answer Quality Score near 0.65, whereas cluster-based chunking had the lowest median, 0.40. The authors attribute part of the instability to preprocessing artifacts and the fact that faithfulness calculation failed in 44\% of cases, making the evaluation noisy (Kreileder et al., 2 Jul 2026).

4. Low-resource procedural text: the Khmer agricultural case

The clearest negative result for sentence-based chunking comes from Khmer agricultural retrieval, where the method is described as the simplest of four segmentation strategies but not the best-performing one. Documents are procedural and hierarchical, containing headings, explanatory paragraphs, and stepwise instructions in which conditions and actions are distributed across adjacent sentences. In this setting, sentence-based chunking can preserve individual sentence meaning while weakening procedural logic (Chhoun et al., 21 May 2026).

The reported sentence-based results are:

Average Retrieval Score: $[S1\text{–}S5]$ 0
Khmer Coverage: $[S1\text{–}S5]$ 1
Answer Relevance: $[S1\text{–}S5]$ 2
Khmer IoU: $[S1\text{–}S5]$ 3
Total chunks: 1,206

These results are competitive, particularly on Khmer-specific fidelity measures, but they are inferior to the best recursive configuration on every main retrieval-oriented metric. Recursive chunking with a 300-character window achieves the lowest L2 distance, $[S1\text{–}S5]$ 4, the highest Answer Relevance, $[S1\text{–}S5]$ 5, and the highest Khmer IoU, $[S1\text{–}S5]$ 6. A paired $[S1\text{–}S5]$ 7-test shows that Recursive significantly outperforms Sentence-Based in Average Retrieval Score, with $[S1\text{–}S5]$ 8 (Chhoun et al., 21 May 2026).

The authors’ explanation is explicitly linguistic and structural. Sentence-based chunking “often separates conditions from actions,” so the retrieved units fail to preserve the “contextual bundles” needed for actionable procedural meaning. This problem is amplified by Khmer-specific constraints: low-resource tooling, irregular punctuation, complex morphology and orthography, and the fact that Khmer lacks whitespace-delimited word boundaries. In this environment, sentence boundaries are not a reliable proxy for semantic boundaries, and structural preservation matters more than sentence-level segmentation (Chhoun et al., 21 May 2026).

The broader implication is not that sentence-based chunking is ineffective, but that its underlying assumption—sentence boundary as semantic boundary—breaks down in low-resource procedural domains. For Khmer agricultural manuals, the study concludes that document structure is a more important signal than sentence boundaries, and that recursive chunking at an appropriately chosen character window better preserves semantic continuity for dense retrieval (Chhoun et al., 21 May 2026).

5. Sentence-based chunking as a substrate for adaptive and hierarchical methods

Several recent systems retain the sentence as the basic unit but reject static sentence-window chunking. Instead of indexing fixed bundles of adjacent sentences, they use sentence-level signals to construct dynamic, query-aware, or hierarchical segments (Sheng et al., 1 Jun 2025, Rastogi, 29 Apr 2026, Zhang et al., 23 Oct 2025, Lu et al., 15 Sep 2025).

Dynamic Chunking and Selection (DCS) begins by parsing a long context into sentences, builds neighbor-aware sentence representations with Sentence-BERT, computes cosine similarity between adjacent sentence embeddings, converts similarity to boundary scores, and splits at the largest semantic gaps. It then refines sizes to satisfy a target chunk length and applies a question-aware classifier for chunk selection. The method is reported to handle contexts up to 256k tokens and to outperform fixed chunking, overlapping chunking, and cosine-similarity-based selection across 12 long-context QA datasets (Sheng et al., 1 Jun 2025).

Query-Adaptive Semantic Chunking (QASC) makes sentence-level chunking explicitly query-conditioned. Sentences and the query are embedded with Sentence-BERT all-MiniLM-L6-v2, seed sentences are chosen from the top 25\% of sentence–query similarities using an adaptive percentile threshold, and a contextual window of radius $[S1\text{–}S5]$ 9 is expanded around each seed. Candidate chunks are filtered by a weighted aggregate relevance score with decay parameter $[S5\text{–}S8]$ 0 and merged when separated by at most $[S5\text{–}S8]$ 1 sentences. On 100 technical documents and 200 queries, QASC reports Precision 0.85, Recall 0.83, F1 0.85, Answer correctness 0.80, and Faithfulness 0.87, outperforming fixed, semantic, recursive, and agentic alternatives in that study (Rastogi, 29 Apr 2026).

FreeChunker pushes the sentence-centric idea further by eliminating static segmentation altogether. A document is first split into sentences,

$[S5\text{–}S8]$ 2

and each sentence is encoded individually. Chunk candidates are then represented through a Chunk Pattern Mask over sentence positions, allowing multi-granularity sentence combinations to be encoded in parallel. On LongBench V2 Single-Document QA, FreeChunker reports competitive or better retrieval accuracy than Traditional, SemanticChunker, PPL Chunking, and Margin Sampling, while remaining close to traditional chunking in runtime; with BGE-M3, for example, it reports 38.29\% Top-5, 36.57\% Top-10, and 9.64s processing time (Zhang et al., 23 Oct 2025).

HiChunk takes a hierarchical route. It begins with sentence tokenization $[S5\text{–}S8]$ 3, predicts global chunk points $[S5\text{–}S8]$ 4, and supports Auto-Merge retrieval so that fine-grained units can be merged upward into larger semantic parents when evidence is distributed across siblings. On HiCBench evidence-dense tasks, hierarchical chunking with Auto-Merge improves evidence recall over a flat sentence-derived baseline such as FC200, indicating that sentence-level decomposition alone is often insufficient when relevant evidence spans substantial semantic units (Lu et al., 15 Sep 2025).

6. Limitations, domain-specific structure, and broader significance

The principal limitation of sentence-based chunking is that sentence boundaries are not universally aligned with the units that matter for retrieval or reasoning. This is evident in legal retrieval, where the German Civil Code is already structured into legislatively meaningful sections and subsections. Sentence retrieval is a strong fine-grained baseline, but subsection retrieval achieves about 0.47 Recall@10 and section retrieval about 0.46, both ahead of sentence retrieval. The legal study reports 4 m 57 s build time and 95.7 MB storage for sentence retrieval, compared with 33.2 MB for section retrieval, and argues that sentence chunks can fragment a norm by separating conditions, consequences, and exceptions that are meant to be read together (Prior et al., 19 May 2026).

A similar structural argument appears in enterprise oil and gas RAG. That study does not evaluate sentence chunking as one of its four main methods, but it explicitly criticizes fixed-length segmentation because it often creates breakpoints mid-sentence. The paper’s framing implies that sentence-based chunking would partially remedy linguistic fragmentation, yet still fall short of structure-aware chunking based on headers and hierarchy, which performed best on top- $[S5\text{–}S8]$ 5 metrics in that corpus (Taiwo et al., 25 Mar 2026).

More broadly, evidence from semantic chunking studies cautions against assuming that sentence-aware methods are automatically superior. A systematic comparison of fixed-size, breakpoint-based, and clustering-based chunking concludes that semantic chunking is not consistently better than fixed-size chunking and that its additional preprocessing cost is usually not justified by downstream gains. The authors’ practical recommendation is to use fixed-size chunking unless the corpus has strong topic heterogeneity and there is empirical evidence that naïve boundaries are causing harm (Qu et al., 2024).

The significance of sentence-based chunking is therefore methodological rather than universalist. It is a robust baseline, an interpretable structural prior, and, increasingly, an atomic representation for more adaptive retrieval systems. Its strongest use cases are settings where local linguistic coherence matters and sentence segmentation is reliable. Its weakest use cases are settings where evidence is distributed across procedural bundles, legal provisions, hierarchical sections, or visually and spatially structured artifacts. The accumulated literature suggests that sentence-based chunking should be treated as a principled default or substrate, but not as a domain-independent optimum (Bennani et al., 20 Jan 2026, Chhoun et al., 21 May 2026, Prior et al., 19 May 2026, Lu et al., 15 Sep 2025).