Projected Similarity Chunking (PSC)

• PSC is a learned semantic segmentation method that creates coherent, variable-length text chunks by leveraging domain-trained similarity projections.
• It employs a single-layer projection of sentence embeddings with dot-product similarity and sigmoid thresholding to accurately detect semantic boundaries.
• PSC enhances retrieval and generation quality in RAG systems, offering low latency and robust performance across both in-domain and out-of-domain datasets.

Projected Similarity Chunking (PSC) is a learned semantic segmentation method designed to produce variable-length, contextually coherent text chunks for Retrieval-Augmented Generation (RAG) tasks. Unlike traditional fixed-length or recursive chunkers, which can produce arbitrary or semantically incoherent splits, PSC locates boundaries based on domain-trained similarity projections, yielding improved retrieval and downstream generation performance. Developed and evaluated on biomedical literature, PSC demonstrates strong in-domain and out-of-domain generalization with minimal latency overhead (Allamraju et al., 29 Nov 2025).

1. Rationale and Context

Standard chunking systems in RAG environments often employ fixed-length splitting (e.g., character- or token-based chunkers with overlap) or recursive approaches based on hierarchical structures such as paragraphs or sentences. These methods tend to sever semantic units, resulting in retrieval contexts that may contain partial sentences or unrelated concepts, thus degrading retrieval accuracy and the quality of subsequent generative tasks.

Semantic chunking aims to preserve the implicit coherence found in human-authored sections. PSC advances this goal by leveraging learned boundary detection, explicitly aligning segment breaks with transitions in semantic similarity, as determined by sentence-level embedding projections. This approach mitigates the problem of retrieving noisy, imprecise context common to conventional chunkers (Allamraju et al., 29 Nov 2025).

2. Algorithmic Framework

PSC operates on consecutive sentences $S_1, S_2, \ldots, S_n$ within a document:

• Embedding: Each sentence $S_i$ is encoded as $E_i = \text{Encoder}(S_i) \in \mathbb{R}^{d}$ using one of three pre-trained models: all-MiniLM-L6-v2 (384-dimensional), all-mpnet-base-v2 (768-dimensional), or e5-large-v2 (1,024-dimensional).
• Projection: Embeddings are linearly projected to a chunking-specialized space: $\bar{E}_i = W E_i + b$, where $W \in \mathbb{R}^{k \times d}$ and $b \in \mathbb{R}^{k}$; typically $k = d$.
• Similarity Computation: The dot product is taken between consecutive projected embeddings: $s_i = \bar{E}_i^{\top} \bar{E}_{i+1}$.
• Boundary Probability: The similarity score is passed through a sigmoid: $p_i = \sigma(s_i) = \frac{1}{1 + e^{-s_i}}$.
• Boundary Placement: A boundary is drawn between $S_i$ and $S_{i+1}$ when $p_i < \tau$ (with $\tau = 0.5$ empirically).

This process yields non-overlapping, variable-length segments whose average chunk size is approximately 471 tokens. Computation per sentence pair is limited to a single matrix multiplication and dot product, resulting in a low-latency operation well suited for real-time or large-scale deployment (Allamraju et al., 29 Nov 2025).

3. Training Data and Optimization

PSC is trained using the PubMedQA PQA-A subset, further augmented with full-text articles from PubMed Central. Positive training pairs consist of sentences within the same human-authored section; negatives comprise sentences from distinct sections of the same document, selected such that they never co-occur.

With approximately 93 million labeled sentence pairs (50.3% positive, 49.7% negative), PSC is optimized using binary cross-entropy with logits:

$$L_i = -[y_i \log \sigma(s_i) + (1 - y_i) \log (1 - \sigma(s_i))]$$

where $y_i \in \{0, 1\}$ is the binary label. Training is performed over 5 epochs on an NVIDIA RTX 6000 (48GB), with convergence typically by the fourth epoch. Total training duration is about 72 hours GPU time (Allamraju et al., 29 Nov 2025).

4. Performance and Evaluation

PSC is benchmarked for both retrieval and generation quality in RAG pipelines using PubMedQA and 12 additional RAGBench datasets. Key evaluation results include:

Table 1: Retrieval Metrics (PubMedQA with e5 Encoder)

Chunker	Hits@3	Hits@5	MRR	QueryTime(s)	TTFT(s)
Rec	0.000	0.000	0.000	0.01	0.19
Sem	0.000*	0.000*	0.010	0.01	0.63
MFC	0.12	0.15	0.34	0.01	0.20
PSC	0.13	0.16	0.36	0.01	0.14

PSC yields up to a 24× improvement in MRR compared to recursive chunkers. Two-tailed t-tests show highly significant differences ($p \approx 1 \times 10^{-11}$, $t > 7$, 99.9% confidence) between PSC and semantic chunkers relying solely on cosine similarity.

Table 2: Generation Metrics (PubMedQA: BLEU/ROUGE/BERTScore)

Encoder	Chunker	BLEU	ROUGE-1	ROUGE-2	ROUGE-L	BERTScore
MiniLM	Char	28.10	30.63	13.16	22.26	87.43
MiniLM	PSC	24.20	27.08	10.00	19.45	86.92
mpnet	MFC	23.60	26.61	9.83	19.71	87.38
e5	PSC	23.41	26.52	9.59	18.77	87.42
e5	MFC	25.34	27.68	10.31	19.94	87.46

MFC+e5 slightly outperforms PSC on some generation metrics, but both surpass baseline chunkers. Generation metrics are statistically significant at the 90% level.

Out-of-Domain Generalization

Despite being trained exclusively on biomedical data, PSC (and MFC) generalize robustly to diverse domains such as CovidQA, HotpotQA, and CUAD, with marked improvements over character chunkers. For example, on CovidQA with MiniLM, BLEU rises from 1.95 (Char) to 6.31 (PSC), a +223% increase.

Latency

PSC incurs only a $\sim$0.7s indexing overhead over character chunkers and exhibits lower runtime retrieval and TTFT compared to typical semantic chunkers (0.14s vs. 0.56s) (Allamraju et al., 29 Nov 2025).

5. Comparative Methodological Analysis

PSC employs a dot-product similarity in a learned projection space with a single projection layer, ensuring computation remains lightweight and throughput high. In contrast, Metric Fusion Chunking (MFC) leverages additional metrics (dot-product, Euclidean, and Manhattan distances) via an extra neural layer, gaining some representational richness with corresponding (~small) computational overhead.

PSC significantly outperforms off-the-shelf semantic chunkers based on cosine similarity (e.g., LangChain SemanticChunker), indicating that domain-trained projection yields greater sensitivity to semantic boundary shifts than default metrics. Across encoder ablations, e5 yields the strongest absolute results, but PSC consistently improves retrieval and generation relative to baseline across all models.

6. Deployment and Limitations

PSC is directly deployable in any RAG pipeline with negligible code and runtime changes. Training requires moderate computational resources (one RTX 6000 GPU, 72h), while inference entails minimal overhead (one linear + dot-product per sentence pair).

Domain bias may affect optimal boundary detection; PSC is trained on biomedical structure, but empirical results exhibit generalization. For radically different discourse structures (e.g., dialogue transcripts), retraining PSC on relevant section boundaries is recommended.

7. Implications and Future Prospects

PSC offers a learned, semantic chunking mechanism that aligns chunk boundaries with section-level coherence, improving retrieval metrics (MRR up to 0.36) and generation quality with low latency. The single-layer projection design demonstrates that semantic chunking need not be computationally expensive to yield substantial gains. A plausible implication is that further advances may arise from adaptive chunking protocols and explorations of domain-adaptive boundary learning, as suggested in (Allamraju et al., 29 Nov 2025).