BoundRL: Reinforced Structured Text Segmentation

• BoundRL is a framework for structured text segmentation that generates only boundary signals and labels, reducing the output length by focusing on compact segmentation representation.
• It employs reinforcement learning with verifiable rewards to jointly predict segment labels and boundary cues, enhancing reconstruction fidelity and semantic alignment.
• The method outperforms conventional SFT approaches on heterogeneous documents such as prompts, code blocks, and tables, ensuring lossless text recovery.

BoundRL is a framework for structured text segmentation that reformulates segmentation as reinforced boundary generation rather than full-span generation. It jointly predicts segment labels and short boundary cues for long structured documents, then reconstructs the corresponding spans from the source text itself. The method is designed for documents whose structure is not well captured by sentence- or paragraph-level segmentation, including prompts, code blocks, placeholders, tables, and other noncanonical formats. In the formulation introduced in “BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generation,” the central gain is that the model generates only boundary signals and labels, which reduces output length, lowers hallucination risk, and makes reinforcement learning with verifiable rewards directly applicable to segmentation quality and document recoverability (Li et al., 23 Oct 2025).

1. Task formulation and target domain

BoundRL addresses structured text segmentation: given a long structured document $d$, the goal is to split it into a sequence of contiguous, non-overlapping segments and assign a semantic label to each segment. The task is formalized as

$$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$

Here $t_i$ is the text span of segment $i$, $l_i \in L$ is its label, and $L$ is the label set. The paper emphasizes that standard sentence- or paragraph-level segmentation is a poor fit for modern structured texts, because such texts often contain code blocks, tables, JSON/XML, placeholders such as {context} or {question}, prompt templates, and noncanonical formatting (Li et al., 23 Oct 2025).

The work uses complex prompts for LLM applications as its principal evaluation domain. In StructSeg, the target labels are instruction, example, context, question, and output format. Human annotation is defined to be mutually exclusive, non-overlapping, and lossless, so concatenating segments in order exactly reconstructs the original prompt. This lossless reconstruction requirement is central to the method’s later reward design and to its rejection of token-level NER-style segmentation as a primary formulation (Li et al., 23 Oct 2025).

A recurrent misconception is to treat BoundRL as a conventional sequence-labeling system. The paper argues against that interpretation on two grounds. Token-level sequence labeling tends to produce overly fragmented segments, while token-level boundary classification requires too many decisions. BoundRL instead treats segmentation as structured autoregressive generation over a compact representation of segment starts and labels, which preserves token-level flexibility without committing to tokenwise supervision (Li et al., 23 Oct 2025).

2. Boundary generation and reconstruction

The defining design choice of BoundRL is to generate only segment boundaries, not segment contents. The model outputs a sequence of label/start-token pairs

$[(\hat{l}_i,\hat{s}_i)_{i=1:n}]=LLM(d),$

where $\hat l_i$ is the predicted label and $\hat s_i$ is a short sequence of starting tokens for segment $i$. The complete segment texts are then reconstructed by locating these starts in the source document rather than regenerating the spans autoregressively (Li et al., 23 Oct 2025).

Reconstruction follows an ordered matching heuristic. The first start $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$0 is located as its leftmost occurrence in $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$1. Each later $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$2 is located as the leftmost occurrence after the previous segment position, preserving order. Segment $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$3 is reconstructed as the span between the position of $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$4 and the position of $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$5, and the final segment extends to the end of the document. If either $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$6 or $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$7 cannot be found, segment $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$8 is discarded. During training, gold segment texts $$[(l_1,t_1),...,(l_n,t_n)]=f(d),~s.t.~l_i \neq l_{i-1}, t_i\cap t_{i-1}=\emptyset,~\forall i=2,...,n.$$9 are converted into start-token sequences $t_i$0, and the sequence lengths are randomly sampled while ensuring uniqueness among segment starts within the same document (Li et al., 23 Oct 2025).

This output formulation changes the complexity of generation. The paper summarizes the shift as reducing output from $t_i$1 generated tokens to $t_i$2, where $t_i$3 is document length and $t_i$4 is the number of segments. On the Synthetic subset, full-text prompting baselines require 1,170 output tokens per prompt on average, whereas BoundRL requires 119 output tokens, a 90% reduction. The paper also compares three output formats—start, end, and start+end—and reports that start performs best overall, especially on exact match (Li et al., 23 Oct 2025).

Architecturally, BoundRL does not introduce a new neural backbone. It uses standard decoder-only LLMs as the policy/model, and casts segmentation as autoregressive generation over the serialized sequence of segment descriptors. This suggests that the novelty lies in output representation, reward design, and RL training strategy rather than in new attention or encoder-decoder machinery.

3. Reinforcement learning with verifiable rewards

BoundRL applies supervised fine-tuning first and then reinforcement learning with verifiable rewards. The paper argues that SFT alone is suboptimal for this output space for two stated reasons: equivalent boundary realizations can be penalized, and small token mismatches can be catastrophic if a predicted start string cannot be localized in the document. Token-level cross-entropy therefore does not align well with the actual objective of reconstructable and semantically correct segmentation (Li et al., 23 Oct 2025).

The RL stage uses GRPO, with the SFT-tuned LLM as policy, $t_i$5 sampled candidate segmentations per input, deterministic reward computation from source text and annotations, and no standard deviation-based reward scaling. Rollout temperature is 1.2 by default, and inference temperature at test time is 0. The reward combines reconstruction fidelity with semantic alignment. The reconstruction ratio is

$t_i$6

Semantic alignment is measured by exact-match F1,

$t_i$7

and character-level F1,

$t_i$8

The final reward is

$t_i$9

The reward is “verifiable” because it is computed mechanically from the document and gold annotations, without a learned reward model or preference model (Li et al., 23 Oct 2025).

The reward’s multiplicative structure is significant. Reconstruction ratio suppresses candidates whose boundaries do not recover the source text, while the averaged exact-match and character-level terms differentiate between structurally plausible but semantically misaligned segmentations. This suggests that BoundRL’s RL stage optimizes not only span accuracy but also the recoverability constraint that is built into the reconstruction mechanism.

4. Intermediate candidates and entropy-collapse mitigation

A major training issue identified in the paper is entropy collapse: rollout generations become concentrated in a narrow, low-reward region, reducing exploration and stalling learning. BoundRL addresses this with intermediate candidates rather than with direct injection of gold outputs. The paper states that gold/reference outputs are often too off-policy, whereas perturbed model outputs can function as stepping stones toward higher-quality solutions (Li et al., 23 Oct 2025).

For each input $i$0, the model samples $i$1 candidate segmentations

$i$2

These are sorted by descending reward, and the medium-reward candidate

$i$3

is perturbed. The paper defines three perturbation types: shorten the segment text by truncating one word from either side, extend the segment text by including one additional word from either side, or change the label to another label in $i$4, excluding labels already assigned to neighboring segments. One perturbation is applied at a time, each candidate differs by exactly one perturbation, and the perturbed candidate with highest reward is selected: $i$5 Replacement is performed only when the reward gain is positive,

$i$6

and for at most $i$7 inputs per batch (Li et al., 23 Oct 2025).

Appendix details make the perturbations operational at the boundary-token level. To shorten a segment on the left by one word, the first word of $i$8 is truncated; to extend on the left, the word immediately before $i$9 is prepended; analogous edits are applied to $l_i \in L$0 for right-boundary changes. Segments of one word are not shortened or extended, and neighboring start sequences are adjusted if overlaps arise. Ablations show that two-step perturbations, replacing all beneficial candidates rather than top-$l_i \in L$1, or perturbing a random candidate rather than the medium-reward candidate all underperform the full design. The appendix also reports reward-standard-deviation curves in which BoundRL maintains healthier reward variance during training while plain SFT+RLVR collapses quickly (Li et al., 23 Oct 2025).

5. Dataset, training configuration, and empirical results

Evaluation is conducted on StructSeg, a benchmark for structured prompt segmentation comprising Synthetic prompts generated with Claude 3.5 Sonnet and real-world prompts collected from LangSmith / LangChain Hub. The dataset contains 15.3K annotations. The Synthetic subset has 15,132 prompts, average 900 tokens, and average 6.1 segments; Langchain has 197 prompts, average 914 tokens, and average 7.6 segments. The Synthetic split is 14,732 train / 200 validation / 200 test, while all 197 Langchain prompts are used only for testing. The evaluated models are Qwen3-1.7b, Qwen3-4b, and Llama-3.1-8b-Instruct (Li et al., 23 Oct 2025).

Training is two-stage. SFT uses 1 epoch, batch size 16, and learning rates $l_i \in L$2 for Qwen3 and $l_i \in L$3 for Llama-3.1-8b. RLVR is run on a random 25% subset of the training data with batch size 6 input documents, $l_i \in L$4 sampled candidates per input, GRPO without standard deviation-based reward scaling, learning rates $l_i \in L$5 for Qwen3 and $l_i \in L$6 for Llama-3.1, selective replacement threshold $l_i \in L$7 for Qwen3 and $l_i \in L$8 for Llama-3.1, max gradient norm 0.1, weight decay 0.01, linear scheduler, warmup ratio 0.03, checkpointing every 0.2 epochs, and rollout truncation at the first end-of-response token (Li et al., 23 Oct 2025).

The main average-score comparisons reported in the paper are as follows:

Model	SFT	SFT+RLVR	BoundRL
Qwen3-1.7b	81.1	83.9	84.5
Qwen3-4b	83.5	86.0	86.6
Llama-3.1-8b	82.5	84.3	84.8

The paper states that BoundRL versus SFT+RLVR is statistically significant by paired t-test, $l_i \in L$9, and that SFT+RLVR versus SFT with 2 epochs is also significant, $L$0. It also reports that the biggest gains appear on Langchain, with 5–11% absolute improvements in exact match from RLVR over SFT on this out-of-domain setting. A notable headline comparison is that Qwen3-1.7b with BoundRL reaches an average score of 84.5, whereas Claude4-Sonnet full and Claude4-Sonnet start each score 68.8. The NER baseline achieves high $L$1 but poor EM, $L$2, and label F1, which the paper attributes to many short fragmented segments (Li et al., 23 Oct 2025).

6. Interpretation, scope, and limitations

BoundRL’s strongest practical properties are efficiency, lower hallucination risk, and joint segmentation-label prediction. Because segment contents are reconstructed from the source text rather than generated token by token, the method is especially suited to long, structurally heterogeneous documents where source-text availability and exact span recovery matter. The paper explicitly identifies prompt segmentation as such a setting, but the formulation is broader and applies to structured texts containing code, placeholders, and other mixed-format components (Li et al., 23 Oct 2025).

Several limitations are also explicit. Reconstruction depends on recoverable start strings; if a predicted start-token sequence cannot be located, the corresponding segment is dropped. Repeated prefixes can introduce ambiguity because reconstruction uses the leftmost valid occurrence heuristic. The method is sensitive to tokenization or string-form choices in how starts are represented and matched. The experiments use a fixed five-label taxonomy, so transfer to substantially different domains may require new annotations and modified task instructions. The current formulation is flat rather than hierarchical, and training requires annotated segmented data (Li et al., 23 Oct 2025).

These constraints help delimit what BoundRL is and is not. It is not a full-span generator, and it is not primarily a token-labeling system. It is also not a generic reinforcement-learning algorithm in the control sense; the “RL” component is RLVR over segmentation outputs. The paper argues that the method is preferable when documents are long, structurally heterogeneous, expensive to generate in full, and available for exact recovery. It is less attractive when exact source-text reconstruction is impossible or unnecessary, when boundaries cannot be reliably anchored by unique start strings, or when the task requires deeply nested hierarchical segmentation (Li et al., 23 Oct 2025).