---
title: 'TreeSeg: Hierarchical Segmentation & Interval Queries'
url: https://www.emergentmind.com/topics/treeseg
type: topic
---

# TreeSeg: Hierarchical Segmentation & Interval Queries

TreeSeg refers to distinct concepts in the literature: (1) an algorithm for hierarchical topic segmentation of large transcripts, and (2) a dynamic data structure for efficient segment storage and interval queries (the BITS-Tree). Both rely on tree-based representations to organize, partition, or index sequential data. This article reviews both domains for completeness.

## 1. Hierarchical Topic Segmentation: TreeSeg Algorithm

TreeSeg is an approach for hierarchical, structure-preserving topic segmentation of long, noisy transcripts, notably those generated by Automatic Speech Recognition (ASR) systems. The core objective is to partition a temporally ordered sequence of utterances $U = [U_1, \ldots, U_T]$ into $K$ contiguous, semantically coherent segments $(P_1, \ldots, P_K)$, despite noise, uncertain ground-truth $K$, and large $T$ [2407.12028].

The method addresses three principal challenges: persistent ASR noise, variable and ambiguous segment counts, and computational efficiency for large transcripts. TreeSeg outputs a binary tree segmentation, allowing flexible control of granularity by tree cutting at arbitrary depth.

## 2. Mathematical Formulation and Algorithm

Each utterance $U_t$ is embedded via a windowed approach using a pre-trained, frozen embedding model $f$ (e.g., ADA, SBERT, RoBERTa), computing $e_t = f(B_t) \in \mathbb{R}^d$ for the block $B_t = [U_{\max(1, t-W)}, \ldots, U_t]$ with window width $W$.

TreeSeg employs divisive clustering recursively, searching for split index $i$ within a segment $E_v = [e_s, \ldots, e_e]$ to minimize the within-cluster squared Euclidean distance:

\[
\mathcal{L}_v(i) = \sum_{t=s}^{i-1} \|e_t - \mu_L\|^2_2 + \sum_{t=i}^{e} \|e_t - \mu_R\|^2_2
\]
where
\[
\mu_L = \frac{1}{i - s} \sum_{t=s}^{i-1} e_t, \quad \mu_R = \frac{1}{e - i + 1} \sum_{t=i}^{e} e_t
\]

A minimum segment size $M$ is enforced for split validity ($s+M\leq i \leq e-M+1$). The binary tree is constructed recursively: at each step, the “best” leaf to split and its best split point are selected (via a min-heap over candidate losses), and splitting proceeds until a user-specified number of leaves $K$ is reached, or segments become too small.

This recursive, batch-divisive procedure yields a hierarchical representation, allowing for efficient “zooming” into transcript structure at arbitrary resolutions.

## 3. Noise Robustness, Embedding Model Integration, and Computational Properties

By embedding blocks instead of isolated utterances and averaging in clustering, TreeSeg attenuates the effect of local ASR noise. The model $f$ is not fine-tuned; no additional dimensionality reduction or smoothing is applied except for windowing and averaging. This fosters portability and reproducibility.

The complexity is as follows: for $T$ utterances, $d$-dimensional embeddings, and final $S$ segments,
- Embedding: $O(Td)$
- Precomputation (cumulative sums): $O(Td)$
- All splitting: $O(T\log S)$ (expected, from geometric shrinkage)
- Heap maintenance: $O(S\log S)$
Overall, $O(Td + T\log S + S\log S)$, which is near-linear in $T$ for practical $S \ll T$.

Memory usage is $O(Td)$ for embeddings, $O(T)$ for accumulated statistics, and $O(S)$ for tree nodes and heap.

## 4. Empirical Evaluation and Results

TreeSeg was evaluated on three datasets:
- **ICSI**: 75 meetings with up to 4-level segmentation, mean 1454 utterances
- **AMI**: >100 hours, up to 3 levels, mean 636 utterances
- **TinyRec**: 21 technical sessions, 2-level segmentation, mean 267 utterances

Comparisons included RandomSeg, EquiSeg, HyperSeg (TextTiling-style, hyperdimensional embeddings), and BertSeg (TextTiling-style, BERT blocks). Performance was assessed using $P_k$ and WindowDiff metrics (lower is better).

Aggregate multi-level results:
| Corpus  | Metric         | TreeSeg | Next Best (BertSeg) |
|---------|---------------|---------|---------------------|
| ICSI    | $P_k$         | 0.310   | 0.388               |
|         | WinDiff       | 0.353   | 0.432               |
| AMI     | $P_k$         | 0.355   | 0.443               |
|         | WinDiff       | 0.396   | 0.480               |
| TinyRec | $P_k$         | 0.367   | 0.473               |
|         | WinDiff       | 0.382   | 0.486               |

Per-level scores also favor TreeSeg, e.g., on ICSI level 1: $P_k=0.28$ (TreeSeg) vs $0.343$ (BertSeg), WinDiff $0.314$ vs $0.386$. TreeSeg outperforms all baselines across datasets and at multiple resolutions [2407.12028].

## 5. Limitations and Future Directions in Transcript Segmentation

Current validation is limited to structured meeting corpora (ICSI, AMI); diversity and scale in transcript types remain open for exploration. Only ADA embeddings were tested systematically. Direct evaluations against M³Seg—or more recent hierarchical segmentation baselines—are constrained by code availability.

Proposed future work includes:
- Systematic embedding model comparisons (SBERT, RoBERTa, LLaMA, etc.)
- Extending the hierarchical tree for downstream applications: multi-level summarization, chapter labeling, knowledge extraction
- Application to less-structured, noisier, or conversational transcript domains

## 6. The BITS-Tree Data Structure (TreeSeg in Segment Storage Context)

In the context of dynamic segment storage and interval queries, TreeSeg refers to the BITS-Tree (Balanced Inorder Threaded Segment Tree) [1501.03435]. This structure maintains a height-balanced (AVL) binary search tree in which each node stores a non-overlapping interval and the list of original segments containing that interval.

Key characteristics:
- **Insertion/Deletion**: $O(\log n + k)$, where $n$ is the segment count and $k$ the number of affected nodes (overlapping with the inserted/deleted segment). Segments can extend beyond previous tree bounds.
- **Query**: 
  - Point (stabbing) query: $O(\log n + k)$, with $k$ as output size.
  - Range query: $O(\log n + m)$, $m=$ number of nodes intersecting query.
- **Node count**: At most $2n-1$, much lower than $2(U_{\max}-U_{\min})-1$ for classic dynamic segment trees built over a universe $U$.
- **Space**: Worst-case total segment-list storage $O(n^2)$.
- **Height**: $O(\log n)$.
- **Threading**: Inorder threads enable efficient traversal for range queries.

This structure is particularly advantageous when the global universe $U$ is large and $n$ is comparatively small, minimizing node count while supporting efficient dynamic updates and fast queries [1501.03435].

## 7. Relationship to Other Tree-Based Segmentation Techniques

The term “TreeSeg” is also used in TreeSegNet for image segmentation, notably in adaptive CNNs constructed according to class confusion statistics [1804.10879]. Despite methodologically divergent goals, these approaches share the strategic use of tree structures—either for recursively partitioning data (topic segmentation, BITS-Tree) or as an architectural prior in neural segmentation networks (TreeSegNet).

A plausible implication is that tree-based representations enable scalable, multiresolution partitioning or specialization in domains where hierarchical or ambiguous boundaries are intrinsic to the data.

---

**References:**  
- TreeSeg for hierarchical topic segmentation: [2407.12028]  
- BITS-Tree (dynamic segment storage): [1501.03435]  
- TreeSegNet in adaptive CNN segmentation: [1804.10879]

Source: https://www.emergentmind.com/topics/treeseg