---
title: 'HoliTracer: Direct Vectorization in RSI'
url: https://www.emergentmind.com/topics/holitracer
type: topic
---

# HoliTracer: Direct Vectorization in RSI

HoliTracer is a framework for direct vectorization of large-size remote sensing imagery (RSI), introduced to holistically extract vectorized geographic objects from large-size RSI without patch stitching. It targets buildings, water bodies, and roads in very-high-resolution imagery whose spatial extent commonly exceeds \(10{,}000\times 10{,}000\) pixels, where patch-based processing can discard global scene context and produce fragmented vector outputs at patch boundaries. The framework combines a context-aware segmentation module, termed Context Attention Net (CAN), with a two-stage vectorization pipeline composed of the Mask Contour Reformer (MCR) and the Polygon Sequence Tracer (PST) [2507.16251].

## 1. Problem setting and design rationale

HoliTracer is motivated by the observation that modern very-high-resolution RSIs easily exceed \(10{,}000\times 10{,}000\) pixels, whereas most existing vectorization methods are constrained to small patches such as \(512\times 512\). In the motivating formulation, patch-based cropping discards global scene context, which can lead to misclassification of objects that require large-scale cues, such as confusing building rooftops with parking lots, and also yields fragmented vector outputs at patch boundaries. The problem is compounded by the fact that buildings, water bodies, and roads vary drastically in scale and shape, complicating a unified representation across categories [2507.16251].

The framework is organized around four stated contributions. First, it provides a unified framework to directly vectorize entire large-size RSIs without patch stitching. Second, CAN employs a local-to-global attention mechanism on a multi-scale image pyramid to recover lost context and improve segmentation completeness. Third, vectorization is handled by a robust two-stage pipeline in which MCR simplifies and uniformly reconstructs polygonal contours from segmentation masks, while PST refines point positions and predicts true vertices through iterative offset regression and transformer-based classification with geometric angle features. Fourth, the method is reported to outperform state-of-the-art patch-based and large-scale segmentation baselines on building, water body, and road datasets [2507.16251].

A plausible implication is that HoliTracer is best understood not as a pure contour extraction method, but as an end-to-end large-scene mapping pipeline in which segmentation quality and polygon tracing are co-dependent stages.

## 2. Context Attention Net for large-scene segmentation

CAN addresses the contextual deficiencies of patch-limited processing through a multi-scale image pyramid. Given an input RSI \(I\) of size \(>10{,}000\times 10{,}000\), the method constructs downsampled versions with rates \(d_1=1<d_2<\dots<d_n\), with examples \(\{1,3,6\}\) or \(\{1,5,10\}\). A fixed-size window, such as \(512\times 512\), is slid across each scale to yield aligned patch sets \(\{I_1,I_2,\dots,I_n\}\), where \(I_1\) captures fine detail and higher-scale inputs provide increasingly global context [2507.16251].

Feature extraction is performed with a Swin-L transformer pretrained with Skysense weights. Each scale is encoded as
\[
F_k = \mathrm{Encoder}(I_k),
\]
and the upper-scale encoders share weights with the bottom scale. Cross-scale fusion is then performed by local-to-global attention:
\[
Q = F_1,\quad K_k = F_k,\quad V_k = F_k,
\]
\[
F_k' = \mathrm{Softmax}(QK_k^\top/\sqrt d)\,V_k.
\]
The fused representation is
\[
F = \mathrm{Concat}[F_1, F_2', \dots, F_n'].
\]
This fused tensor is decoded by a UPerNet segmentation head into per-pixel class logits, optimized with cross-entropy loss
\[
L_{\mathrm{seg}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C} y_{i,c}\log \hat y_{i,c},
\]
where \(N\) is the number of pixels and \(C\) the number of classes [2507.16251].

The architectural significance of CAN lies in the explicit separation between local detail and large-scale semantic support. The reported qualitative analysis states that holistic segmentation with CAN recovers rooftop and building context and avoids misclassification of parking areas, which suggests that the model’s central contribution is not merely scale augmentation, but aligned cross-scale contextual conditioning over the entire large tile [2507.16251].

## 3. Mask Contour Reformer and polygon reconstruction

MCR converts segmentation masks into polygon sequences suitable for supervised vertex tracing. Its pipeline begins by extracting a raw contour \(S=\{s_1,\dots,s_m\}\) from the binary segmentation mask using TC89-KCOS. The contour is then simplified with Douglas–Peucker, using tolerance \(\epsilon\), to obtain \(S'=[s_1',\dots,s_p']\). From this simplified contour, a uniformly spaced point sequence \(R=[r_1,\dots,r_N]\) is reconstructed by interpolating along each edge \((s_i' \rightarrow s_{i+1}')\) every \(l\) pixels, with
\[
K_i=\left\lfloor \|s_{i+1}'-s_i'\|/l \right\rfloor,
\]
and \(K_i\) inserted points on that segment [2507.16251].

During training, this reconstructed sequence is aligned with the ground-truth polygon \(G=[g_1,\dots,g_M]\). For each \(g_j\), the nearest reconstructed point is selected,
\[
r_{k_j}=\arg\min_r \|r-g_j\|,
\]
matched points are marked as vertices with \(c_{k_j}=1\), and the remaining points receive \(c_i=0\). Between consecutive matched vertices, linear interpolation produces \(G'\) with \(|G'|=|R|\), yielding training pairs \((R,G',C)\). The key correspondence constraint is
\[
|G'| = M + \sum_{k=1}^{M}(i_{k+1}-i_k-1)=N=|R|,
\]
which guarantees a one-to-one correspondence for supervised vertex learning [2507.16251].

This stage performs two distinct functions. First, it suppresses the irregularity of raw mask contours by simplification. Second, it imposes a uniform sampling structure that makes subsequent sequence learning well-posed. The reported qualitative analysis states that MCR yields clean, uniformly sampled contours and prevents boundary fragmentation across large tiles, which indicates that its role is as much representational as geometric [2507.16251].

## 4. Polygon Sequence Tracer and vertex inference

PST takes the reconstructed polygon \(R^{(0)}=[r_1,\dots,r_n]\) and performs point refinement followed by vertex classification. For each point \(r_i\), local image features \(z_i\) are extracted from the frozen encoder feature map, and tokens of the form \([z_i; x_i; y_i]\) are passed into a transformer-based offset regressor that predicts offsets \(\hat o_i=(\Delta x_i,\Delta y_i)\). Refinement is iterative:
\[
R^{(k)} = R^{(k-1)} + \hat O^{(k)},\qquad \hat O^{(k)}=[\hat o_1^{(k)},\dots,\hat o_n^{(k)}].
\]
After the final refinement step, the refined sequence \(R'\) is used to compute corner angles for \(s=1,2,3\):
\[
\theta_i^{(s)} = \arccos\left(\frac{v_{i-s,i}\cdot v_{i,i+s}}{\|v_{i-s,i}\|\,\|v_{i,i+s}\|}\right),
\]
where \(v_{a,b}=r_b'-r_a'\) with cyclic indexing. The angle tuple \([\theta_i^{(1)},\theta_i^{(2)},\theta_i^{(3)}]\) is inserted as a geometric token, and the concatenated descriptor is fed to a transformer classifier producing \(\hat p_i\in[0,1]\) [2507.16251].

Training combines three losses. Offset regression uses Smooth L1,
\[
L_{\mathrm{off}}=\sum_i \mathrm{SmoothL1}(\hat o_i-o_i),\qquad o_i=g_i'-r_i.
\]
Vertex classification uses binary cross-entropy,
\[
L_{\mathrm{vert}}=-\sum_i \left[c_i\log \hat p_i + (1-c_i)\log(1-\hat p_i)\right].
\]
An angle penalty encourages \(\hat p_i\to 1\) when \(\theta_i^{(1)}<\theta_{\mathrm{th}}\) and \(\hat p_i\to 0\) otherwise:
\[
L_{\mathrm{angle}}=\sum_i
\begin{cases}
\max(0,\theta_i^{(1)}-\theta_{\mathrm{th}}), & c_i=1\\
\max(0,\theta_{\mathrm{th}}-\theta_i^{(1)}), & \text{otherwise.}
\end{cases}
\]
The total loss is
\[
L=\lambda_1L_{\mathrm{off}}+\lambda_2L_{\mathrm{vert}}+\lambda_3L_{\mathrm{angle}},
\]
with \(\lambda_1=\lambda_2=\lambda_3=1\) and \(\theta_{\mathrm{th}}=135^\circ\) in the reported implementation [2507.16251].

The PST formulation makes a specific claim about polygon vectorization: vertices are not treated as direct contour samples, but as latent salient points inferred from refined geometry and angle structure. This suggests a distinction between contour regularization and semantic vertex recovery, with the latter delegated to transformer sequence modeling.

## 5. Datasets, implementation, and quantitative evaluation

The reported experiments use three large-size RSI datasets. WHU-building contains 400 large tiles of size \(10{,}000\times 10{,}000\) at \(0.075\) m resolution, split \(320/40/40\) for train/validation/test. GLH-water contains 250 tiles of size \(12{,}800\times 12{,}800\) at \(0.3\) m, split \(200/25/25\). VHR-road contains 208 tiles of size \(12{,}500\times 12{,}500\) at \(0.2\) m, split \(166/21/21\). Multi-scale pyramids use \(\{1,3,6\}\) for buildings and \(\{1,5,10\}\) for water and roads. MCR uses \(\epsilon=5\), with interpolation \(l=25\) for buildings and \(l=50\) for water and roads. Optimization uses Adam with learning rate \(10^{-4}\) for CAN and \(10^{-2}\) for PST, with training in PyTorch on four NVIDIA A100 GPUs [2507.16251].

The principal quantitative comparisons are summarized below.

| Dataset | Baseline | HoliTracer |
|---|---:|---:|
| WHU-building | HiSup: PoLiS 5.49, C-IoU 42.23, AP 56.77, IoU 85.79, F1 92.21 | PoLiS 3.63, C-IoU 82.30, AP 61.07, IoU 91.60, F1 95.41 |
| GLH-water | DeepSnake: PoLiS 115.51, C-IoU 43.53, AP 2.54, IoU 64.19, F1 75.42 | PoLiS 81.87, C-IoU 59.24, AP 20.84, IoU 85.68, F1 91.51 |
| VHR-road | HiSup: PoLiS 299.03, C-IoU 3.43, AP 1.51, IoU 36.40, F1 49.77 | PoLiS 134.13, C-IoU 6.10, AP 1.58, IoU 46.48, F1 60.63 |

On WHU-building, the reported absolute gain is PoLiS \(-1.86\), C-IoU \(+40.07\%\), AP \(+4.30\) points, and IoU \(+5.81\) points. Visual comparisons are stated to confirm that HoliTracer produces more connected and accurate polygons for large buildings, continuous water bodies, and complex roads [2507.16251].

These results emphasize that the gains are not limited to mask overlap metrics such as IoU and F1. The largest reported improvements occur in polygon-sensitive measures such as C-IoU and PoLiS, which suggests that the method’s main benefit lies in vector quality rather than in segmentation quality alone.

## 6. Ablations, failure modes, and implications

The ablation studies isolate contributions from CAN, PST, and geometric angle modeling. For CAN on WHU-building, the best reported image pyramid is \(\{1,3,6\}\), with IoU \(92.21\) and F1 \(95.94\), compared with IoU \(91.23\) and F1 \(95.29\) for the single-scale setting \(\{1\}\). For the vectorization stage on WHU-building, CAN+TC89-KCOS yields PoLiS \(3.83\), C-IoU \(18.47\), AP \(58.75\), IoU \(91.55\), and F1 \(95.54\); adding Douglas–Peucker gives PoLiS \(4.02\), C-IoU \(60.32\), AP \(58.42\), IoU \(91.50\), and F1 \(95.52\); adding PST yields PoLiS \(3.63\), C-IoU \(82.30\), AP \(61.07\), IoU \(91.60\), and F1 \(95.41\). For angle modeling, the best result is obtained with angle features and \(\theta_{\mathrm{th}}=135^\circ\), giving AP \(61.07\), C-IoU \(82.30\), PoLiS \(3.63\), and IoU \(91.60\) [2507.16251].

The qualitative analysis reports several characteristic effects. CAN recovers large-scene context. MCR produces clean and uniformly sampled contours. PST refines vertices to produce smooth, well-aligned polygons on irregular shapes such as water shorelines and road junctions. At the same time, the reported failure modes state that extremely dense road networks still pose connectivity challenges and that very elongated thin structures may require finer interpolation [2507.16251].

A common simplifying assumption in large-image mapping is that patch stitching is primarily an engineering inconvenience rather than a representational problem. The HoliTracer results argue against that assumption: the strongest gains occur precisely where global context and boundary continuity matter most. The reported future directions—real-time acceleration, integration of height or semantic attributes, multi-class simultaneous mapping including vegetation and bridges, fusion with LiDAR or multi-spectral data, and deployment in GIS and urban planning pipelines for automated map updating and change detection—suggest that the framework is intended as a general large-scene vector mapping substrate rather than a task-specific polygonizer [2507.16251].

Source: https://www.emergentmind.com/topics/holitracer