D2Dewarp: Dual Dimensions Document Dewarping

• D2Dewarp is a document dewarping model that restores a flat, fronto-parallel view by jointly modeling horizontal and vertical line structures.
• It employs a dual-decoder UNet to segment geometric features, then uses a coordinate-aware HV Fusion Module to predict a fine-grained 2D deformation field.
• Experimental benchmarks show that integrating both text and non-text boundaries significantly enhances OCR accuracy over traditional horizontal-only approaches.

D2Dewarp, short for Dual Dimensions Geometric Representation Learning Based Document Dewarping, is an end-to-end document rectification model that explicitly models horizontal and vertical line structures and fuses them through an X/Y coordinate-aware module to produce a fine-grained 2D deformation field for document dewarping. It was introduced to address a recurrent limitation of text-line-aware dewarping systems that focus predominantly on a single horizontal dimension, thereby overlooking vertical structures and the boundaries of figures, tables, and paragraphs (Li et al., 11 Jul 2025).

1. Definition and problem setting

D2Dewarp targets the recovery of a fronto-parallel, flat view of a camera-captured document affected by perspective distortion, page curvature, ripples, folds, and complex background interference. The intended rectified image is geometrically aligned to its scanned counterpart and optimized for readability and downstream OCR. The central claim of the method is that horizontal-only constraints are insufficient when vertical deformation is strong, or when non-text regions dominate; accordingly, the model jointly represents horizontal and vertical line features and learns their mutual constraints, complementarity, and cross-dimensional interactions (Li et al., 11 Jul 2025).

The line notion used by D2Dewarp is broader than text baselines alone. Its fine-grained deformation perception includes top/bottom and left/right boundaries of the document foreground, tables, figures, and paragraphs, plus dense text lines within paragraphs. This broadening of the geometric signal is a defining feature: the model treats document dewarping as a problem of recovering distortion trends in two dimensions rather than in one (Li et al., 11 Jul 2025).

The output representation is a dense backward deformation field. If the predicted field is denoted by $\hat{G}$, then the rectified coordinate for a pixel $p=(x,y)$ is written as

$p' = T(p) = p + W(p),$

and image sampling is performed with standard differentiable bilinear grid sampling:

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$

In practice, the network outputs the backward map used by grid_sample to construct the rectified image (Li et al., 11 Jul 2025).

2. Architecture and geometric representation

The overall pipeline first segments horizontal and vertical line masks using a dual-decoder UNet-like segmentation network, then extracts horizontal- and vertical-aware feature maps $F_h$ and $F_v$, and finally fuses them with an HV Fusion Module to predict the dense deformation field. During training, the input is

$I \in R^{H\times W\times 3}, \quad H=W=448,$

and the outputs are (i) Horizontal line mask; (ii) Vertical line mask; (iii) Dense backward deformation field $\hat{G}$ (2 channels) (Li et al., 11 Jul 2025).

The backbone is UNet-style with 5 blocks: ConvBNReLU×2, max-pool for downsampling. The encoder produces feature maps

$\{E_1,\ldots,E_5\}$

with spatial sizes

$\{448,224,112,56,28\}$

and channels

$p=(x,y)$0

To capture long-range dependencies, $p=(x,y)$1 self-attention layers follow the encoder. The architecture then branches into two decoders (horizontal and vertical) that share the encoder. Each decoder has 4 upsampling blocks (bilinear resize + ConvBNReLU×2) and produces intermediate line predictions via 1×1 conv followed by bilinear upsampling to input size. Decoder features

$p=(x,y)$2

have spatial sizes

$p=(x,y)$3

and channels

$p=(x,y)$4

and are resized to $p=(x,y)$5 and concatenated across scale to yield

$p=(x,y)$6

These line-aware tensors are the direct substrate for the deformation estimation stage (Li et al., 11 Jul 2025).

The method’s representation learning objective is thus not merely to regress a flow field from raw RGB content. It first constructs direction-specific geometric descriptors, then constrains them through a dedicated fusion mechanism. This suggests a closer alignment between the intermediate representation and the eventual deformation field than in pipelines where geometry is only implicit.

3. HV Fusion Module and optimization objective

The HV Fusion Module performs coordinate-aware pooling along X and Y, then applies mixed attention, followed by direction-specific self-attention and gating. For a feature map $p=(x,y)$7, the coordinate descriptors are

$p=(x,y)$8

Applied to $p=(x,y)$9 and $p' = T(p) = p + W(p),$0, these yield $p' = T(p) = p + W(p),$1. Cross-direction mixing is then constructed as

$p' = T(p) = p + W(p),$2

After mixed attention and channel splitting, same-direction components are concatenated, passed through XSelfAttention or YSelfAttention, and transformed into gates:

$p' = T(p) = p + W(p),$3

$p' = T(p) = p + W(p),$4

The original features are reweighted multiplicatively:

$p' = T(p) = p + W(p),$5

Finally, $p' = T(p) = p + W(p),$6 and $p' = T(p) = p + W(p),$7 are concatenated channel-wise and passed to an upsampling head, following DocTr/DocGeoNet/FTDR design, to predict $p' = T(p) = p + W(p),$8 (Li et al., 11 Jul 2025).

The training objective combines line segmentation losses and a rectification loss. The binary cross-entropy losses for horizontal and vertical masks are

$p' = T(p) = p + W(p),$9

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$0

Weighted $I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$1 line losses are also used:

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$2

With multi-scale aggregation over decoder layers, the paper writes

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$3

The rectification term is

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$4

and the total objective is

$I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$5

The paper explicitly notes that D2Dewarp does not introduce explicit smoothness, curvature, or cross-term regularizers; straightness and consistency are instead learned via line masks and attention-based fusion (Li et al., 11 Jul 2025).

4. Automatic annotation and the DocDewarpHV dataset

Because public dewarping datasets do not provide the horizontal/vertical line annotations required by the model, D2Dewarp introduces an automatic fine-grained annotation method and a new large-scale training dataset named DocDewarpHV. The rendering engine is Blender (bpy). English document textures come from the PubLayNet development set, and Chinese document textures come from CDLA, CDDOD, and M6Doc test sets. PaddleOCR is used for text-line detection (Li et al., 11 Jul 2025).

The annotation procedure defines:

• Horizontal lines: top/bottom boundaries of the page foreground, figures, tables, paragraphs, and text lines within paragraphs.
• Vertical lines: left/right boundaries of the page foreground, figures, tables, paragraphs, and text lines within paragraphs.

These masks are merged with OCR-derived text-line coordinates and rasterized as binary masks. Distortion synthesis follows DewarpNet, DocMAE, and LA-DocFlatten: Blender simulates 3D document surfaces with varying shape (bends, waves), camera viewpoints, lighting, and complex backgrounds. The original scanned texture is mapped to the 3D surface through the UV map, rendered into a warped image, and the same warp is applied to the horizontal and vertical masks. The pipeline also exports ground-truth UV map and 3D coordinates (Li et al., 11 Jul 2025).

DocDewarpHV contains ~114,385 warped images at 512×512 resolution, with paired 3D coordinates, UV maps, and dual line masks. It is positioned as a complement to Doc3D (102,027 images at 448×448) by adding horizontal/vertical line annotations and Chinese content. The reported labels are horizontal/vertical line masks (binary), 3D coordinates, UV map, and backward deformation field $I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$6 derived from the UV or the known rendering pipeline (Li et al., 11 Jul 2025).

Training uses images that are cropped then resized to 448×448, with AdamW, initial max learning rate $I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$7, cosine decay to $I'(x,y) = I(x + \Delta x(x,y), y + \Delta y(x,y)).$8, warm-up 10,000 steps, batch size 28, and 80 epochs. The implementation is in PyTorch; quantitative metrics are computed in Matlab R2019a. OCR evaluation uses Tesseract v5.0.1 (pytesseract 0.3.8) for English datasets and PaddleOCR for DocReal (Li et al., 11 Jul 2025).

5. Benchmarks, ablations, and empirical profile

D2Dewarp is evaluated with MS-SSIM, LD, AD, ED, and CER on DocUNet, DIR300, and DocReal. On DocUNet (130 real images), the reported results are:

• MS-SSIM 0.50
• LD 7.71
• AD 0.349
• ED 656.30/351.08 (50/60 images)
• CER 0.1543/0.1338

The paper describes these as excellent OCR improvements over textline- and layout-aware baselines, while also noting slightly less advantage on MS-SSIM/LD due to many figure/table boundaries that are harder than text lines. On DIR300 (300 real English documents), D2Dewarp reports MS-SSIM 0.65; LD 5.73; AD 0.186; ED 466.94; CER 0.1676. On DocReal (200 real Chinese documents), it reports MS-SSIM 0.58; LD 8.69; AD 0.227; ED 191.25; CER 0.2588 (Li et al., 11 Jul 2025).

The ablation studies isolate the HV Fusion Module. On DocUNet, adding the fusion module reduces ED from 401.08 to 351.08 and CER from 0.1439 to 0.1338. On DIR300, it reduces ED 509.82 to 466.94 and CER 0.1682 to 0.1676. On DocReal, it reduces ED 199.72 to 191.25 and CER 0.2665 to 0.2588. The paper interprets this as evidence that the fusion module consistently improves readability (OCR metrics) with small fluctuations in LD/AD, thereby confirming its role in fine-grained deformation perception (Li et al., 11 Jul 2025).

A second ablation compares horizontal-only, vertical-only, and horizontal+vertical modeling on DocReal:

• H-only: MS-SSIM 0.58, LD/AD 9.01/0.253, ED 197.98, CER 0.267
• V-only: MS-SSIM 0.57, LD/AD 9.00/0.243, ED 201.12, CER 0.265
• H+V: MS-SSIM 0.58, LD/AD 8.69/0.227, ED 191.25, CER 0.259

The reported conclusion is that both dimensions contribute, and that joint modeling yields complementary gains (Li et al., 11 Jul 2025).

6. Position within document dewarping research and limitations

D2Dewarp is situated against several adjacent dewarping paradigms. Relative to horizontal-only text-line aware methods, including those summarized in the paper as RDGR, DocGeoNet, and FTDR, it explicitly models both horizontal and vertical lines, including non-text boundaries (tables/figures/paragraphs), and fuses them via coordinate-aware attention and gating (Li et al., 11 Jul 2025). This places it close to a broader movement in document dewarping that uses structural constraints rather than relying only on dense regression.

That broader movement includes methods such as “Revisiting Document Image Dewarping by Grid Regularization” (Jiang et al., 2022), which learns boundary points and the pixels in the text lines and then solves a constrained optimization over a UV grid so that boundaries and lines become straight and axis-aligned. It also includes “MataDoc” (Dai et al., 2023), which targets arbitrary/incomplete boundary scenarios through margin-aware background consistency and text-aware word-position consistency, and “Axis-Aligned Document Dewarping” (Wang et al., 20 Jul 2025), which imposes an axis-aligned geometric constraint in UV space and introduces the Axis-Aligned Distortion (AAD) metric. Compared with these methods, D2Dewarp’s distinctive emphasis is the dual-dimension segmentation-and-fusion design rather than boundary completion, quadratic-program grid optimization, or axis-alignment variance minimization.

The paper also states several limitations. Boundary ambiguity can degrade performance when document foreground boundaries are unclear, for example when the camera is within the page or the edges have low contrast. Background text interference can cause the model to misinterpret prominent background text as document lines. Language-specific line-height differences can weaken fixed pooling. The method further assumes the presence of line structure; heavily non-text documents or graphics-only pages reduce the effectiveness of line-based constraints. Finally, although the HV Fusion Module is lightweight by design, the attention blocks still add some cost (Li et al., 11 Jul 2025).

A common misconception is that D2Dewarp is merely a text-line straightening system. The paper’s design and annotation protocol show otherwise: its geometric signal includes page foreground, figures, tables, paragraphs, and text lines, and the model learns deformation from their horizontal-vertical interaction rather than from horizontal baselines alone. Another misconception is that it enforces deformation regularity through explicit smoothness penalties; in fact, the paper explicitly states that no explicit smoothness, curvature, or cross-term regularizers are used, and that regularity is learned through line supervision and attention-based feature coupling (Li et al., 11 Jul 2025).