PP-DocLayout: A Unified Document Layout Detection Model to Accelerate Large-Scale Data Construction (2503.17213v1)

Abstract: Document layout analysis is a critical preprocessing step in document intelligence, enabling the detection and localization of structural elements such as titles, text blocks, tables, and formulas. Despite its importance, existing layout detection models face significant challenges in generalizing across diverse document types, handling complex layouts, and achieving real-time performance for large-scale data processing. To address these limitations, we present PP-DocLayout, which achieves high precision and efficiency in recognizing 23 types of layout regions across diverse document formats. To meet different needs, we offer three models of varying scales. PP-DocLayout-L is a high-precision model based on the RT-DETR-L detector, achieving 90.4% [email protected] and an end-to-end inference time of 13.4 ms per page on a T4 GPU. PP-DocLayout-M is a balanced model, offering 75.2% [email protected] with an inference time of 12.7 ms per page on a T4 GPU. PP-DocLayout-S is a high-efficiency model designed for resource-constrained environments and real-time applications, with an inference time of 8.1 ms per page on a T4 GPU and 14.5 ms on a CPU. This work not only advances the state of the art in document layout analysis but also provides a robust solution for constructing high-quality training data, enabling advancements in document intelligence and multimodal AI systems. Code and models are available at https://github.com/PaddlePaddle/PaddleX .

• The paper introduces a unified framework that detects 23 distinct layout regions across various document types.
• The paper employs knowledge distillation and semi-supervised learning to achieve 90.4% mAP and sub-15 ms inference times.
• The paper offers three scalable model variants, enabling balanced performance for both high-precision and real-time applications.

Overview and Contribution

PP-DocLayout proposes a unified document layout detection framework aimed at accelerating large-scale data construction for document intelligence tasks. The work addresses the limitations in generalizing across heterogeneous document formats, managing complex layouts, and delivering real-time inference performance. The framework is designed to identify 23 distinct layout region types—from titles and text blocks to tables and formulas—across diverse document categories such as academic articles, exam papers, magazines, and newspapers. Three variants of the model are provided: PP-DocLayout-L, PP-DocLayout-M, and PP-DocLayout-S, each tailored to different trade-offs between accuracy and computational efficiency. The paper reports strong numerical performance, including a 90.4% [email protected] achieved by PP-DocLayout-L on a T4 GPU with an end-to-end inference time of 13.4 ms per page.

Methodological Details

Unified Architecture and Model Variants

• Model Architectures:
  • PP-DocLayout-L: Built on the RT-DETR-L detection architecture, this variant employs a robust PPHGNetV2-B4 backbone. It incorporates knowledge distillation from a larger pre-trained model (Vary-VIT-B) as a teacher, where the distillation loss is formulated as:

  $L_{distill} = \| T(p) - S(p) \|_2^2$

  Here, $T(p)$ and $S(p)$ denote feature tensors from teacher and student networks after respective linear projections. - PP-DocLayout-M and PP-DocLayout-S: These models are based on variants of the PicoDet architecture. Their performance is enhanced via semi-supervised learning, wherein the high-performing PP-DocLayout-L model generates pseudo-labels to refine training on unlabeled samples. An adaptive threshold mechanism maximizes the F-score on the validation set to select reliable pseudo-labels.

• Scalable Implementation:

By offering multiple model scales, the framework accommodates scenarios ranging from high-precision offline analysis (PP-DocLayout-L) to resource-constrained or real-time applications (PP-DocLayout-S). The system effectively balances detection accuracy against latency metrics, with PP-DocLayout-S achieving an inference time as low as 8.1 ms per page on a T4 GPU and 14.5 ms per page on a CPU.

Training Protocols and Data Construction

• Training Regimen:

  • PP-DocLayout-L is trained using a constant learning rate of 0.0001 over 100 epochs with a mini-batch size of 2 per GPU, leveraging 8 V100 GPUs.
  • PP-DocLayout-M and PP-DocLayout-S use dynamic CosineDecay learning rate scheduling over 100 epochs with similar per-GPU batch sizes.
• Data Sources and Annotation:

The training dataset is composed of 30,000 images with heavy augmentation and manual annotation for the 23 layout classes. Additional public datasets such as DocLayNet and PubLayNet complement this dataset, ensuring robustness across diverse document modalities.

• Semi-Supervised Learning:

The framework leverages the superior performance of the large model to generate pseudo-labels for unlabeled data, a strategy that not only enlarges the training set but also improves overall generalization, particularly for the more efficient PP-DocLayout-M and PP-DocLayout-S models.

Performance Metrics and Comparative Analysis

• Detection Accuracy:

The highest accuracy reported is 90.4% [email protected] for PP-DocLayout-L, demonstrating its efficacy in precisely localizing and classifying the document layout regions. In contrast, PP-DocLayout-M offers 75.2% [email protected] while maintaining a lower inference runtime.

• Latency Considerations:
  • PP-DocLayout-L: 13.4 ms per page on a T4 GPU.
  • PP-DocLayout-M: 12.7 ms per page on a T4 GPU.
  • PP-DocLayout-S: As low as 8.1 ms per page on a T4 GPU and 14.5 ms per page on CPU.
• Trade-offs:

The differing computational requirements and performance trade-offs allow practical deployment in varied environments—from high-throughput cloud-based processing pipelines to on-device inference in resource-constrained settings.

Practical Applications and Implementation Considerations

Accelerated Data Annotation

The unified detection method of PP-DocLayout significantly speeds up the data annotation process by automating the layout analysis task, thereby accelerating the creation of structured datasets for downstream applications such as OCR, document summarization, and information extraction. The integration of pseudo-labeling ensures that even unlabeled documents contribute to model improvements, reducing the manual effort required in large-scale data curation.

Deployment Strategies

• Resource Allocation:

The different model sizes allow practitioners to choose an appropriate model based on the target deployment environment. For instance, PP-DocLayout-L is ideal for batch processing in data centers, while PP-DocLayout-S supports real-time inference on edge devices or CPU-only scenarios.

• Integration with Existing Pipelines:

The open-source codebase available at the GitHub repository facilitates seamless integration with current document processing and multimodal AI pipelines. The modular architecture of the PP-DocLayout models supports custom integration with OCR engines, table recognition modules, and document retrieval systems.

• Scalability and Performance Tuning:

Extensive profiling on hardware (e.g., NVIDIA T4 GPUs, CPUs) provides insights into optimizing inference pipelines. Applying techniques such as mixed-precision inference and model quantization can further reduce latency and memory footprint without significant loss of accuracy.

Potential Limitations and Future Directions

• Domain Adaptation:

Although PP-DocLayout demonstrates broad adaptability, deployment in highly specialized document domains might still require fine-tuning or additional data augmentation strategies.

• Compute-Efficiency Balance:

The trade-off between mAP and inference speed necessitates careful consideration when selecting a model variant. Future work may explore dynamic model scaling techniques to further optimize the balance between accuracy and latency.

• Semi-Supervised Enhancements:

Advances in semi-supervised learning frameworks may enhance the robustness of the pseudo-labeling approach, reducing error propagation from inaccurate pseudo-labels and further improving model generalization.

Conclusion

The PP-DocLayout framework delivers a robust and scalable solution for document layout detection, rigorously addressing challenges related to generalization, efficiency, and large-scale data construction. With its unified architecture, incorporation of knowledge distillation, and semi-supervised learning strategies, the model exhibits state-of-the-art performance metrics (e.g., 90.4% [email protected] and sub-15 ms per page inference times). Its flexible deployment options and comprehensive categorization of document elements mark it as a significant advance for practical document intelligence applications, enabling improved downstream processing in multimodal AI systems.