Develop improved handling of marginalia and paratext

Develop improved methods for addressing page numbers, running headers and footers, marginalia, and related paratextual elements in the Institutional Books: Harvard Library Enriched Text pipeline while preserving their navigational and bibliographic value.

Background

The pipeline removes page numbers and running headers and footers because, after page boundaries are discarded, these elements frequently appear as recurring noise or fragments in the computational text stream. The authors note that such marginalia nonetheless carries useful navigational and bibliographic information.

The paper explicitly leaves broader treatment of these paratextual elements unresolved. Improving their handling would allow the enriched dataset to preserve or represent marginalia more accurately rather than removing it or allowing it to remain intermixed with the main text.

References

Addressing these forms of paratext (and marginalia more broadly) is left to future work.

— Institutional Books - Enriched Text: A customizable multilingual open-source pipeline for denoising, deduplicating, and annotating OCR text at scale  (2608.19026 - Lowry-Duda et al., 19 Aug 2026) in Section 3, “Goals and Scope,” paragraph “Configurability”