SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posters (2407.19787v1)

Abstract: Scientific posters are used to present the contributions of scientific papers effectively in a graphical format. However, creating a well-designed poster that efficiently summarizes the core of a paper is both labor-intensive and time-consuming. A system that can automatically generate well-designed posters from scientific papers would reduce the workload of authors and help readers understand the outline of the paper visually. Despite the demand for poster generation systems, only a limited research has been conduced due to the lack of publicly available datasets. Thus, in this study, we built the SciPostLayout dataset, which consists of 7,855 scientific posters and manual layout annotations for layout analysis and generation. SciPostLayout also contains 100 scientific papers paired with the posters. All of the posters and papers in our dataset are under the CC-BY license and are publicly available. As benchmark tests for the collected dataset, we conducted experiments for layout analysis and generation utilizing existing computer vision models and found that both layout analysis and generation of posters using SciPostLayout are more challenging than with scientific papers. We also conducted experiments on generating layouts from scientific papers to demonstrate the potential of utilizing LLM as a scientific poster generation system. The dataset is publicly available at https://huggingface.co/datasets/omron-sinicx/scipostlayout_v2. The code is also publicly available at https://github.com/omron-sinicx/scipostlayout.

SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posters

The paper "SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posters" presents an extensive effort to address the challenges associated with the automated generation of scientific posters. The authors introduced SciPostLayout, a dataset comprising 7,855 manually annotated scientific posters and 100 associated scientific papers, all under the Creative Commons CC-BY license. This dataset aims to serve as a benchmark for both layout analysis and layout generation tasks specific to scientific posters.

Background and Motivation

The creation of scientific posters that effectively summarize research contributions in a graphical format is a labor-intensive and time-consuming process. The automation of this task has potential benefits, including reduced workload for researchers and improved accessibility of the research contributions. Despite its importance, progress in the development of automated systems for poster layout has been hampered by the lack of comprehensive datasets. SciPostLayout addresses this gap, providing a publicly available, well-annotated dataset that researchers can use for benchmarking layout analysis and generation models.

Contributions

The contributions of this paper are manifold:

1. Dataset Construction: SciPostLayout includes 7,855 posters with manual annotations across nine categories: Title, Author Info, Section, Text, List, Table, Figure, Caption, and Unknown. This level of detailed annotation surpasses previous datasets, which often focused on broader categories and lacked the granularity necessary for nuanced layout analysis.
2. Associated Paper Collection: The dataset supplements 100 scientific papers paired with their corresponding posters. This pairing enables the evaluation of poster generation models that can process raw scientific manuscripts.
3. Evaluation of Existing Models: The authors conducted benchmark experiments on layout analysis and layout generation using existing models such as LayoutLMv3, DiT, LayoutDM, LayoutFormer++, and LayoutPrompter. These evaluations highlight the challenges and potential improvements in processing and generating scientific poster layouts.

Experimental Findings

Layout Analysis

The experiments using LayoutLMv3 and DiT models revealed that while high accuracy could be achieved in recognizing certain elements like Titles and Author Info, overall, the performance on SciPostLayout was lower compared to datasets like PubLayNet. Specifically, the mean average precision (mAP) indicated a more complex layout structure in posters, suggesting that scientific posters have a higher variability in font styles, figure placements, and general layout.

Layout Generation

For layout generation, models were assessed using various settings, including unconditional and conditional generation. Metrics such as maximum IoU (mIoU), Alignment, Overlap, and Fréchet Inception Distance (FID) were employed. The results demonstrated that generating aligned layouts was feasible; however, creating layouts that closely mirror real-world examples remains a significant challenge. Notably, LayoutPrompter outperformed other models in minimizing overlap and achieving higher similarity to real layouts under the refinement setting.

Paper-to-Layout Generation

To extend the practicality of SciPostLayout, the authors implemented models to directly generate poster layouts from scientific papers. This involved extracting layout constraints via GPT-4 and generating layouts using models like LayoutPrompter. Although the generated layouts did not perfectly replicate real posters, the experiments showcased the potential of LLMs in contributing to automated poster generation. The results underscored the difficulty in accurately extracting detailed layout constraints solely from textual content.

Implications and Future Work

The creation of SciPostLayout is a significant step forward in addressing the lack of datasets tailored to scientific poster layout tasks. It sets a comprehensive benchmark, encouraging further research in this domain. The implications of this work are both practical and theoretical:

• Practical Implications: The dataset provides a foundation for developing tools that can streamline the poster creation process for researchers, potentially integrating with conference management systems or academic networking platforms.
• Theoretical Implications: The complexity of scientific poster layouts presents a fertile ground for advancing layout analysis and generation models. Insights gained from experiments on SciPostLayout can propel improvements in multimodal learning, encompassing both visual and textual data.

The paper suggests future research directions, including improving layout analysis models' performance and refining the extraction of content from scientific papers to create more accurate and visually appealing poster layouts. Enhanced LLM-based systems could eventually facilitate end-to-end poster generation directly from scientific manuscripts, marking a significant milestone in automated academic dissemination.

In conclusion, SciPostLayout represents a valuable resource, laying the groundwork for future advancements in layout analysis and generation within the specific context of scientific posters. The benchmarks set forth by this research will be instrumental in guiding future innovations in this space.