PatentVision: A multimodal method for drafting patent applications (2510.09762v1)

Abstract: Patent drafting is complex due to its need for detailed technical descriptions, legal compliance, and visual elements. Although Large Vision LLMs (LVLMs) show promise across various tasks, their application in automating patent writing remains underexplored. In this paper, we present PatentVision, a multimodal framework that integrates textual and visual inputs such as patent claims and drawings to generate complete patent specifications. Built on advanced LVLMs, PatentVision enhances accuracy by combining fine tuned vision LLMs with domain specific training tailored to patents. Experiments reveal it surpasses text only methods, producing outputs with greater fidelity and alignment with human written standards. Its incorporation of visual data allows it to better represent intricate design features and functional connections, leading to richer and more precise results. This study underscores the value of multimodal techniques in patent automation, providing a scalable tool to reduce manual workloads and improve consistency. PatentVision not only advances patent drafting but also lays the groundwork for broader use of LVLMs in specialized areas, potentially transforming intellectual property management and innovation processes.

• The paper demonstrates the effectiveness of integrating textual and visual inputs using LVLMs to generate precise patent specifications.
• It introduces a novel claim+diagram-to-specification methodology that improves both technical accuracy and legal compliance.
• Experimental results show that PatentVision outperforms text-only models with superior BLEU, METEOR, and ROUGE scores.

PatentVision: A Multimodal Method for Drafting Patent Applications

PatentVision represents a methodological advancement in patent drafting by integrating both textual and visual elements into the generation process of patent specifications. Leveraging Large Vision-LLMs (LVLMs), this approach aims to improve the accuracy and coherence of automated patent drafting, addressing limitations found in traditional text-only methodologies.

Introduction and Motivation

The process of drafting a patent specification is inherently complex, requiring the translation of intricate technical concepts into precise legal documentation. Traditional methods have predominantly relied on textual analysis, often neglecting the crucial role of patent drawings in conveying design intent and functional details. This oversight can result in specifications that fail to accurately reflect inventors' intentions. PatentVision addresses this gap by utilizing a multimodal approach that incorporates both patent claims and visual illustrations, thereby bridging the gap between textual and visual elements in patent drafting.

Figure 1: PatentVision is a framework that generates high-quality patent specifications using multimodal inputs like images, patent claims, and optional figure descriptions.

The integration of LVLMs enables a deeper understanding of contextually rich scenarios by processing multimodal data streams. By utilizing models such as Gemma, LLAVA, and LLaMA, PatentVision can combine patent claims with corresponding drawings, thereby enhancing the quality of the generated specifications. This dual-input architecture allows for a holistic interpretation of the invention, ensuring technical accuracy and compliance with legal standards.

Related Work

Previous research in patent text generation has primarily focused on specific sections, such as claims or abstracts, rather than full patent specifications. Efforts have been made to generate claims using models like GPT-2 and BERT-based modules, while others have explored summarizing patents for titles, abstracts, or figure captions. However, these approaches have not fully addressed the integration of visual elements with textual input. PatentVision builds upon PatentFormer, which first tackled the task of generating full patent specifications from claims and drawings, to introduce a more holistic multimodal approach.

Figure 2: PatentFormer processes text by taking an image, the claim, and the image description, outputting an enriched textual representation.

Methodology

PatentVision's core innovation lies in its claim+diagram-to-specification task. This multimodal task involves generating a patent specification from a set of inputs, including claims, brief descriptions, images, and component details. The framework utilizes a training approach that maps each claim feature to a specification paragraph, integrating visual data for enhanced context and accuracy.

The model uses enhanced tokens in the input and output specifications to mark important elements such as component names and numbers, aiding in the generation of coherent and contextually appropriate descriptions. By employing special tags and enriched context in its training data, PatentVision significantly improves upon previous models in generating full-length specifications.

Experimental Setup

PatentVision was trained using a newly constructed dataset containing patent data under the CPC code 'G06F,' encompassing electronic digital data processing patents. The evaluation involved several large vision-LLMs (LVLMs), including Gemma, LLAVA, and LLaMA, which were fine-tuned on this dataset. Key evaluation metrics included natural language generation benchmarks such as BLEU, METEOR, and ROUGE scores, with models trained on NVIDIA A100 GPUs for efficiency.

Experimental Results

The experiments revealed that PatentVision outperformed text-only models like PatentFormer across all evaluation metrics when integrating visual content with textual input. Fine-tuned LVLMs demonstrated significant improvements over their pretrained versions, especially in adhering to legal and technical writing standards.

Figure 3: Comparison between PatentVision with different base LVLMs and LoRA ranks and PatentFormer.

The ablation paper showed that higher image resolutions and optimal training configurations led to enhanced performance, underscoring the importance of multimodal data integration. Furthermore, PatentVision exhibited robust performance even in the absence of explicit image descriptions, indicating its ability to derive meaningful information directly from visual data.

Conclusions

PatentVision represents a significant advance in the automated drafting of patent specifications by effectively combining textual and visual information. This multimodal approach not only improves the quality of generated specifications but also highlights the potential of LVLM applications in specialized domains. The framework offers a scalable tool for enhancing consistency and reducing manual workload in patent drafting, with the potential to transform intellectual property management practices. Future developments may focus on further enhancing model interactivity and exploring additional applications in intellectual property and beyond.