Structured Legal Document Generation in India: A Model-Agnostic Wrapper Approach with VidhikDastaavej (2504.03486v1)

Abstract: Automating legal document drafting can significantly enhance efficiency, reduce manual effort, and streamline legal workflows. While prior research has explored tasks such as judgment prediction and case summarization, the structured generation of private legal documents in the Indian legal domain remains largely unaddressed. To bridge this gap, we introduce VidhikDastaavej, a novel, anonymized dataset of private legal documents, and develop NyayaShilp, a fine-tuned legal document generation model specifically adapted to Indian legal texts. We propose a Model-Agnostic Wrapper (MAW), a two-step framework that first generates structured section titles and then iteratively produces content while leveraging retrieval-based mechanisms to ensure coherence and factual accuracy. We benchmark multiple open-source LLMs, including instruction-tuned and domain-adapted versions, alongside proprietary models for comparison. Our findings indicate that while direct fine-tuning on small datasets does not always yield improvements, our structured wrapper significantly enhances coherence, factual adherence, and overall document quality while mitigating hallucinations. To ensure real-world applicability, we developed a Human-in-the-Loop (HITL) Document Generation System, an interactive user interface that enables users to specify document types, refine section details, and generate structured legal drafts. This tool allows legal professionals and researchers to generate, validate, and refine AI-generated legal documents efficiently. Extensive evaluations, including expert assessments, confirm that our framework achieves high reliability in structured legal drafting. This research establishes a scalable and adaptable foundation for AI-assisted legal drafting in India, offering an effective approach to structured legal document generation.

Lawful AI: A Model-Agnostic Framework for Structured Legal Document Generation in India

The paper "Structured Legal Document Generation in India: A Model-Agnostic Wrapper Approach with VidhikDastaavej" presents an innovative methodological framework for generating private legal documents in the Indian legal domain. The research specifically focuses on addressing the complexities involved in automating the drafting of legal documents while maintaining coherence, consistency, and factual accuracy. This is achieved through a novel Model-Agnostic Wrapper (MAW) approach that is capable of enhancing structured legal drafting.

Research Contributions

The primary contributions of this paper include:

1. VidhikDastaavej Dataset: The introduction of a novel anonymized dataset featuring a diverse array of Indian private legal documents. This dataset, critical for training and evaluating models within the Indian legal framework, advances the scope of AI capabilities in legal contexts where datasets are often limited due to confidentiality.
2. NyayaShilp Model: A domain-adapted LLM fine-tuned on Indian legal texts. NyayaShilp undergoes pretraining on publicly available legal corpora to embed domain-specific knowledge, followed by supervised fine-tuning on the VidhikDastaavej dataset. This two-stage training process is designed to ensure relevance and applicability in generating legally sound content.
3. Model-Agnostic Wrapper (MAW): A significant methodological innovation entailing a two-phase framework that first generates structured section titles and iteratively produces section content. The MAW improves long-form text generation by enforcing consistency and factual accuracy, pivotal for mitigating hallucinations—a common challenge in AI-generated text.
4. Human-in-the-Loop (HITL) System: An interactive tool enabling legal professionals to specify document types, refine section details, and generate drafts. This system emphasizes the importance of human oversight in validating AI-generated outputs.
5. Expert-Based Evaluation: Rigorous assessments conducted by legal experts focusing on factual accuracy and comprehensiveness. The evaluation methodology ensures that AI-generated drafts adhere to legal standards, offering reliability beyond conventional lexical and semantic metrics.

Results and Implications

The results underscore the MAW's effectiveness in producing structured and coherent legal documents. When benchmarked, the wrapper-assisted models demonstrated improvements comparable to the performance of proprietary solutions like GPT-4o in generating coherent and legally valid texts. However, instruction tuning on limited datasets did not yield the anticipated enhancement, indicating a need for broader and more diverse data coverage, especially in underrepresented legal document categories.

Despite advancements, fine-tuned models such as NyayaShilp sometimes struggled with consistency and hallucination issues due to the constrained size of the fine-tuning dataset. The paper highlights the need for more expansive datasets to effectively leverage the capabilities of sophisticated LLMs in legal drafting.

Ethical Considerations and Future Research

The research critically addresses ethical concerns, emphasizing data privacy, transparency, and bias mitigation. Anonymization of the VidhikDastaavej dataset ensures compliance with ethical and legal standards. Moreover, the paper stresses that AI-generated drafts are not substitutes for human expertise but should serve as assistive tools complemented by professional oversight.

Future research directions proposed include expanding the dataset's diversity, refining fine-tuning processes, and integrating advanced mechanisms such as retrieval-augmented generation or reinforcement learning to improve factual accuracy.

By advancing structured legal document generation through the MAW framework and NyayaShilp model, this work sets a foundation for modernizing legal workflows in India while addressing the challenges of AI deployment in sensitive domains.