ADO Method: Optimizing LLM Data

• ADO Method is a framework that enhances LLM performance by optimizing raw input data through systematic content engineering and structural reformulation.
• It improves data quality by imputing missing values, removing irrelevant features, and enriching context with derived domain indicators.
• The method reformats and reorders data to exploit LLM sensitivity, achieving measurable gains in accuracy and robustness across diverse benchmarks.

Automatic Data Optimization (ADO) is a prompt engineering framework that enhances the effectiveness of LLMs by directly optimizing input data used in natural language prompts. Proceeding from the observation that LLM outputs are highly sensitive not only to instructions and in-context examples but also to the structure and fidelity of input data, ADO employs a two-pronged approach of "content engineering" and "structural reformulation." This framework, presented as a multi-agent LLM system, algorithmically transforms raw input data (denoted D) through a series of data enhancement and formatting steps, resulting in optimized input D′ that drives improved and more robust model inferencing (Lin et al., 17 Feb 2025).

1. Principles and System Overview

ADO is defined as a composite functional transformation on input data: D′ = f_format(f_content(D)) = f(D)

Here, D is the original input data and D′ is the optimized output. The ADO system employs three specialized LLM modules:

• Prompt Generation LLM: Produces candidate data optimization instructions (Pₒ) using a meta prompt (Pₘ).
• Data Optimization LLM: Executes selected instructions to transform D into D′.
• Task Inference LLM: Processes the optimized input together with the main instruction, generating the desired output.

This architecture enables modular and concurrent execution of content and format optimization, with factual correctness validated by an additional hallucination mitigation step.

2. Content Engineering

Content engineering aims to improve the inherent quality of D before presentation to the LLM. Key operations include:

• Imputation of Missing Values: Leveraging model-driven inference, missing attributes are filled using domain knowledge (e.g., estimating body mass index from height and weight).
• Removal of Irrelevant Attributes: Features deemed non-essential or noisy are eliminated, enhancing the salience and focus of the prompt data.
• Profile Enrichment: Supplementation with synthesized or derived indicators inferred from existing attributes (e.g., generating health risk scores or aggregating multiple features).

The result is that the raw input data is cleansed and contextually enriched, enabling subsequent LLM inferencing to focus on salient, information-dense features.

3. Structural Reformulation

Structural reformulation addresses the format and organization of data after content engineering, exploiting the LLM’s sensitivity to the presentation of inputs:

• Reformatting: Transformation from plain text to structured formats (tables, XML, attribute-value lists), or into symbolic representations (e.g., logic operators, emojis).
• Optimized Layout: Strategic ordering and arrangement to highlight key features and group correlated attributes.

By customizing the way data is formatted, this step ensures the LLM can exploit both semantic and syntactic cues to enhance interpretability and task fidelity.

4. Algorithms and Implementation Details

ADO relies on automated prompt and data optimization procedures:

• Meta-Prompts: Carefully crafted templates incorporating modality-specific constraints, preventing the LLM from attempting out-of-scope actions (e.g., principal component analysis or one-hot encoding which require symbolic computation).
• Diverse Prompt Search (DPS): An algorithm that generates multiple diverse candidate prompts per iteration, enforcing diversity via constraints: cos(Pₒ^i, Pₒ^j) < c₁; METEOR(Pₒ^i, Pₒ^j) < c₂ where cos is cosine similarity and METEOR is the machine translation evaluation metric. Hyperparameters (k, c₁, c₂) are tuned via Bayesian search for optimal performance.
• Parallelization: Data optimization procedures run concurrently across multiple LLM instances, increasing throughput and suppressing redundancy.
• Hallucination Mitigation: Outputs are fact-checked using a secondary LLM to confirm factual consistency.

Source code for the framework is published at [https://anonymous.4open.science/r/ADO-6BC5/].

5. Empirical Performance

ADO enhances LLM performance across multiple datasets and tasks, as verified by extensive ablations and benchmarks:

• Performance Metrics: On systems such as GPT-3.5 Turbo, ADO with DPS increased mean performance (accuracy) from approximately 0.443 (baseline, unoptimized data) to 0.536 across nine datasets. Comparable gains were observed on Gemini-1.5 Flash and Llama-3.1 70B.
• Benchmark Comparisons: ADO outperformed alternative prompt optimization schemes including APE and OPRO. Furthermore, methods that were suboptimal when used in isolation (Chain-of-Thought, ICL, PE2) saw boosted performance when integrated with ADO.
• Synergistic Gains: Combined application of ADO with other prompt engineering techniques produced even higher accuracy and robustness, demonstrating the method’s additive value.

6. Future Directions

Future research avenues identified in the paper include:

• Joint Optimization of Data and Instructions: Moving from a sequential optimization paradigm to simultaneous, end-to-end tuning of data and prompt instructions.
• Application Domains: Extending ADO beyond recommendations and classification to domains such as healthcare decision support, financial forecasting, and multilingual tasks.
• Multi-modal Extensions: Investigating applicability of ADO strategies to non-textual data modalities (images, audio, sensor data) by analogous content and structural optimization.
• Advanced Meta-Prompting and Retrieval-Augmentation: Using retrieval-augmented generation or dynamic context sourcing for more robust and scalable input optimization.

7. Significance and Context

ADO introduces a distinct axis of prompt engineering that focuses on the often-neglected input data container, systematically optimizing both the content and the format of LLM inputs. It does not supplant instruction tuning or demonstration selection but complements these by removing noise, highlighting critical attributes, and exploiting the LLM’s formatting sensitivities. Empirical evidence (Lin et al., 17 Feb 2025) supports that such data-centric optimizations produce measurable gains in LLM-driven tasks, pointing to an expanded research landscape at the intersection of data curation and prompt engineering.