Fine-tuned Model Querying Methods

• Fine-tuned model querying is a technique where pre-trained models are adapted to map natural language queries to domain-specific outputs using methods like LoRA and QLoRA.
• It employs parameter-efficient fine-tuning, retrieval-on-demand, and execution-based feedback loops to convert queries into structured outputs such as SQL statements, Python code, or refined predictions.
• The approach finds applications in legal, medical, financial, and retail domains and has demonstrated significant improvements in accuracy and efficiency across various evaluation protocols.

Fine-tuned model querying denotes a family of systems in which a pre-trained model is adapted, partially updated, or calibrated so that a natural-language query can be mapped to a domain-specific answer, a corrected query, a Python program, an API call, a SQL statement, or a refined prediction. In recent work, this pattern appears in Arabic legal question answering, misleading-query correction, financial document reasoning, medical question answering, natural-language access to structured data, retail text-to-SQL, and query-conditioned test-time adaptation, with parameter-efficient updates such as LoRA and QLoRA, retrieval-on-demand components, and execution-based feedback loops playing central roles (Fasha et al., 24 Jan 2026, Li et al., 15 Apr 2025, Phogat et al., 2024, Anaissi et al., 2024, Valentin-Micu et al., 3 Apr 2026, Rebei, 2023, Song et al., 13 May 2026).

1. Query modalities and task formulations

The most direct form of fine-tuned model querying is domain-specific question answering. The Jordanian-law study fine-tunes two Arabic Llama-3.1 variants on structured legal prompts of the form “Context: … Question: … Answer:”, while the medical QA study fine-tunes LLaMA2-7B-chat and Mistral-7B and then augments answering with retrieval on demand. A second form inserts an explicit query-revision stage: the three-stage misleading-query method first decides whether a query contains inaccurate or deceptive content, then rewrites it, and only then generates an answer. A third form does not answer directly, but instead emits an executable intermediate representation such as Python, an API call, or SQL, after which an external executor returns the final result (Fasha et al., 24 Jan 2026, Li et al., 15 Apr 2025, Anaissi et al., 2024, Phogat et al., 2024, Valentin-Micu et al., 3 Apr 2026, Rebei, 2023).

Setting	Fine-tuned model behavior	Final output form
Arabic legal QA	Answer generation from structured legal prompts	Arabic answer
Misleading-query handling	Detection, correction, then answering	Label, corrected query, answer
Financial document QA	Program-of-Thought generation	Python and ans
Medical QA	Retrieval-on-demand answer generation	Answer conditioned on relevant chunks
Structured-data querying	Executable API-call generation	<API>...</API> and final answer
Retail NL→SQL	Dialect-specific SQL generation	SQL query

A broader interpretation appears in tabular foundation models and model-knowledge-base systems. In TabPFNv2, a downstream test object functions as a query over in-context examples, and fine-tuning sharpens the dot products between query-representations and key-representations. In M-DESIGN, a user’s task query is treated as an adaptive query over prior task–model–performance records, and model refinement itself is executed through SQL-style joins over Tasks, Architectures, and ModificationGains (Rubachev et al., 10 Jun 2025, Wang et al., 21 Jul 2025).

2. Data construction and prompt interfaces

Fine-tuned model querying depends heavily on how query–target pairs are constructed. The Jordanian-law case study collected the full text of 18 Jordanian laws, extracted 3,578 distinct articles, used the OpenAI GPT-3.5-Turbo API to generate {context, question, answer} triples, manually corrected all triples, and produced a final dataset of approximately 6,000 examples with an 80/20 train/test split. Each triple was converted into the uniform prompt template “Context: … Question: … Answer:” for PEFT fine-tuning (Fasha et al., 24 Jan 2026).

The misleading-query pipeline constructs a paired clean/misleading corpus rather than a conventional QA set. HaluEval-QAₘᵢₛ and CQAₘᵢₛ were generated by prompting Qwen2.5-72B-Instruct to perturb original questions with wrong entities, times, or relations, after which candidate queries were filtered by surface similarity $S_{\text{sim}} > 0.8$ and answer error rate $E_{\text{error}} > 0.5$. The training set contains 4 K clean QA and 4 K misleading QA, and the test protocol separates clean and misleading variants so that the effect of input corruption can be measured directly (Li et al., 15 Apr 2025).

Program-generating systems use teacher-produced reasoning traces as supervision. In financial document QA, GPT-4 is prompted with four Program-of-Thought exemplars and an “Answer Hint,” and only code that executes and matches ground truth is retained. The filtered retention rates are approximately 91% of FinQA, 93% of ConvFinQA, and 97% of TATQA, after which prompt–completion pairs are built so that the student model emits complete Python snippets ending in ans (Phogat et al., 2024).

Structured-data and SQL-querying systems likewise rely on synthetic generation pipelines. The Durangaldea structured-data system enumerates semantically meaningful projections, uses Gemini 2.5 Pro and DeepSeek R1 for template generation, instantiates templates with $N=358$ representative coordinate points from the Overpass API, executes the resulting API calls against the database, and paraphrases questions, yielding 44,849 distinct question–answer pairs. The retail text-to-SQL workflow uses GPT-4 for topic expansion, question generation, SQL synthesis, a self-healing execution loop, semantic validation, and four question rewrites per surviving SQL query, producing 822 unique SQL/NL pairs, augmented to 3,732 by rewriting, with an 80/20 train/test split (Valentin-Micu et al., 3 Apr 2026, Rebei, 2023).

Medical QA uses “Medical Meadow,” a mixture of MediQA and GPT-3.5-rephrased Anki flashcards, comprising approximately 34 K flashcards and approximately 2.2 K MediQA QA pairs. Financial-news multitask analysis uses approximately 4,000 Kaggle news items paired with instructions that require analysis, main-point highlighting, summarization, and named-entity extraction with sentiment, with a 75/25 train/validation split (Anaissi et al., 2024, Pavlyshenko, 2023).

3. Adaptation mechanisms and optimization regimes

Parameter-efficient fine-tuning is the dominant adaptation mechanism. In the Jordanian-law system, the original Llama 3.1 weight matrix $W_0 \in \mathbb{R}^{d\times k}$ is frozen and a low-rank update $\Delta W = B \cdot A$ is learned, with the effective weight

$W = W_0 + \frac{\alpha}{r}\cdot \Delta W,$

using $r = 16$, $\alpha = 16$, no adapter dropout, and trainable matrices $A \in \mathbb{R}^{r\times k}$ and $B \in \mathbb{R}^{d\times r}$. The base weights are stored in 4-bit form, the LoRA update is stored in FP16/BF16, and Unsloth provides automatic kernel fusion, gradient accumulation, mixed precision, CPU offloading, overlapping of data loading with forward/backward passes, and sparse checkpointing of frozen layers (Fasha et al., 24 Jan 2026).

The same low-rank principle underlies QLoRA and LoRa-based query generation in structured-data and text-to-SQL systems. The Durangaldea system freezes all original 8B parameters of DeepSeek R1-Distill-8B, quantizes weights to 4 bits, and trains roughly 10 million LoRA parameters with AdamW, a peak learning rate of $E_{\text{error}} > 0.5$0, batch size 32, and 4 epochs. The retail SQL system similarly freezes the base matrices and adds $E_{\text{error}} > 0.5$1 scaled as $E_{\text{error}} > 0.5$2, with inference-time int8 quantization via EETQ (Valentin-Micu et al., 3 Apr 2026, Rebei, 2023).

A more elaborate PEFT variant appears in rsDoRA+. There, the update is

$E_{\text{error}} > 0.5$3

and DoRA decomposes the pre-trained weight as $E_{\text{error}} > 0.5$4, so that adaptation is applied to the direction component. rsDoRA+ also uses differential learning rates, $E_{\text{error}} > 0.5$5 with typical $E_{\text{error}} > 0.5$6, and adds small uniform embedding noise through the NEFtune step. On the Flashcards test set with LLaMA2-7B-chat, rsDoRA+ improves as rank increases and reaches BLEU-4 of 0.5137 at $E_{\text{error}} > 0.5$7 (Anaissi et al., 2024).

Not all querying systems favor PEFT. For TabPFNv2, the study compares full fine-tuning, LoRA, last-layer-only tuning, and embedding/LayerNorm/head tuning, and concludes that full fine-tuning is the most practical solution in terms of time-efficiency and effectiveness. The reported interpretation is retrieval-based: after fine-tuning, the dot products $E_{\text{error}} > 0.5$8 more accurately reflect target similarity, the softmax weights $E_{\text{error}} > 0.5$9 become sharper, and prediction improves by weighting more relevant in-context samples (Rubachev et al., 10 Jun 2025).

Test-time adaptation extends the idea of fine-tuned querying from offline training to per-query updates. QueST samples $N=358$0 auxiliary problem–solution pairs from the input query itself, performs LoRA updates on attention projections with rank $N=358$1, $N=358$2, and dropout $N=358$3, and then answers the original query using the adapted parameters. The default configuration uses $N=358$4 generated pairs and $N=358$5 adaptation steps, with $N=358$6 for AIME tasks (Song et al., 13 May 2026).

4. Query execution pathways and control flow

A central distinction among fine-tuned querying systems is whether the model answers directly or via an intermediate control structure. The three-stage misleading-query method defines a sequential inference pipeline: input query to a detection head, conditional routing through a correction head if the query is misleading, and then final answering by a generation head. The losses are separated accordingly: binary cross-entropy for detection, a Sentence-BERT cosine-similarity objective for correction, and token-level cross-entropy for answer generation. The paper also notes an optional combined objective $N=358$7, although the reported method trains the stages sequentially (Li et al., 15 Apr 2025).

Medical QA implements a different control flow. ReRAG first predicts a binary Retrieve/No-retrieve token, retrieves top-$N=358$8 document chunks only when necessary, applies an IsRel classifier to drop irrelevant chunks, and invokes a Question-Rewrite module if no retrieved chunk passes relevance filtering. The generator is then prompted with the relevant chunks concatenated with the original question. This architecture couples fine-tuning and retrieval rather than treating retrieval as a static external preprocessor (Anaissi et al., 2024).

Executable-query systems externalize the final computation. In the structured-data system, the model is instructed to generate exactly one <API>...</API> call and then stop. A wrapper detects the API block, pauses generation, executes the call, injects the JSON result back into the context, and resumes generation to produce the natural-language answer. Validation is regex-based, and a fallback re-prompt is used if syntax fails, which is reported as rare and below 1% (Valentin-Micu et al., 3 Apr 2026).

Financial document QA follows the same execution-oriented logic but with Python rather than an API. At query time, the fine-tuned student receives a passage and question, generates Python in a standardized Program-of-Thought format, and an external interpreter executes the code and returns ans. The paper attributes performance gains not only to reasoning-chain induction but also to improved entity extraction from table/text formats (Phogat et al., 2024).

Other systems constrain output structure without external execution. The financial-news Llama 2 model is trained to return four sections in order—Analysis, Main Points, Summary, and JSON Data—and the JSON array is post-processed with schema validation. This is still a query interface, but the control target is output regularity rather than SQL, API, or Python executability (Pavlyshenko, 2023).

5. Evaluation protocols and empirical behavior

Evaluation protocols differ substantially across querying modes. The Jordanian-law study uses BLEU and ROUGE to compare fine-tuned and base models. The misleading-query work combines detection accuracy or F1, instruction-following failure rate, QA accuracy, BLEURT, BLEU, ROUGE, and hallucination detection accuracy. Financial document QA uses end-to-end QA accuracy after executing ans, program execution accuracy, Program-of-Thought concept accuracy, and entity extraction accuracy. Structured-data and SQL systems emphasize exact-match or execution-result correctness rather than only text overlap (Fasha et al., 24 Jan 2026, Li et al., 15 Apr 2025, Phogat et al., 2024, Valentin-Micu et al., 3 Apr 2026, Rebei, 2023).

In Arabic legal QA, the reported results are:

• llama3.1 (base): BLEU 0.058, ROUGE-1 0.026, ROUGE-2 0.000, ROUGE-L 0.026
• llama3.1_instruct (base): BLEU 0.128, ROUGE-1 0.070, ROUGE-2 0.013, ROUGE-L 0.070
• fine_tuned_llama3.1: BLEU 0.290, ROUGE-1 0.081, ROUGE-2 0.020, ROUGE-L 0.081
• fine_tuned_llama3.1_instruct: BLEU 0.270, ROUGE-1 0.063, ROUGE-2 0.005, ROUGE-L 0.063

The paper summarizes this as BLEU increasing by approximately 0.21 on the vanilla base and ROUGE-1 roughly tripling on the non-instruct variant. It also reports that 4-bit quantization reduces model storage from approximately 16 bytes per parameter to approximately 0.5 bytes per parameter, that LoRA adapters add approximately 0.5% extra parameters, and that training fit on a single NVIDIA A100-40 GB GPU with peak 39.6 GB and 6 GB reserved (Fasha et al., 24 Jan 2026).

The three-stage misleading-query method reports, for Qwen2.5-14B, HaluEval-QA accuracy moving from approximately 62.7% to 68.2% to 73.2%, HaluEval-QAₘᵢₛ from 54.3% to 60.7% to 71.3%, CQA from 80.9% to 81.4% to 83.1%, CQAₘᵢₛ from 71.6% to 73.3% to 78.6%, and TruthfulQA BLEURT from 0.709 to 0.721 to 0.748. For Llama-3.2B on HaluEval-QAₘᵢₛ, instruction-following failure rate drops from 5.67% to 2.86% to 2.09% (Li et al., 15 Apr 2025).

Financial document QA shows that fine-tuned smaller models can approach the teacher. On FinQA test accuracy, Mistral-7B moves from 41.6% zero-shot and 50.8% few-shot to 76.63% fine-tuned, Orca-2-7B from 2.8% and 15.3% to 71.6%, phi-3-mini from 57.6% and 42.4% to 73.5%, and phi-3-medium from 70.4% and 66.9% to 77.6%, within 1% of GPT-4 few-shot at 78.5%. On TATQA_arith, fine-tuned models reach 88–93%, and phi-3-medium attains 93.7%, above GPT-4’s 93.0%. Entity extraction accuracy on FinQA dev is reported to jump from 58–83% to 90–93% after one epoch (Phogat et al., 2024).

Medical QA and structured-query generation also report strong gains under execution-aware or retrieval-aware evaluation. On Flashcards, full ReRAG reaches BLEU 0.7186, ROUGE-1 0.7508, ROUGE-2 0.6068, ROUGE-L 0.6937, and Mauve 0.9502, compared with rsDoRA+ alone at BLEU 0.5137 and Mauve 0.7959. In the Durangaldea structured-data system, exact match is 89.0 on unseen locations, 94.2 on semantic variants, and 80.0 on the full 3,300-example evaluation, with multilingual results ranging from 24.0 in Basque to 93.0 in Galician. The retail text-to-SQL study reports, on held-out retail questions, Code-Llama at 81.6% Snowflake accuracy and 82.7% GoogleSQL accuracy, Mistral-7B at 79.6% and 79.4%, and zero-shot GPT-4 at 45.6% and 48.9%; average latency is 5.5 s for Code-Llama, 5.7 s for Mistral-7B, and 47.4 s for GPT-4 (Anaissi et al., 2024, Valentin-Micu et al., 3 Apr 2026, Rebei, 2023).

Test-time fine-tuning yields a different empirical profile because adaptation occurs per query. Across seven math reasoning benchmarks and GPQA-Diamond, QueST improves Qwen3-4B average accuracy from 36.59% to 42.58% and Qwen3-8B-Base from 42.00% to 47.77%. On GPQA-Diamond, Qwen3-4B-Base rises from 22.22% to 46.97%, and post-trained Qwen3-4B rises from 32.32% to 51.01% (Song et al., 13 May 2026).

6. Reliability, calibration, and broader research directions

A recurring concern is whether fine-tuning damages general competence outside the fine-tuning distribution. The calibration study argues that, for classifiers fine-tuned on a subset of classes, the main failure mode is not degraded features or lost inter-class relationships, but a discrepant logit scale between fine-tuning classes and absent classes. The proposed post-processing rule adds a constant bias $N=358$9 to all absent-class logits at inference,

$W_0 \in \mathbb{R}^{d\times k}$0

and the paper reports large recoveries in absent-class accuracy, including ImageNet-Variants Acc_{U/Y} from 3.5% after fine-tuning to 54.0% with $W_0 \in \mathbb{R}^{d\times k}$1, Office-Home from 22.4% to 63.1%, and VTAB from 39.1% to 67.8% (Mai et al., 2024).

Other reliability mechanisms operate on the query itself rather than on logits. The three-stage misleading-query method explicitly trains detection and correction stages because existing methods often focus only on correcting the output. ReRAG reduces hallucinations by retrieving only when needed and filtering retrieved chunks with IsRel, while QueST treats the query as a source of supervision and adapts the model to its reasoning structure during inference. A plausible implication is that fine-tuned model querying has evolved from a single supervised mapping to a layered control problem involving input auditing, selective retrieval, execution validation, and per-query adaptation (Li et al., 15 Apr 2025, Anaissi et al., 2024, Song et al., 13 May 2026).

The literature also records clear limits. The misleading-query work notes reliance on the quality of internal or external knowledge, experiments limited to models up to 14B, and relatively small training data of 8 K samples. TabPFNv2 is reported to improve on almost all tasks up to 50K objects, but to be less stable under gradual temporal shifts and rich feature sets. M-DESIGN depends on benchmark coverage, and its Bayesian sliding window and OOD threshold require tuning, although the graph instantiation reports that it delivers the optimal model in 26 of 33 data-task pairs within limited budgets and keeps SQL joins and Bayesian updates below 0.31 s per iteration, or below 0.44 s with OOD adaptation (Li et al., 15 Apr 2025, Rubachev et al., 10 Jun 2025, Wang et al., 21 Jul 2025).

Taken together, these works indicate that fine-tuned model querying is not a single technique but an architectural pattern. The query may be answered directly, rewritten, calibrated against absent classes, translated into code or SQL, augmented with retrieved evidence, or used to trigger test-time self-training. What remains constant is the attempt to align model behavior with the structure of the query while preserving efficiency through PEFT, quantization, selective retrieval, or lightweight post-processing (Fasha et al., 24 Jan 2026, Anaissi et al., 2024, Mai et al., 2024).