SEQ-GPT: Exemplar-Driven Query System

• SEQ-GPT is an exemplar-driven framework that uses multiple semantically rich examples to enable joint retrieval and reasoning in spatial, biological, and sequence contexts.
• It employs a hybrid client-server architecture integrating specialized LLM modules with structured data sources, providing dynamic dialogue and multimodal query conversion.
• Empirical benchmarks demonstrate its robust performance, rapid response times, and high accuracy in spatial layout, RNA, protein, and DNA analysis.

A SEQ-GPT system is a LLM-powered architecture for performing complex spatial, biological, or sequence-based queries using examples rather than single-point constraints. The distinguishing feature of the SEQ-GPT paradigm is its focus on "exemplar-driven" query formulation, where the input consists of multiple semantically and structurally rich user-specified examples. This design supports joint retrieval or reasoning tasks that require matching latent, multi-faceted patterns, such as spatial layouts, molecular properties, or biological encoding, drawing on LLMs' capacity for interactive dialogue and flexible multimodal adaptation (Lim et al., 14 Aug 2025, Xiao et al., 2024, Xiao et al., 2024, Zhang et al., 2023).

1. Formalization of Spatial Exemplar Queries

SEQ-GPT is exemplified by the "Spatial Exemplar Query" (SEQ) framework. In this setting, the user provides $k$ exemplar locations:

$E = \{e_1, \ldots, e_k\}$

Each exemplar $e_i$ is described by:

• $A(e_i)$: Attributes (e.g., "gym", "hotel")
• $p(e_i) \in \mathbb{R}^2$: Euclidean coordinates
• $d(e_i, e_j) = \|p(e_i) - p(e_j)\|_2$: Pairwise distances

The objective is to retrieve a candidate set of $k$ locations $X = \{x_1, \ldots, x_k\}$ from a target area, optimizing a similarity (or cost) function that matches both attributes and spatial relations:

$$S(E, X) = \alpha \cdot \sum_{i=1}^k \operatorname{sim}_{\text{attr}}\left(A(e_i), A(x_i)\right) + \beta \cdot \sum_{1 \leq i < j \leq k} \left| d(e_i, e_j) - d(x_i, x_j) \right|$$

where $\operatorname{sim}_{\text{attr}}$ is an attribute-category similarity (e.g., embedding or one-hot), $E = \{e_1, \ldots, e_k\}$0 are weights. The system solves:

$E = \{e_1, \ldots, e_k\}$1

This paradigm generalizes to other sequential or multimodal domains, e.g., biological sequences, by replacing spatial attributes and distances with appropriate domain-specific similarity structures (Lim et al., 14 Aug 2025).

2. System Architecture and LLM Orchestration

SEQ-GPT instantiates a hybrid client-server architecture, integrating multiple specialized LLM modules with structured data sources. The principal components include:

• Front End (React client):
  • Map Mode: Users select exemplars directly on an interactive map.
  • Chat Mode: Users specify queries in natural language or voice.
  • Renders query suggestions, result maps with similarity scores, and maintains dialogue history.
• Back End (Gin web server):
  • Spatial Database API: Issues SEQ searches over public map APIs, uses caching.
  • LLM Orchestration Layer:
  • Coordinates distinct LLMs for data parsing and dialogue management.
  • Implements a token-level state transition protocol for multi-model switching.
  • LLM Modules:
  • Generator LLM (frozen): Used for offline synthetic dialogue generation.
  • Chat LLMs (fine-tuned):
  • Data Alignment Model: Structures natural input or map clicks into formal SEQ queries.
  • Dialogue Manager: Maintains state, handles clarification, and session flow.
  • Multi-Model Cooperation: Determined by a state token emitted at each turn, which guides routing between LLMs.
  • (Lim et al., 14 Aug 2025, Xiao et al., 2024, Xiao et al., 2024).

3. LLM Adaptation Pipeline and Training

SEQ-GPT operationalizes LLMs via a tailored adaptation and training pipeline:

• Synthetic Dialogue Synthesis: Constructs a directed transition graph of dialogue states (31 in SEQ-GPT), uses a SynTOD-style random walker to sample realistic dialogue flows. GPT-4o (frozen) produces utterances and structured annotation per state, yielding state-annotated (dialogue, query) pairs.
• Supervised Fine-Tuning: Chat LLMs (GPT-4o-mini, LLaMA-3-8B-Instruct) are trained (2,000 samples, 3 epochs) and selected based on self-BLEU (diversity ≈ 0.28), intent accuracy (≈ 96%), and slot F$E = \{e_1, \ldots, e_k\}$2 (≈ 94%).
• Multi-Model Orchestration: Special tokens (e.g., "<STATE_i>") signal transitions. The orchestrator either retains the current model or switches to alternate parsing/dialogue roles (Lim et al., 14 Aug 2025).

The RNA-GPT and ProteinGPT frameworks extend this by employing domain-specific encoders (e.g., RNA-FM, ESM-2, Inverse-Folding) with linear adapters, aligning their embeddings to LLM spaces, and leveraging two-stage alignment plus instruction tuning over large auto-curated QA datasets (Xiao et al., 2024, Xiao et al., 2024).

4. Core SEQ Search Algorithm and Multimodal Extension

The SEQ-GPT search engine implements a ranked retrieval over possible candidate k-tuples matching user-provided exemplars:

1. Retrieve all candidates $E = \{e_1, \ldots, e_k\}$3 matching the attributes in the specified area.
2. For each $E = \{e_1, \ldots, e_k\}$4-tuple $E = \{e_1, \ldots, e_k\}$5, compute $E = \{e_1, \ldots, e_k\}$6.
3. Return the top-N with lowest cost.

Efficient execution employs k-nearest neighbor (per exemplar) filtering and combinatorial pruning, supported by caching repeated results.

In multimodal biological settings (e.g., ProteinGPT), encoding pipelines may involve both sequence (FASTA) and structure (PDB) representations. Each is embedded, projected, and concatenated to form a "soft prompt" embedding for the LLM, which enables joint inferencing or response generation conditioned on the full set of multimodal features (Xiao et al., 2024, Xiao et al., 2024).

SEQ-GPT Variant	Input Modalities	Domain-Specific Encoder(s)	Query Structure
Spatial SEQ-GPT	Text, map clicks (locations)	None (uses attributes)	Exemplars + spatial graph
ProteinGPT	Protein sequence, structure	ESM-2, Inverse-Folding	Sequence/structure fusion
RNA-GPT	RNA sequence	RNA-FM	Sequence-to-soft prompt
DNAGPT	DNA sequence, numeric tokens	Custom GPT	K-mer, instruction tokens

5. Interactive Operations and User Scenarios

SEQ-GPT systems support advanced user interactions and feedback mechanisms:

• Clarification Dialogues: LLM dynamically requests missing details or refines exemplar sets based on context (e.g., asks for the name and distance for a hotel if not specified).
• Dynamic Feedback: On assembling exemplars, the model solicits further constraints such as search area, and confirms settings before search execution.
• Mode Conversion: Seamlessly transitions between modes; e.g., converting graphical selections into textual dialogue for downstream parsing and display.
• Session Latency: End-to-end response times (utterance to map) are reported under 1.5 s on a single GPU, demonstrating practical usability (Lim et al., 14 Aug 2025).

6. Evaluation, Benchmarks, and Empirical Results

Evaluation of SEQ-GPT centers on dialogue and extraction fidelity, search quality, and latency:

• Dialogue Model Metrics: self-BLEU ≈ 0.28 (response diversity), intent prediction ≈ 96%, slot extraction F$E = \{e_1, \ldots, e_k\}$7 ≈ 94%.
• User Scenario Validation: E.g., a tourist scenario with three exemplars (“Suntec City”, “Anytime Fitness City Hall”, and “hotel within 200 m”, area “downtown Sydney”) converges in 4–5 turns, retrieving and visualizing the five best triplets.
• Large-Scale Modality Alignment Benchmarks:
  • RNA-GPT: On a held-out 5k subset, F$E = \{e_1, \ldots, e_k\}$8 = 0.8494, ROUGE-L = 0.4747 (doubling ablation baselines) (Xiao et al., 2024).
  • ProteinGPT: ROUGE-Lsum up to 0.457 (Mistral); semantic F$E = \{e_1, \ldots, e_k\}$9 up to .829 (S_BERT) (Xiao et al., 2024).
  • DNAGPT: SOTA performance in GSR, mRNA regression, and synthetic genome generation; e.g., Human-TIS(ATG) classification accuracy 98.02% (Zhang et al., 2023).

7. Broader SEQ-GPT Paradigm and Domain-Specific Implementations

The formal SEQ-GPT template—exemplar-driven query structuring, multimodal LLM alignment, dual-phase fine-tuning—has been generalized beyond spatial search. RNA-GPT integrates sequence encoders with LLMs via learned projection, using multi-million sample QA datasets for instruction tuning. ProteinGPT fuses sequence and structural modalities, matching each to the LLM input through parallel adapters, achieving superior response generation and protein property understanding. DNAGPT extends the paradigm to DNA, employing granular k-mer tokenization, joint sequence–numeric representation, and triple-loss objective for multi-task DNA analysis. Each instantiation maintains the fundamental architectural and training blueprints originally articulated in the reference SEQ-GPT design (Lim et al., 14 Aug 2025, Xiao et al., 2024, Xiao et al., 2024, Zhang et al., 2023).