NeurIDA: In-DB Analytics & Neuron Identification

• NeurIDA is a dual-purpose framework integrating dynamic in-database machine learning for optimizing analytics pipelines and automatic neuron type identification for connectomics.
• It employs LLM-driven query parsing, relational graph modeling, and dynamic fusion to enhance predictive accuracy over complex, multi-table datasets.
• In neuroanatomy, the system leverages 3D skeletonization and affinity propagation clustering to achieve high precision in neuron type classification.

NeurIDA denotes distinct systems in computational neuroscience, machine learning, and database analytics. Most prominently, "NeurIDA" refers to a dynamic in-database analytics framework integrating ML with relational database management systems (RDBMS), as well as a prior system for automatic neuron type identification in neuroanatomy research. This summary provides a rigorous exposition of both usages, focusing on core algorithms, architectures, evaluation regimes, and practical implications.

1. Dynamic Modeling for In-Database Analytics

NeurIDA introduces an autonomous system to reconcile the static nature of traditional ML models with the inherently dynamic query workloads in production RDBMS. The motivation stems from the observation that analytics over relational data require recurrent construction of bespoke ML pipelines, involving data extraction, transformation, and model retraining for each new task, incurring substantial overhead and inhibiting widespread adoption of ML within databases (Zeng et al., 9 Dec 2025).

System Architecture

NeurIDA consists of four principal modules executed fully within the database engine:

1. Query Intent Analyzer: Utilizes LLMs to translate natural language analytical queries (NLQs) into well-structured "task profiles" (target table, column, task type) and "data profiles" (relevant tables, join keys, predicate SQL fragments).
2. Conditional Model Dispatcher: Maintains exponentially moving average (EMA) performance metrics for a pool of heterogeneous pre-trained base models (e.g., FT-Transformer, ARM-Net, Trails). For a new task, evaluates a zero-cost proxy score on a sample and selects whether to use the base model directly or to invoke augmentation with the NeurIDA dynamic-modeling engine.
3. Dynamic In-Database Modeling Engine (DIME): Dynamically assembles and fine-tunes task-appropriate model components (see below), leveraging relation-aware GNNs and self-attention modules to capture inter-table dependencies.
4. Analytical Report Synthesizer: Deploys LLM-based agents to generate human-readable reports that summarize model results (charts, feature importances, statistics).

Base Model Pool and Augmentation

NeurIDA maintains a composable architecture:

• Base Table Embedding: Each tuple $x = (x_1, ..., x_M)$ is embedded as

$$e_i = \begin{cases} E_i[x_i] & \text{(categorical)} \ x_i \cdot \hat{e}_i + b_i & \text{(numerical)} \ \mathrm{Linear}(\mathrm{LM}(\text{col\_name} \parallel x_i)) & \text{(text)} \ \mathrm{Time2Vec}(x_i) & \text{(timestamp)} \end{cases}$$

followed by multi-head self-attention for contextual tuple representation.

• Dynamic Relation Modeling: Represents the extracted data slice as a heterogeneous graph $G = (V, E, R)$, where each node $v$ receives iterative updates over relation types $r$ and layers $\ell$:

$$s_{v, r}^{(\ell)} = \mathrm{AGG}(\{h_u^{(\ell)} : (u, v, r) \in E\}),\quad h_{v, r}^{(\ell+1)} = W_r^h\,h_v^{(\ell)} + W_r^s\,s_{v, r}^{(\ell)}$$

• Dynamic Model Fusion: Aggregates relational context signals and fuses them with the node embedding via a task-specific multi-head self-attention, yielding the final feature $z_v$.
• Task-Aware Prediction: Task-specific heads (e.g., logistic regression, linear regression) operate on the fused embedding, using loss functions tailored to task type (binary cross-entropy, MAE).

Online Task Handling

Upon query, if no base model's proxy score on the new data slice exceeds a threshold, DIME is invoked for further augmentation. The pipeline avoids extract/export steps—data movement and augmentation operate natively as user-defined functions and stored procedures within the RDBMS. Output is materialized as SQL tables or views, with analytical reports serialized as JSON/Latex.

2. Natural Language Analytics and LLM Integration

A hallmark of NeurIDA is the deep integration of LLM-based agents into both query parsing and report synthesis:

• Parsing: LLMs formalize user NLQ as task/data profiles checked for schema consistency.
• Report Generation: Generated outputs (interpretability highlights, error analysis, statistical summaries) are constructed by a chain-of-thought LLM process, providing interpretability aligned with the database's schema and results.

3. Experimental Protocols and Quantitative Results

NeurIDA is evaluated across five multi-table, multi-relation real-world benchmarks (e.g., Event, Beer, Trial, Avito, HM datasets). Tasks cover both classification (AUC-ROC) and regression (MAE). Baselines comprise strong static models (LR, RF, LightGBM, FT-Transformer, TabPFN, TabICL) and AutoML systems.

Key findings:

• NeurIDA yields up to 12% AUC-ROC increase and 10–25% MAE reduction over the strongest base model alone.
• The relational augmentation is especially effective for language-model-based tabular learners (TP-BERTa AUC 0.55→0.77).
• Ablation removing Dynamic Relation Modeling accounts for up to 60% performance loss; eliminating fusion layers degrades 20–30% of the gain.

Datasets/Tasks	Model Family	Representative Gain (AUC/MAE)
Event, Beer, etc.	Tabular LMs	up to +22 AUC, -18% MAE
Avito, HM	FT-Transformer	+8 AUC, -13% MAE
All	NeurIDA (full)	+12% AUC, -25% MAE

Overhead includes an additional 0.7–3M parameters, and 1.2–1.7× inference latency. No external ETL/extraction is required; all computation runs in-DB.

4. Application Domains, Practical Impact, and Constraints

NeurIDA is designed for environments where predictive signal emerges from complex join patterns—e-commerce, clinical trials, digital advertising among them. The system enables rapid deployment over evolving or ad-hoc relational schemas, supporting instantaneous analytics such as churn prediction or sales forecasting initiated via English queries.

Current limitations include:

• Only classification/regression tasks are supported; extensions to ranking and sequence forecasting are under development.
• Overhead in parameter size and inference time, which may be mitigated using quantization or sparse graph neural networks.
• Dependence on consistent and well-engineered prompts for LLM query interpretation; major schema changes require prompt/model adaptation.
• Extreme corner cases in NLQ parsing or complex schemas may reduce reliability.

5. NeurIDA for Automatic Neuron Type Identification

A distinct application under the same acronym, NeurIDA is also an established pipeline for high-throughput clustering of neuron types based on neurite localization, with a focus on Drosophila medulla connectomics (Zhao et al., 2014). This pipeline comprises:

• 3D Skeletonization: Reconstructs neuron shapes as volumetric graphs via TEASAR algorithm, post-processing for disconnected fragments (minimum-spanning tree over centroids).
• Location-Sensitive Similarity: Defines a deformation-compatible similarity $S(\mathcal{N}_1, \mathcal{N}_2)$ based on 1D projection of branch density along the medulla's columnar axis, with DP sequence alignment, tangential-to-columnar calibration by a log-normal mixture model, and normalization for size bias.
• Affinity Propagation Clustering: Operates on similarity matrices, obviating pre-specification of cluster number. Exemplar selection maximizes intra-cluster affinity.
• Evaluation: On a dataset of 379 Drosophila medulla neurons, the method achieved 91% accuracy for supervised type-calling (1-NN leave-one-out), 74.5% average unsupervised precision, and 72.3% recall. Visualizations via Laplacian Eigenmaps recover medulla layer structure.

Principal limitations include reliance on rigorous neuron registration to a common frame, reduction of 3D data to 1D (coding only depth/columnar span and global tangential:columnar ratio), and sensitivity of affinity propagation to parameters and rare cluster absorption. The system demonstrates utility in error detection (e.g., partial reconstructions mapping to correct cluster types).

6. Comparative Perspective and Terminological Distinctions

The NeurIDA moniker encompasses unrelated yet technically sophisticated systems: a dynamic in-database modeling suite (Zeng et al., 9 Dec 2025), and a neuromorphological clustering pipeline (Zhao et al., 2014). There is no evidence of overlap in technical basis, implementation, or application domain between the two, apart from a shared ambition to automate analytics for high-complexity relational or structural data. Each system sets state-of-the-art benchmarks within its respective context.

7. Related Methodologies

NeurIDA's dynamic modeling strategy may be contrasted with static approaches (TabPFN, TabICL) and prior efforts in learning from structured relational data in-database. Its composable architecture leverages both traditional ML (DNN, GNN, LightGBM) and table-reasoning LLMs (TP-BERTa, Nomic), with relational graph augmentation shown to be essential for extracting predictive signal within multi-table environments. In connectomics, the neurite-localization approach under NeurIDA offers a location- and shape-aware alternative to earlier purely topological or geometric clustering methods.

References: (Zeng et al., 9 Dec 2025, Zhao et al., 2014)