DatologyAI Models: Data-Centric ML

• DatologyAI Models are data-centric machine learning frameworks that explicitly model how individual data points drive predictions, enhancing interpretability and counterfactual analysis.
• They utilize methodologies such as subset experiments, LASSO-regularized regression, and LLM-based feature extraction to produce actionable insights from complex datasets.
• Applications span dataset debugging, dynamic model assembly for relational data, and compressing AutoML ensembles into efficient, production-ready artifacts.

DatologyAI Models are a class of data-centric machine learning frameworks and model architectures that prioritize the central role of data in shaping model predictions, interpretability, and selection. These approaches move beyond the traditional emphasis on neural parameterization or opaque black-box architectures, instead favoring explicit modeling of sample-to-prediction relationships, data-driven feature extraction, and dynamic adaptation to specific analytical tasks, particularly in complex and relational domains.

1. Foundational Concepts and Formal Definitions

DatologyAI Models emerged from the recognition that many challenges in model robustness, interpretability, and deployability hinge not merely on architectural advances but on understanding and parameterizing how the training data itself drives predictions. A formative example is provided by the "datamodel" framework of Ilyas et al. (Ilyas et al., 2022), which formalizes a datamodel for a target example $x$ as a parameterized function

$g_\theta: \{0,1\}^d \to \mathbb{R},$

mapping a binary indicator of training subset membership to the model’s outcome on $x$, trained via empirical risk minimization over random training subsets. Here, for any subset $S' \subset S$ of the training set, $f(x; S')$ denotes the prediction for $x$ after model training on $S'$.

A datamodel thus explicitly models the causal effect of individual data points or data subsets on specific model outputs, supporting detailed empirical analysis of prediction mechanisms, data counterfactuals, and dataset composition effects.

2. Core Methodologies and Architectures

The methodological axis of DatologyAI encompasses several distinct, but complementary, modeling and algorithmic strategies:

Linear Datamodels via Subset Experiments

The baseline datamodel conducts large-scale experiments by repeatedly sampling subsets $S_i$ of the training data at fixed fractions $\alpha$ and measuring the effect on model outputs. The parameters $\theta$ are typically learned via LASSO-regularized regression:

$$\theta = \arg\min_{w \in \mathbb{R}^d} \frac{1}{m} \sum_{i=1}^m (w^\top 1_{S_i} - f(x; S_i))^2 + \lambda \|w\|_1,$$

where $1_{S_i}$ is the subset indicator vector and $\lambda$ is a cross-validated regularization constant (Ilyas et al., 2022). The model is estimated over hundreds of thousands to millions of subset experiments.

Interpretable Feature Extraction with LLMs (DSAI)

The DSAI model advances DatologyAI to the latent feature discovery regime by imposing a multi-stage pipeline around an LLM (Cho et al., 2024):

1. Perspective Generation: The LLM proposes perspectives (e.g., “clarity,” “conciseness”) given a sample of positive and negative data points.
2. Perspective–Value Matching: The LLM rates each data point along each perspective, assembling a data matrix $v_{i,j}$.
3. Value Clustering: Cluster labels are generated and assigned, yielding interpretable feature categories.
4. Verbalization: Features are quantified via directional scores

$\delta_{j,k} = 2 P_{j,k} - 1,$

where $P_{j,k}$ is the empirical frequency of “positive” cases within each feature cluster.

1. Prominence-Based Selection: Features are filtered by a quantitative prominence threshold $\pi_{j,k} = |\delta_{j,k}|$, ensuring data-driven significance.

This pipeline is designed to eliminate reliance on LLM pre-trained biases, with all extracted features traceable to subsets of input data.

Distillation and Dynamic Modeling for Tabular and Relational Data

For tabular domains, DatologyAI supports distillation frameworks such as FAST-DAD (Fakoor et al., 2020), which compresses complex AutoML ensembles into efficient, interpretable models (e.g., trees, small NNs) using self-attention pseudolikelihood estimation and block-Gibbs-augmented data sampling.

In relational or dynamically queried settings, NeurIDA (Zeng et al., 9 Dec 2025) exemplifies a fully dynamic DatologyAI Model. At inference time, NeurIDA parses a user's natural language query, constructs a task and data profile with LLM agents, then dynamically assembles a relationally grounded model from a pre-trained backbone—comprising a unified tuple encoder, relation-aware message passing, and context-aware fusion layers—optionally fine-tuning or retraining components on the fly.

3. Theoretical Justification and Empirical Properties

DatologyAI Models are undergirded by several theoretical insights:

• Linear datamodels, despite the nonlinearity of the underlying models (e.g., deep neural networks), can match the optimal mean squared error up to SGD stochasticity, with correlations $r \approx 0.99$ between predicted and actual test outputs in large-scale experiments (Ilyas et al., 2022).
• The datamodel regression coefficients coincide, up to a scaling factor, with classical average treatment effect estimators (Ilyas et al., 2022).
• DSAI’s feature extraction pipeline quantitatively scores and ranks features, with recall of expert-defined criteria exceeding 75% at stringent prominence thresholds and Discriminative Power metrics $>0.6$ across applications (Cho et al., 2024).
• In dynamic relational modeling, jointly pre-trained modules generalize across tasks and schemas, and composable architectures enable consistent improvements in both classification (AUC-ROC gain up to 12%) and regression (MAE reduction up to 25%) without task-specific retraining (Zeng et al., 9 Dec 2025).

4. Large-Scale Experiments and Benchmarks

Extensive experimental validation across different DatologyAI paradigms highlights their scalability and robustness.

Table: Key Empirical Results

Model/Framework	Task Domain	Accuracy/Uplift	Reference
Linear Datamodel (CIFAR-10)	Vision (ResNet-9)	Pearson $r \approx 0.99$ (on-distribution)	(Ilyas et al., 2022)
DSAI	NLP, Classification	$>75\%$ recall (τ=0.692), DP $>0.6$	(Cho et al., 2024)
FAST-DAD	Tabular (AutoML)	$+0.7$ pp class gain, 10x faster inference	(Fakoor et al., 2020)
NeurIDA	Relational DB	$+$4–12% AUC; $-$10–25% MAE	(Zeng et al., 9 Dec 2025)

In vision, up to four million models were retrained over diverse subsets. In tabular and relational tasks, DatologyAI distillation reduces prediction latency (from $\sim$100ms to 1–5ms per row) and compresses model size by an order of magnitude, with minimal or negative loss in accuracy (Fakoor et al., 2020, Zeng et al., 9 Dec 2025). DSAI validated its pipeline on both synthetic (expert-annotated) and real-world datasets, outperforming or matching the recall of direct LLM or human enumeration baselines (Cho et al., 2024).

5. Practical Applications

DatologyAI Models demonstrate utility across a diverse array of analysis tasks:

• Dataset Debugging: Identifying influential, mislabeled, or spurious examples, including brittle sets whose removal alters model predictions (Ilyas et al., 2022).
• Model Interpretability: Explaining individual predictions in terms of causally relevant training examples or interpretable human-language features (Ilyas et al., 2022, Cho et al., 2024).
• Counterfactual Analysis: Predicting the effect of removing or altering specific examples directly via the linear surrogate (Ilyas et al., 2022).
• Feature Discovery and Auditing: Extracting actionable and interpretable data-driven features for enhanced model transparency and operational audits (Cho et al., 2024).
• Efficient Model Deployment: Shrinking complex AutoML ensembles into compact, production-ready artifacts with maintained or improved performance (Fakoor et al., 2020).
• Relational/Natural Language Analytics: Dynamically answering analytical queries over RDBMS, with model selection and assembly handled automatically in-database (Zeng et al., 9 Dec 2025).

6. Limitations, Extensions, and Positioning

While DatologyAI Models deliver measurable gains in interpretability, fidelity, and model selection, several limitations and open directions persist:

• In DSAI, interpretability depends on LLM quality and clustering, with discrete thresholds introducing heuristic choices (Cho et al., 2024).
• Linear datamodels, while effective, may underfit subtle nonlinearities in deeply overparameterized regimes; however, empirical results show near-optimal practical performance (Ilyas et al., 2022).
• Dynamic modeling and in-database assembly introduce moderate computational overhead (20–70% latency, up to 2× parameter count), though offset by gains in workflow automation and prediction quality (Zeng et al., 9 Dec 2025).

Possible extensions include weak supervision or reinforcement for perspective refinement, deployment to multimodal/multilingual settings, or hybrid schemes embedding datamodel-derived features into standard deep learning pipelines (Cho et al., 2024).

This suggests that DatologyAI Models function as a unifying framework for data-aware modeling, leveraging both explicit surrogate modeling and modern representation learning to deliver interpretable, dynamic, and data-grounded predictions across a broad spectrum of machine learning applications.

7. References and Research Landscape

• Ilyas, A., Park, S. M., Engstrom, L., Leclerc, G., & Madry, A. “Datamodels: Predicting Predictions from Training Data” (Ilyas et al., 2022).
• Zhang, J., et al. “DSAI: Unbiased and Interpretable Latent Feature Extraction for Data-Centric AI” (Cho et al., 2024).
• Nakkiran, P., et al. “Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation” (Fakoor et al., 2020).
• Zeng, B., et al. “NeurIDA: Dynamic Modeling for Effective In-Database Analytics” (Zeng et al., 9 Dec 2025).

DatologyAI Models have thus become central to the modern data-centric AI paradigm, providing principled, empirically validated pathways for understanding, interpreting, and harnessing the effects of data on model predictions in both classical and emerging ML scenarios.