Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning

Abstract: Machine learning for tabular data remains constrained by poor schema generalization, a challenge rooted in the lack of semantic understanding of structured variables. This challenge is particularly acute in domains like clinical medicine, where electronic health record (EHR) schemas vary significantly. To solve this problem, we propose Schema-Adaptive Tabular Representation Learning, a novel method that leverages LLMs to create transferable tabular embeddings. By transforming structured variables into semantic natural language statements and encoding them with a pretrained LLM, our approach enables zero-shot alignment across unseen schemas without manual feature engineering or retraining. We integrate our encoder into a multimodal framework for dementia diagnosis, combining tabular and MRI data. Experiments on NACC and ADNI datasets demonstrate state-of-the-art performance and successful zero-shot transfer to unseen schemas, significantly outperforming clinical baselines, including board-certified neurologists, in retrospective diagnostic tasks. These results validate our LLM-driven approach as a scalable, robust solution for heterogeneous real-world data, offering a pathway to extend LLM-based reasoning to structured domains.

• The paper introduces a schema-adaptive paradigm that leverages LLMs to convert heterogeneous tabular data into natural language, enabling zero-shot generalization across diverse clinical schemas.
• It integrates MRI image features through a gated transformer fusion mechanism and achieves significant improvements in diagnostic accuracy, surpassing both conventional models and expert performance.
• The approach demonstrates sample-efficient learning and interpretability via SHAP analysis, providing a scalable and clinically meaningful framework for multimodal reasoning.

Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning

Introduction and Motivation

The presented work proposes a schema-adaptive tabular representation learning paradigm that leverages pretrained LLMs for robust, generalizable clinical reasoning across heterogeneous, multimodal datasets. Addressing schema heterogeneity—a primary impediment to the scalability of AI in deployment settings—the framework recasts tabular data as semantically structured text, which is then encoded using a pretrained LLM. This approach obviates the need for manual feature harmonization and is explicitly designed for zero-shot transfer to unseen schemas, with key validation in dementia diagnosis using diverse EHR and imaging modalities.

Architecture Overview

The architecture comprises three principal modules: a schema-adaptive tabular encoder, an auxiliary modality encoder (here, a frozen Swin UNETR for MRI), and a gated transformer-based multimodal fusion mechanism with label-specific [CLS] tokens. The core innovation lies in the semantic tokenization of tabular variables, whereby each column-value pair is translated into a natural language statement, processed by an LLM embedding model. The resulting token sequence for tabular and image features is concatenated and processed by the fusion transformer for final multi-label predictive inference.

Figure 1: The end-to-end multimodal architecture, with schema-adaptive LLM tabular encoding and Swin UNETR for MRI, gated transformer fusion, and MGDA-based composite loss for robust multi-label classification.

Schema-Adaptive Tabular Encoder

Structured data features—categorical and numerical—are mapped into human-readable statements utilizing lightweight metadata augmentation and normalization. Categorical variables are rewritten with descriptive context before embedding, while numerical variables undergo normalization and are scaled against their semantic context embedding. Importantly, all features are embedded using the same pretrained LLM, ensuring schema-independent representations. The output is projected via a shared linear layer, constructing an expressive, semantically aligned token sequence irrespective of syntactic schema divergences.

This design underpins the model’s ability to achieve zero-shot schema generalization—the encoder is insensitive to superficial schema discrepancies and instead leverages the LLM’s semantic abstraction capabilities. This establishes a clear deviation from previous tabular models that fundamentally depend on strictly defined column semantics and order.

Multimodal Fusion and Optimization

The transformer fusion backbone concatenates language-derived tabular embeddings and image tokens, appending per-label [CLS] tokens. The model is optimized using a multi-objective strategy (MGDA) with $2L$ loss terms per batch: a focal loss for handling class imbalance and a supervised, label-aware contrastive regularizer for semantic space structuring. This ensures no single label or task dominates the shared parameter updates, maintaining equilibrium across correlated multi-label clinical targets.

Experimental Results

Zero-Shot Schema Generalization

The schema-adaptive LLM encoder achieves state-of-the-art zero-shot transfer. When trained only on the NACC dataset and evaluated without any fine-tuning on the schema-disjoint ADNI dataset, the method yields a macro AUROC of 0.727, dramatically surpassing both random and pretrained-but-not-contextualized baselines (0.513 and 0.611, respectively). This experimentally validates the claim that semantic tokenization using LLM priors enables meaningful feature alignment across disparate table schemas—an empirical property not previously attained by competing tabular or multimodal models.

In-Domain Multimodal Diagnosis and Expert Comparison

Within the training domain (NACC), the model not only exceeds conventional tabular architectures (e.g., TabPFN: AUROC 0.868) but also outperforms large-scale multimodal LLMs (e.g., Gemini-2.5: Balanced Accuracy 0.663). Notably, in a head-to-head comparison with 12 board-certified neurologists scoring a macro AUROC of 0.680, the schema-adaptive model achieves 0.904—a 32.9% improvement over human experts, especially pronounced on rare/complex diagnostic labels. This establishes the method’s clinical validity and its capacity to perform consistent, high-fidelity inference in imbalanced high-dimensional settings.

Figure 2: Diagnostic comparison between table-only and multimodal variants, showing the regularization benefit and convergence stability of unimodal table encoding.

Figure 3: Comparative performance for image-only versus multimodal training, illustrating overfitting with images alone and improved stability with tabular context.

Sample Efficiency and Low-Resource Adaptation

The model demonstrates robust low-shot learning: fine-tuning the multilingual encoder on as little as 300 samples leads to an ADNI AUROC of 0.9362, outperforming models trained from scratch even on the full dataset (AUROC 0.8943). This attests to the parameter efficiency and strong inductive bias conferred by language-grounded representations, holding particular relevance in medical applications with constrained datasets.

Ablations and Interpretability

Architectural ablations clarify that gains arise primarily from semantic representation, not model complexity—best results are obtained with a shallow (2-layer) transformer and simple projection, with diminishing returns for deeper modules.

Interpretability is addressed with SHAP analysis: for AD prediction, the model prioritizes medically coherent variables (e.g., seizure and Parkinson’s histories), and MRI-derived features are less prominent among the top attributions. This suggests that the framework’s learned representations encode clinically meaningful patterns, rather than spurious image-text correlations.

Figure 4: SHAP summary plot for AD prediction, highlighting the primacy of clinical tabular features in the model’s predictive logic.

Implications and Future Work

This work substantiates that LLM-driven tabular encoding offers a scalable, domain-agnostic pathway toward generalization across heterogeneous EHR, imaging, and potentially additional structured data modalities. From a theoretical perspective, recasting tabular learning as language modeling allows for the direct transfer of advances in PLMs and NLU, breaking the dependence on fragile, hand-engineered schema conventions.

Practically, these capabilities portend a future in which AI systems can be trained and validated on disparate sources, transferred without retraining, and achieved with minimal additional annotation—all with interpretable outputs. Remaining limitations include the reliance on well-annotated metadata (the encoder degenerates gracefully toward syntactic embeddings in cryptic or context-poor tables) and the dependency on upstream LLM representation quality and bias.

Potential future directions include:

• Automatic semantic enrichment of tables lacking descriptive metadata,
• Systematic evaluation of open-source versus proprietary LLMs for tabular feature extraction,
• Expansion to additional domains such as clinical time series, finance, or multi-institutional federated learning for data-private schema adaptation.

Conclusion

Schema-Adaptive Tabular Representation Learning, as demonstrated in this work, represents a significant advance in reconciling heterogeneous structured data with modern multimodal AI. By leveraging LLMs for schema-agnostic, semantically dense embedding, the approach enables robust zero-shot generalization, strong in-domain modeling, and sample-efficient transfer, with clear interpretive fidelity to clinical reasoning. This lays the groundwork for scalable, trustworthy multimodal AI systems, facilitating translation from research to practice across domains with complex and evolving data structures.

Reference: "Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning" (2604.11835)