Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices

Abstract: As algorithmic decision-makers are increasingly applied to high-stakes domains, AI alignment research has evolved from a focus on universal value alignment to context-specific approaches that account for decision-maker attributes. Prior work on Decision-Maker Alignment (DMA) has explored two primary strategies: (1) classical AI methods integrating case-based reasoning, Bayesian reasoning, and naturalistic decision-making, and (2) LLM-based methods leveraging prompt engineering. While both approaches have shown promise in limited domains such as medical triage, their generalizability to novel contexts remains underexplored. In this work, we implement a prior classical AI model and develop an LLM-based algorithmic decision-maker evaluated using a large reasoning model (GPT-5) and a non-reasoning model (GPT-4) with weighted self-consistency under a zero-shot prompting framework, as proposed in recent literature. We evaluate both approaches on a health insurance decision-making dataset annotated for three target decision-makers with varying levels of risk tolerance (0.0, 0.5, 1.0). In the experiments reported herein, classical AI and LLM-based models achieved comparable alignment with attribute-based targets, with classical AI exhibiting slightly better alignment for a moderate risk profile. The dataset and open-source implementation are publicly available at: https://github.com/TeX-Base/ClassicalAIvsLLMsforDMAlignment and https://github.com/Parallax-Advanced-Research/ITM/tree/feature_insurance.

• The paper demonstrates that both approaches achieve similar overall alignment accuracy but vary in consistency across different risk tolerance profiles.
• It details a classical AI model using Monte Carlo simulation and Bayesian reasoning alongside an LLM-based model employing zero-shot prompting and weighted self-consistency.
• Findings suggest that integrating structured data into LLM pretraining could improve alignment in moderate-risk decision scenarios.

Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices

This paper investigates the capabilities of classical AI and LLM-based decision-makers (DMs) in aligning their outputs with human decision-makers' preferences, specifically in the health insurance domain. The study evaluates two approaches: a classical AI model utilizing structured reasoning techniques and an LLM-based model leveraging zero-shot prompting and weighted self-consistency using GPT-4 and GPT-5.

Background and Methodology

Decision-Maker Alignment (DMA)

DMA aims to tailor AI system outputs to reflect human cognitive attributes such as risk tolerance, thereby accounting for the variability in human decision-making under uncertainty. Traditionally, classical AI methods have employed case-based reasoning, Bayesian inference, and other structured approaches to achieve alignment.

Classical AI Approach

The classical AI DM employs a model developed by Molineaux et al., integrating methods like Monte Carlo simulation and Bayesian reasoning to emulate human decision-making patterns. This model constructs a case base with decision outcomes linked to decision-maker attributes, allowing alignment across scenarios with different risk levels.

LLM-Based Approach

The LLM-based DM uses methods from Hu et al., which include zero-shot prompting and weighted self-consistency sampling. The system queries LLMs (GPT-4 and GPT-5) with structured prompts designed to align decisions with specific target profiles characterized by varying risk tolerance levels (Alex: 0, Brie: 0.5, Chad: 1.0).

Figure 1: Schematic overview of the dataset structure with example probes, contextual attributes of the decision-maker, a target decision maker attribute, risk tolerance, and four available choices. The ground truth indicates the most aligned option.

Dataset

The study utilizes a health insurance dataset comprising cost-related decision-making probes. Probes include contextual features like family composition, medical history, and lifestyle factors, contributing to decision-maker behavior characterization. The dataset defines decision-maker attributes of risk tolerance, used to evaluate models across multiple scenarios.

Results and Comparison

Performance Evaluation

The models were evaluated against three synthetic targets across multiple probes, emphasizing varying risk tolerance levels. A key finding was the parity in overall alignment accuracy between classical AI and LLM-based approaches, although the classical AI DM showed more consistent performance across different risk profiles.

LLM-Based Results

While both GPT-4 and GPT-5 demonstrated near-identical overall accuracy, suggesting less dependency on deep reasoning capabilities, their performance for moderately risk-averse contexts (target: Brie) was less effective. This highlights a limitation in handling scenarios that fall between extremes, likely due to inherent limitations of natural language in expressing nuanced cognitive constructs.

Figure 2: Implementation of classical AI and LLM-based algorithmic DMs, where both models receive a scenario probe and output a final decision. The final decision must be aligned with the decision made by a target decision-maker of the same risk level.

Figure 3: Performance of the three models across three targets with varying risk tolerances (Alex: 0, highly risk-averse; Brie: 0.5, moderately risk-averse; Chad: 1.0, risk-tolerant). Bars indicate individual target alignment accuracy, and the legend denotes the model.

Challenges and Implications

The principal challenge in comparing these approaches arises from the disparity in input types and the resultant model expectations. While classical AI models offer stability and granularity, especially in scenarios with varying cognitive traits, LLM-based models need robust prompting to leverage pretrained knowledge effectively. This suggests a potential benefit in augmenting LLM training with structured data to bridge expressive gaps, particularly in nuanced decision contexts.

Conclusion

The research underlines the complementary strengths of classical AI and LLM-based DMs, suggesting that future studies could integrate structured data into LLM pretraining to better accommodate moderate-risk scenarios. Further exploration into prompting strategies and fine-tuning may enhance LLMs' capacity to model diverse decision-making attributes accurately, thereby broadening the applicability of DMA across various domains.