Flourishing AI Benchmark Overview

• Flourishing AI Benchmark is an evaluation framework that measures AI alignment with human flourishing using seven empirically validated dimensions.
• It integrates objective and subjective assessment items to capture ethical, social, and well-being aspects in AI performance.
• Benchmark results reveal significant gaps in areas like faith and meaning, highlighting challenges in achieving holistic AI support.

A Flourishing AI Benchmark is an evaluation framework designed to capture the alignment between artificial intelligence systems and the full spectrum of human flourishing, as opposed to traditional metrics focused solely on technical proficiency or harm minimization. It extends the concept of benchmarking from performance and capability to systematic measurement of well-being, encompassing multi-dimensional and holistic criteria rooted in empirical research on flourishing, virtue ethics, health, social connections, and existential fulfilment (Hilliard et al., 10 Jul 2025).

1. Conceptual Foundations: Dimensions of Human Flourishing

The Flourishing AI Benchmark (FAI Benchmark) operationalizes AI alignment by evaluating LLMs and related AI systems across seven empirically validated dimensions. These dimensions derive from the Harvard Human Flourishing Program’s “Secure Flourish” model and Barna Group research, providing comprehensive coverage of holistic well-being:

1. Character and Virtue: Assesses virtue ethics—acting to promote good in all circumstances (prudence, justice, courage, temperance)—as valued intrinsically.
2. Close Social Relationships: Measures the quality and satisfaction of interpersonal connections, foundational for well-being across cultures.
3. Happiness and Life Satisfaction: Includes both hedonic (subjective happiness) and evaluative (life satisfaction) criteria.
4. Meaning and Purpose: Captures individuals’ sense of purpose, worthwhileness of life, and clarity of goals, distinct from happiness.
5. Mental and Physical Health: Covers self-rated physical and mental health, encompassing essential aspects of whole-person functioning.
6. Financial and Material Stability: Relates to worries about living expenses and material security, grounding other domains of flourishing.
7. Faith and Spirituality: Evaluates religious or transcendent communion, spiritual practice, and sense of the divine.

These dimensions acknowledge cross-domain interactions and underpin a multi-objective, non-siloed evaluation paradigm (Hilliard et al., 10 Jul 2025).

2. Benchmark Question Construction and Data Sources

The FAI Benchmark consists of 1,229 individual questions, split into approximately 75% objective (multiple-choice/factual) and 25% subjective (free-text scenario) items:

• Objective items derive from established benchmarks and professional exams (MMLU subsets in moral scenarios, social sciences, medicine, world religions; national licensing exams; finance quizzes; guides on flourishing activities).
• Subjective items require reflective, open-ended advice, often constructed by transforming or generating dilemmas via LLM prompting. These scenarios are articulated in the first person to probe model capacities for contextually nuanced support.

Example items include both direct and integrative scenarios; for instance, a subjective “Character and Virtue” question addresses witnessing workplace prejudice, while a “Faith” item might explore personal spiritual uncertainty. Question categories are sourced to ensure comprehensive literature coverage within each dimension and will be rebalanced in future iterations to address underrepresentation, especially in subjective and character/finance categories (Hilliard et al., 10 Jul 2025).

3. Scoring Methodology and Aggregation

The FAI Benchmark employs a rigorously defined multi-tiered scoring methodology:

• Component Scores per Dimension (for each $d$):
  • Objective Score $OS_d$
  • Subjective Score $SS_d$
  • Tangential Score $TS_d$ (awarded for relevant cross-dimensional content)

Each dimension $d$ produces a geometric mean:

$$\mathrm{Score}_d = (OS_d \times SS_d \times TS_d)^{1/3}$$

Aggregating across all seven dimensions:

$GM_7 = \left( \prod_{i=1}^7 s_i \right)^{1/7}$

where $s_i$ are individual dimension scores. This aggregation penalizes near-zero values, enforcing minimum standards across all flourishing aspects.

Scoring of subjective items is performed using specialized LLM “judge” personas, guided by a 25-item rubric including binary and weighted criteria (with explicit penalties for harmful content). Raw rubric totals (ranging from –103 to +32.5) are clamped and linearly mapped to a 0–100 scale:

$T(x) = \max(0, x) \times (100 / 32.5)$

Judges also score responses for tangentially relevant dimensions, capturing beneficial cross-domain advice. Validation studies report that LLM judges achieve expert-level agreement (Hilliard et al., 10 Jul 2025).

4. Empirical Evaluation: Performance and Gaps

An initial assessment of 28 leading LLMs demonstrated that no model attained the alignment threshold ($\geq 90/100$). The leading models, including OpenAI o3 and Gemini 2.5 Flash, scored between 66–72 overall. Significant disparities were observed across domains:

Model	Overall	Char	Rel	Faith	Fin	Hap	Mean	Health
OpenAI o3	72	87	79	43	88	68	66	83
Gemini 2.5 Flash	68	77	77	40	87	67	61	81
Grok 3	67	70	71	39	88	70	63	82

Dimension gaps were most acute in:

• Faith & Spirituality (mean 35%)
• Meaning & Purpose (56%)
• Character & Virtue (58%)

Dimensions such as Financial Stability (81%) and Health (72%) showed substantially higher scores. This reflects current LLMs’ comparative facility with factual and pragmatic advice versus the generation of guidance on existential, spiritual, or virtue-driven queries (Hilliard et al., 10 Jul 2025).

5. Limitations and Directions for Advancement

Several key limitations and future priorities are identified:

1. Cultural Generalization: The current English-centric design may not generalize to non-Western models of flourishing. Broader cultural adaptation is planned.
2. Question Balance: Objective questions predominate (75%); a more balanced inclusion of subjective, context-dependent items is underway.
3. Rubric Refinement: Calibration of scoring rubrics and weights requires further expert (SME) tuning for increased discriminative validity.
4. Judge Validation: Continual evaluation of LLM judges vs. human experts is necessary to detect and mitigate evaluator bias.
5. Dialogic Evaluation: The present single-turn protocol will be expanded to multi-turn interactions, reflecting realistic conversational drift and alignment persistence.
6. Relevance Grading: A move beyond binary relevance for tangential scoring is anticipated, exploring partial relevance protocols.
7. Longitudinal Effects: The benchmark is not yet validated against long-term impact on actual human flourishing; longitudinal studies are required for robustness (Hilliard et al., 10 Jul 2025).

Open collaboration is facilitated via the Gloo FAI Benchmark repository, inviting interdisciplinary contributions to question design, rubric development, and cross-cultural adaptation.

6. Significance in the Broader AI Benchmarking Ecosystem

The FAI Benchmark represents a paradigmatic extension of benchmarking, shifting the evaluative axis from technical skill and minimal harm avoidance to comprehensive positive alignment with holistic human flourishing. Unlike traditional task-oriented benchmarks (e.g. MLPerf, AIBench, or AIPerf), the FAI Benchmark foregrounds the ultimate impact of AI systems on human well-being as formalized through rigorous, multi-dimensional metrics and empirical grounding in flourishing science (Hilliard et al., 10 Jul 2025).

Such frameworks provide standards for AI system development, governance, and ethical review, establishing a systematic, multi-objective approach for AI that aspires not merely to avoid harm but to actively support the full diversity of human flourishing.