TBJ: Evaluating AI with Truth, Beauty, and Justice

• Truth, Beauty, and Justice (TBJ) is an evaluative framework defining AI performance through accuracy, interpretability, and ethical responsibility.
• It categorizes evaluation into Truth (data integrity and model validity), Beauty (explainability and generative discovery), and Justice (fairness and societal impact).
• The framework guides human-AI collaboration by recommending balanced roles across planning, execution, and activation stages in dynamic VUCA environments.

Searching arXiv for the specified paper and a few directly related works mentioned in the provided data. Looking up the paper via the arXiv API. Using Python to query the arXiv API for bibliographic verification. $\hat{y}$2 Truth, Beauty, and Justice (TBJ) is a framework for evaluating AI, machine learning, and computational models in terms of effective and ethical use. In Timpone and Yang’s treatment, TBJ is applied to analytic, generative, and agentic AI across the data-science life cycle, with the central claim that AI can augment human analysts but should not be treated as a substitute for methodological understanding, domain expertise, and ethical judgment. The framework organizes evaluation around accuracy and robustness (“Truth”), interpretability and richness (“Beauty”), and fairness and societal responsibility (“Justice”), while embedding these dimensions in a human-machine collaboration perspective and in decision settings characterized as volatile, uncertain, complex, and ambiguous (VUCA) (Timpone et al., 15 Jul 2025).

1. Origins and conceptual orientation

TBJ is presented as an evaluative framework with antecedents in Lave and March (1993) and Taber and Timpone (1996), and is extended in the 2025 treatment to contemporary AI contexts. It is not introduced as a narrow benchmark or a single scalar metric. Rather, it functions as a structured lens through which data, models, outputs, and organizational practices can be assessed simultaneously for epistemic quality, interpretive adequacy, and ethical acceptability (Timpone et al., 15 Jul 2025).

Within this framing, AI is situated in the broader transformation of research workflows. The paper identifies uses of AI in constructing surveys, synthesizing data, conducting analysis, and writing summaries. However, the framework is explicitly deployed against the assumption that greater automation necessarily produces better research. The warning is historical as well as methodological: earlier increases in the ease of survey analysis through statistical software enabled analyses that some researchers did not fully understand, and the paper argues that newer AI systems may create analogous but larger risks.

The framework is also tied to Daugherty and Wilson’s spectrum of activities: human-only; human complements machine; machine gives superpowers to humans; machine-only. In Timpone and Yang’s account, TBJ supports calibrated allocation of work across this spectrum rather than indiscriminate automation. The emphasis on VUCA conditions further sharpens that claim. A plausible implication is that TBJ is intended not merely as a model-evaluation schema, but as a decision-governance schema for allocating epistemic and ethical responsibility under uncertainty.

2. Truth

Truth focuses on accuracy, reliability, validity, and overall robustness of data, models, and insights. In the paper’s exposition, this dimension includes three closely related components.

First, data integrity concerns correct sampling and the absence of data errors or mis-measurement. This places Truth upstream of modeling; validity failures can arise before any estimator or learning system is applied.

Second, model validity concerns the use of appropriate statistical or machine-learning methods, correct assumptions, and proper training and testing procedures. Truth therefore includes procedural soundness, not only predictive or inferential performance in the narrow sense.

Third, insight fidelity concerns summaries and visualizations that faithfully represent the underlying analyses. This extends the truth criterion into the communicative layer of data science. Outputs can fail Truth even when the underlying computation is technically correct if summarization distorts the substantive result.

The human role under Truth is defined in terms of deep domain expertise, critical thinking, and systematic validation. Human analysts are tasked with catching biases, logical inconsistencies, or blind spots in AI outputs. The framework therefore treats validation as an irreducibly socio-technical activity: model checking and output checking remain connected to substantive knowledge, not only to software execution.

3. Beauty

Beauty is defined more broadly than aesthetics. In the TBJ framework, it includes explainability, interpretability, richness and variance representation, and fertility and surprise.

Explainability refers to how clearly the system’s reasoning process, denoted as $\pi$, can be articulated to humans. This is a property of the relation between the system and its audience, not merely of internal architecture.

Interpretability refers to the ease with which outputs, denoted as $\hat{y}$, can be understood and their implications grasped. The focus is therefore on the analyst’s capacity to move from output to meaning.

Richness and variance representation concerns the degree to which analyses expose meaningful heterogeneity, represented as $\mathrm{Var}[X]$, and avoid oversimplification. This element is especially important because compressed or homogenized outputs may appear elegant while suppressing substantively important variation.

Fertility and surprise refers to the capacity to surface new patterns or hypotheses. Beauty in this sense is partly generative: an analysis is more beautiful when it supports discovery rather than merely restating what is already obvious.

The human role under Beauty is contextualization. Human analysts convert AI-generated patterns into compelling narratives, identify spurious correlations, and ensure that “black-box” outputs do not obscure important subtleties. A plausible implication is that Beauty functions as an anti-reductionist dimension within TBJ: it resists the idea that compressed performance summaries are sufficient for scientific understanding.

4. Justice

Justice addresses the ethical and societal implications of AI and data science. The framework identifies three principal elements.

Fairness concerns avoiding or mitigating bias in algorithmic decisions, illustrated by the requirement that $P(Y \mid X, \mathrm{Group}=g)$ should not systematically disadvantage group $g$. The formulation is illustrative rather than a complete fairness doctrine, but it makes explicit that distributional asymmetry across groups is a central concern.

Privacy and security concerns compliance with legal frameworks and organizational policies governing sensitive data. Justice thus encompasses institutional and legal constraints in addition to model behavior.

Workforce impacts concerns the pipeline of junior data scientists and the distributional effects of automation. Justice is therefore not limited to end-user harms or representational harms; it includes labor-market and training consequences within the research enterprise itself.

The human role under Justice includes governance, ongoing bias audits, ethical review boards, and the design of upskilling and reskilling programs. The paper also links this dimension to continuous monitoring for bias amplification and privacy breaches, and to the use of human judgment to uphold societal values, legal requirements, and workforce equity. This suggests that Justice operates both as an ex ante design criterion and as an ex post monitoring obligation.

5. Formalization and operationalization

In the 2025 paper, TBJ is presented conceptually rather than through new mathematical formulas. The authors note, however, that earlier related work allows any model or AI tool $M$ to be viewed as possessing three latent scores: $T(M) \in [0,1]$, $B(M) \in [0,1]$, and $J(M) \in [0,1]$. A generic combination consistent with prior TBJ treatments is given as

$$TBJ(M) = \lambda_T \cdot T(M) + \lambda_B \cdot B(M) + \lambda_J \cdot J(M),$$

with the $\hat{y}$0 values summing to $\hat{y}$1 to reflect relative priorities (Timpone et al., 15 Jul 2025).

This expression is not presented as the paper’s main operational device. Instead, the paper emphasizes question-based checklists, especially for Truth and Justice in the execution phase. The stated reason is practical as well as conceptual: TBJ does not provide an off-the-shelf quantitative scoring system. It relies on human judgments and checklists.

That limitation is material rather than incidental. TBJ is described as actionable because it supports workflow-specific interrogation, but it is also conceptual because it does not reduce evaluation to a closed-form metric. A plausible implication is that the framework is deliberately resistant to full procedural codification. The absence of a definitive scalar score preserves room for domain-sensitive judgment, but it also means that successful use of TBJ depends on institutional capacity, reviewer competence, and sustained oversight.

6. Application across the data-science workflow

Timpone and Yang segment the workflow into three phases—Planning, Execution, and Activation—with two stages in each phase. For each stage, the paper identifies the primary TBJ dimension and a recommended human–AI actor balance (Timpone et al., 15 Jul 2025).

Stage	Primary TBJ	Recommended actor balance
Ideate & define	Beauty	AI complements humans
Design & plan	Truth	AI complements humans
Gather & process data	Truth & Justice	Humans complement AI
Conduct analysis	Truth	Humans complement AI
Create insights	Truth & Justice / Beauty	AI complements humans
Activate insights	Truth & Justice	Human led

The structure is notable because it does not treat AI assistance as uniform across the workflow. In the planning stages, AI is cast as complementary to human ideation and planning. In execution, especially data gathering and analysis, human oversight becomes more prominent. In activation, the asymmetry persists: AI may assist in insight creation, but action on insights remains human led.

The paper provides stage-specific illustrations. At the ideation stage, Ashkinaze et al. (2024) are cited as showing that humans exposed to AI-generated prompts produce a more diverse set of ideas without loss of creativity. At the design stage, generative AI can draft questionnaires or analysis plans, which humans then refine for validity and compliance. In the data-gathering stage, “silicon samples” from LLMs may understate variance, as discussed in relation to Park et al. (2024) and Timpone and Yang (2024), so human teams must audit heterogeneity. In the analysis stage, AI agent code generators are reported to be able to “lie” about implementation, making human truth checking essential. In activation, AI can draft insight summaries, but human data scientists are assigned responsibility for ensuring fairness of recommendations and for tailoring communications to stakeholders.

A common misconception is that the framework licenses generalized push-button automation. The workflow recommendations indicate the opposite. TBJ is used to differentiate tasks according to where AI can assist, where human review must dominate, and where decision authority should remain human because VUCA conditions make ambiguity and contextual tradeoffs central.

7. Strengths, limitations, and implications

The paper identifies three strengths of TBJ. It is multi-dimensional, because it goes beyond technical accuracy to include interpretability and ethics. It is flexible, because it applies to analytic AI, generative AI, and agentic systems. It is actionable, because checklists and actor-balance recommendations support human–AI collaboration (Timpone et al., 15 Jul 2025).

The limitations are equally explicit. TBJ is conceptual, not an off-the-shelf quantitative scoring system. It requires sustained human oversight; if that oversight is neglected, the paper warns of “blindness-by-design,” citing Winograd and Flores (1986). It also involves pipeline risk, because automation of routine tasks may threaten the junior-analyst pipeline, with implications for the long-run reproduction of expertise.

The best practices recommended in the paper follow directly from these strengths and limitations. They include using TBJ checklists at each workflow stage; adopting the human–AI balance guidelines to align tasks with the actors best suited to volatility, uncertainty, complexity, and ambiguity; ensuring continuous reskilling and upskilling so data scientists can focus on high-order conceptual framing, ethical governance, and narrative communication; mandating “check everything” when using AI agents, including verification of code, inspection of assumptions, and validation of outputs against domain knowledge; and maintaining diversity in research teams to safeguard Truth and Justice while broadening perspectives and enriching Beauty.

Taken together, TBJ functions as a structured guide for human–AI collaboration rather than as a doctrine of either algorithmic substitution or categorical human exceptionalism. The framework assumes that advanced AI can materially assist research, but it also assigns continuing centrality to human judgment in validity assessment, heterogeneity auditing, fairness governance, and activation under VUCA conditions. In that sense, TBJ is best understood as a triadic normative and operational framework for ensuring that data science remains accurate, interpretable, and socially responsible even as automation expands.