Trust Inclination Index (TII) Overview

• TII is a family of quantitative metrics that formalizes and measures trust propensity and dynamics in computational systems using scalar ratios and vector-based indices.
• The scalar TII in role-playing frameworks quantifies trust versus suspicion among agents, enabling direct inter-model comparisons with standardized percentages.
• The vector-valued trust index assesses solution quality by aggregating semantic and emotional dimensions, offering nuanced insights into LLM performance.

The Trust Inclination Index (TII) is a family of quantitative metrics designed to formalize, measure, and analyze trust propensity and trust dynamics in computational systems, particularly those involving LLMs and interactive, multi-agent environments. TII instantiates as either a scalar or vector-valued index, depending on the application domain, and has emerged as a critical metric in recent frameworks evaluating LLMs' ability to simulate advanced human-like social cognition, trust calibration, and solution quality under ambiguity. The term applies most concretely and formally in two contexts: (1) the MIRAGE role-playing evaluation framework for LLMs in social environments (Cai et al., 3 Jan 2025), and (2) the multidimensional vector-valued trust index for solution quality assessment in natural language tasks (Rug et al., 21 Jul 2025).

1. Conceptual Motivation and Definitions

In social and computational sciences, trust has been historically conceptualized via constructs such as ability, integrity, benevolence, and dispositional propensities (Borgo et al., 2020). The need for operational indices arises due to the complex, context-sensitive, and multi-dimensional nature of trust—particularly when assessing the nuanced trade-off between belief and doubt among agents, or between solution robustness and affective adequacy. In contemporary modeling, TII quantifies either:

• The relative inclination of an agent (typically an LLM) to trust versus suspect other agents across interactive episodes (Cai et al., 3 Jan 2025).
• The ease with which a solution to a problem elicits trust (or other evaluative responses) across multiple quality dimensions, capturing robustness, consistency, and emotional valence (Rug et al., 21 Jul 2025).

2. Formal Definitions and Mathematical Formulation

Two primary instantiations of TII are formally documented:

A. Scalar TII in Interactive Role-play (MIRAGE Framework)

Let $C=\{c_1,\dots,c_N\}$ denote all characters. For each phase $t=1,\dots,T$, each character $c_i$ assigns:

• $P_T^{(t)}(c_i,c_j) \in \{0,1,2\}$ — trust score
• $P_S^{(t)}(c_i,c_j) \in \{0,1,2\}$ — suspicion score

Aggregate over phases:

• $$\overline{P}_T(c_i, c_j) = \sum_{t=1}^{T} P_T^{(t)}(c_i,c_j)$$
• $$\overline{P}_S(c_i, c_j) = \sum_{t=1}^{T} P_S^{(t)}(c_i,c_j)$$

Then, the Trust Inclination Index for character $c'$ is:

$$\mathrm{TII}_{c'} = \frac{\sum_{c \in C, c \ne c'} \overline{P}_T(c, c')}{\sum_{c \in C, c \ne c'} [\overline{P}_T(c, c') + \overline{P}_S(c, c')]}$$

This ratio yields a scalar in $[0,1]$, reported as a percentage. It expresses the global inclination of “others” to trust $t=1,\dots,T$0. High values indicate a greater tendency to trust than to suspect (Cai et al., 3 Jan 2025).

B. Vector-valued Trust Index $t=1,\dots,T$1 in Solution Quality Assessment

For problem $t=1,\dots,T$2 and candidate solution $t=1,\dots,T$3:

$t=1,\dots,T$4

$t=1,\dots,T$5 captures the degree to which $t=1,\dots,T$6 meets the $t=1,\dots,T$7-th criterion (e.g., factuality, clarity, robustness, emotional impact) (Rug et al., 21 Jul 2025).

Principal components:

• $t=1,\dots,T$8: 1 minus normalized bi-semantic entropy, quantifying semantic stability of solutions across prompt paraphrases/hypotheses.
• $t=1,\dots,T$9: Mean emotional valence, aggregating human or LLM persona evaluators' responses.

3. Methodologies and Empirical Computation

In MIRAGE, trust and suspicion scores are explicitly elicited from LLMs after each interaction round. Scores are tabulated and processed as described above. No additional weighting or post-processing is applied by default, but proposals exist for phase-weighting or fine-grained scaling (Cai et al., 3 Jan 2025).

In solution assessment frameworks, $c_i$0 is estimated by pooling diverse outputs over multiple paraphrased prompts and categorizing via semantic classes. $c_i$1 aggregates evaluative signals, typically in $c_i$2, from a panel of raters (Rug et al., 21 Jul 2025).

Empirical benchmarks in (Cai et al., 3 Jan 2025) illustrate TII distributions across leading LLMs. For MIRAGE’s murder-mystery scripts, reported TII values (averaged over scripts):

Model	TII (%)
GPT-3.5	47.13
GPT-4	76.32
GPT-4o	78.69
Qwen-2-7B	75.78
GLM-4-9B	53.85

This tabular comparison encapsulates both model-specific social inclination and the discriminatory power of the index (Cai et al., 3 Jan 2025).

4. Theoretical Properties, Sensitivities, and Limitations

The TII (in both scalar and vector forms) exhibits several mathematically desirable properties:

• Normalization: Values are confined to $c_i$3, facilitating direct interpretability and inter-model comparison.
• Continuity: For $c_i$4, small perturbations in output distributions yield proportionally small changes in the index.
• Diagnostic Sensitivity: In $c_i$5, high semantic entropy reflects brittleness or sensitivity to prompt reformulation; low values correspond to robust, consistent outputs (Rug et al., 21 Jul 2025).

Assumptions and documented limitations include:

• Reliance on self-reported trust/suspicion (MIRAGE) may be gamed or reflect “politeness bias” in LLMs (Cai et al., 3 Jan 2025).
• Score granularity (use of discrete $c_i$6 scales) may fail to detect subtle distinctions; finer-grained or continuous scoring is suggested for future work.
• All trust opinions are equally weighted; meta-weighting based on rater (or model-agent) reliability is proposed as an avenue for refinement.
• For $c_i$7, the choice of components is application-dependent, and trade-offs between robustness and affective valence must be explicitly managed.

5. Applications and Empirical Insights

TII serves as a core operational metric in the MIRAGE evaluation framework, underpinning systematic assessment of LLMs in agentic, multiplex social domains (e.g., murder mystery games). Scalar TII directly reveals relative trust/suspicion propensities among LLMs, with higher values indicating models prone to trust and lower values denoting skepticism or guardedness. Critical findings include the persistence of high TII in LLMs even when deception is overt, suggesting that current models tend to under-penalize confessing culprits (Cai et al., 3 Jan 2025).

The vector-valued trust index $c_i$8 enables multi-criteria optimization and nuanced calibration for natural-language solutions: iteratively maximizing $c_i$9 and $P_T^{(t)}(c_i,c_j) \in \{0,1,2\}$0 leads to configurations aligning with both objective robustness and positive emotive reception in pilot studies (Rug et al., 21 Jul 2025).

6. Prospects for Refinement and Future Directions

Key directions for extension include:

• Transitioning from discrete to continuous scales for both trust/suspicion and emotional valence, to capture subtler nuances.
• Phase-specific weighting or introducing temporal decay in aggregating judgments, to privilege later, evidence-rich rounds in dynamic interactions.
• Incorporation of behavioral (rather than self-reported) cues, such as detection of verbal inconsistency or strategic deception.
• Meta-weighting agent opinions based on demonstrated reliability or role.
• For multidimensional trust assessment, calibration of “good enough” thresholds for each $P_T^{(t)}(c_i,c_j) \in \{0,1,2\}$1 via human-in-the-loop optimization (Rug et al., 21 Jul 2025).

A plausible implication is that, as LLMs acquire more advanced theory-of-mind and nuanced social simulation capabilities, TII-style metrics will become increasingly central to model benchmarking, policy design, and human-LLM interface engineering.

7. Comparison to Prior Conceptualizations and Related Indices

Earlier work in visualization psychology and trust research (Borgo et al., 2020) emphasizes the need for trust metrics but does not operationalize TII or formal alternatives. Constructs such as trust propensity, trustworthiness, ability, integrity, and benevolence are qualitatively surveyed, and visualization-based tools (e.g., network diagrams, semantic zoom) are proposed as aids to “unpacking” trust, but no explicit, aggregatable index or psychometric protocol is advanced. The scalar and vector-valued TIIs thus fill a critical methodological gap, transitioning trust from abstract construct to quantifiable, comparable, and empirically actionable metric.

Context	TII Role	Index Structure
MIRAGE (LLM gameplay)	Captures trust/suspicion	Scalar ratio in $P_T^{(t)}(c_i,c_j) \in \{0,1,2\}$2
Solution Quality (LLM)	Assesses answer robustness & affect	Vector $P_T^{(t)}(c_i,c_j) \in \{0,1,2\}$3
Visualization Psychology	Conceptual groundwork, not operational	No concrete metric proposed

In summary, the Trust Inclination Index designates a family of modern, mathematically rigorous trust metrics that enable systematic, granular analysis of trust dynamics and solution quality in computational agents, advancing both empirical assessment and theoretical understanding of trust in artificial and human-computer interactive domains (Cai et al., 3 Jan 2025, Rug et al., 21 Jul 2025).