Semantic Differential HATAS

• Semantic Differential HATAS is a validated two-dimensional scale that measures both cognitive trust (e.g., reliability, rationality) and affective trust (e.g., empathy, warmth) in AI agents.
• It utilizes 27 bipolar adjective pairs on a symmetric five-point scale to minimize bias and ensure robust psychometric assessment.
• Empirical analyses, including exploratory and confirmatory factor analyses, confirm its reliability and validity for profiling trust in human–AI interactions.

Semantic Differential HATAS

The Human–AI Trust Affective–Semantic (HATAS) instrument is a rigorously validated, two-dimensional semantic differential scale developed to measure trust in AI agents by targeting both cognitive and affective trust components. Its motivation arises from the recognition that trust in AI is not a unidimensional cognitive construct but possesses both cognitive and emotional (affective) facets, each playing a distinct role in shaping user perceptions and acceptance of human-like, LLM-powered conversational agents. The design and psychometric validation of HATAS address the methodological gap in theoretically grounded, generalizable measures for affective and cognitive trust in human–AI interaction (Shang et al., 2024).

1. Conceptual Framework

HATAS is structured as a two-dimensional, 27-item semantic differential scale, divided into an 18-item cognitive-trust subscale and a 9-item affective-trust subscale. Each item is a bipolar adjective pair (e.g., "Unreliable" to "Reliable" for cognitive trust; "Apathetic" to "Empathetic" for affective trust), administered on a symmetric five-point scale ranging from –2 (entirely negative) through 0 (neutral) to +2 (entirely positive). This format enables resistance to acquiescence bias and ensures interpretability across diverse contexts.

The cognitive trust dimension evaluates the agent's functional competence, dependability, and rationality, while the affective trust dimension assesses warmth, empathy, and perceived social orientation. The two-factor structure is informed by contemporary trust theories in human–AI interaction and is empirically distinguished via robust psychometric procedures.

2. Instrument Structure and Administration

The HATAS instrument presents all 27 bipolar adjective pairs in randomized order following exposure to a short, scenario-based description of AI–human interaction. Participants are instructed: “For each of the following pairs, please indicate how you would describe your level of trust in the AI assistant in this scenario. Place an X on the five-point scale, where –2 means you feel entirely the left-hand adjective, 0 means neutral, and +2 means you feel entirely the right-hand adjective.” Anchors are standardized, and no reverse coding is required.

The semantic differential pairs are as follows:

Subscale	Pairs (negative → positive)	N
Cognitive Trust	Unreliable→Reliable; Inconsistent→Consistent; Unpredictable→Predictable; Undependable→Dependable; Fickle→Dedicated; Careless→Careful; Unbelievable→Believable; Clueless→Knowledgeable; Incompetent→Competent; Ineffective→Effective; Inexperienced→Experienced; Amateur→Proficient; Irrational→Rational; Unreasonable→Reasonable; Incomprehensible→Understandable; Opaque→Transparent; Dishonest→Honest; Unfair→Fair	18
Affective Trust	Apathetic→Empathetic; Insensitive→Sensitive; Impersonal→Personal; Ignoring→Caring; Self-serving→Altruistic; Rude→Cordial; Indifferent→Responsive; Judgmental→Open-minded; Impatient→Patient	9

Randomization of item order is recommended to avoid response bias. For translation or cross-cultural adaptation, retaining the bipolar adjective structure is emphasized to preserve bias resistance.

3. Psychometric Foundation

Exploratory and Confirmatory Factor Analysis

The suitability of data for factor analysis was confirmed (Bartlett’s test χ²(528) = 12,574, p < .001; KMO = .98). Both parallel analysis and eigenvalue criteria (λ > 1) indicated a two-factor solution, validated with oblique promax rotation that revealed a substantial factor correlation (r = .78). Items were retained with primary loadings ≥ .40, cross-loadings < .30, and based on Saucier’s 2:1 rule. The final structure accounted for 43% (cognitive) and 23% (affective) of total variance, with all loadings ≥ .55 on respective primary factors.

Confirmatory factor analysis compared the two-factor model against a one-factor baseline using ML and DWLS estimators. The two-factor model demonstrated superior fit:

• DWLS: χ²(494) = 250.94, p = 1.000, CFI = 1.000, TLI = 1.003, RMSEA = 0.000, SRMR = 0.038
• ML: CFI = 0.920, TLI = 0.914, RMSEA = 0.082, SRMR = 0.046, χ²(494) = 1,506.17, p < .001 Fit improvement over the one-factor model was significant (ML Δχ²(34) = 11,626, p < .001; DWLS Δχ²(34) = 78,664, p < .001).

Reliability

Internal consistency is high for both subscales:

• Cognitive-trust (18 items): Cronbach’s α = 0.98
• Affective-trust (9 items): Cronbach’s α = 0.96

All item-total correlations exceeded 0.60. Reliability coefficients were computed as α = (N/(N–1))·(1 – Σσ²_i/σ²_total).

4. Validity Evidence and Scale Properties

Construct Validity

Experimental manipulations of “high vs. low trustworthiness” in scenarios demonstrated strong sensitivity:

• Cognitive scale: t = 45.74, p < .001 for high vs. low cognitive-trust conditions
• Affective scale: t = 43.00, p < .001 for high vs. low affective-trust conditions

Mixed-effect models confirmed significant main and interaction effects in expected directions across subscales.

Concurrent and Discriminant Validity

Both subscales exhibited positive correlations with a general-trust single-item measure (Likert 1–5): cognitive r = .881, p < .001; affective r = .253, p < .001. Discriminant validity was established by showing the HATAS affective subscale's distinction from the moral-trust dimension of the MDMT instrument—EFA on pooled items yielded three separable factors (cognitive, affective, moral), with only HATAS scales predicting general trust.

Cognitive × Affective Interaction

A significant moderation effect in predicting general trust was observed:

• High cognitive trust conditions: variations in affective trust produced minimal change in overall trust.
• Low cognitive trust conditions: affective trust had substantial influence on overall trust.

The high-cognitive × high-affective interaction coefficient was negative and significant (b = –1.726, p < .001), indicating a ceiling effect on trust under strong cognitive assurance.

5. Scoring, Interpretation, and Application

Subscale and Composite Scoring

• Cognitive-trust score: sum (–36 to +36) or average (–2 to +2) of the 18 cognitive items
• Affective-trust score: sum (–18 to +18) or average (–2 to +2) of the 9 affective items

An overall trust metric may be calculated as a simple additive index (cognitive + affective), or kept as two dimensions for analytic richness. For summary prediction, general trust can be regressed onto the subscales (e.g., GeneralTrust ≈ 0.88·Cognitive + 0.25·Affective), with context-dependent weights.

Interpretation and Benchmarks

No universal cutoffs are prescribed. Researchers are encouraged to compare subgroup means (e.g., by experimental condition) or use quartiles to demarcate low/medium/high trust. Subscale means near +2 indicate high trust, near zero indicate neutrality, and negative values reflect distrust, enabled by the scale's symmetric structure.

Practical Implications

To optimize interpretability and experimental control:

• Scenarios should be designed to elicit the intended trust pathway (cognitive or affective) as needed.
• Randomization of item order mitigates response tendencies.
• Retain the semantic differential, bipolar-adjective format if translating or adapting the instrument cross-culturally.

6. Research Context and Use Cases

The HATAS instrument provides a valid and reliable diagnostic tool for profiling user trust in AI agents, isolating the contributions of cognitive and affective routes. It enables the empirical study of interface and explanation manipulations, the calibration of human–AI interaction strategies, and the assessment of trust modulation effects in experimental and applied settings. By deploying HATAS, trust researchers and system designers gain nuanced insight into the multidimensional nature of trust as experienced in interactions with advanced, conversational AI systems (Shang et al., 2024).