---
title: Human-AI Collaboration Index
url: https://www.emergentmind.com/topics/human-ai-collaboration-index-hac
type: topic
---

# Human-AI Collaboration Index

The Human-AI Collaboration Index (HAC) is a class of quantitative measures designed to assess the effectiveness or quality of collaboration between human and artificial agents. HAC frameworks provide rigorous, operationalized constructs with explicit scoring functions and measurement protocols, seeking to move beyond simplistic notions of automation or human-in-the-loop operation. Instead, they quantify the degree to which a joint human-AI system achieves genuine collaborative affordances, complementary information use, and mutual adaptation. Multiple distinct formalizations of HAC have been proposed, reflecting different foundational paradigms: decision-theoretic information value [2411.10463], dialogue and grounding in shared tasks [2602.21337], and multi-level teaming affordances [2606.15509]. Each approach prioritizes different metrics, system prerequisites, and experimental designs but converges on the core aim: distinguishing simple assistance from robust collaboration.

## 1. Theoretical Foundations and Taxonomies

A central theme in HAC research is the rigorous distinction between collaboration and related modalities such as delegation, consultation, or governance. Cukurova et al. [2606.15509] introduce a five-level diagnostic taxonomy of human–AI teaming that underpins collaboration assessment:

1. **Transactional**: The AI system executes discrete requests without modeling user state or goals.
2. **Situational**: The system maintains and surfaces a shared situational model, providing interpretability and contestability.
3. **Operational**: Users can set explicit goals or plans governing system behavior over tasks.
4. **Praxical**: The AI adapts its internal models based on ongoing user feedback but does not challenge the human.
5. **Synergistic**: The system maintains an inspectable epistemic state, challenges and reasons with the user under shared goals, and demonstrates mutual modeling and social regulation.

Collaboration, in the strict sense, emerges only at the praxical and (especially) synergistic levels. HAC thus encodes not just task completion but deep reciprocal understanding and adaptive coordination.

## 2. Decision-Theoretic HAC Formulation

Decision-theoretic HAC formally quantifies the incremental value gained by combining human and AI judgments beyond what either achieves independently. Let $\theta$ denote the true task state, $V_1, ..., V_n$ denote basic features, and $D^H$, $D^{AI}$ the human and AI decisions. For a proper scoring rule $S(d, \theta)$ (e.g., Brier score), the marginal value of feature $X_i$ for each agent and jointly is computed as follows:
- $V_{human}(X_i) = R(X_i, D^H) - R(D^H)$
- $V_{AI}(X_i) = R(X_i, D^{AI}) - R(D^{AI})$
- $V_{joint}(X_i) = R(X_i, D^H, D^{AI}) - R(D^H, D^{AI})$

The *unexploited value* of information is
$$
U_{unexp}(X_i) = V_{joint}(X_i) - [V_{human}(X_i) + V_{AI}(X_i)]
$$
The Human-AI Collaboration Index is then normalized as
$$
HAC = \frac{\sum_{i=1}^n U_{unexp}(X_i)}{\sum_{i=1}^n V_{joint}(X_i)}
$$
$HAC\approx 0$ implies redundancy (no information synergy); $HAC\approx 1$ signals strong complementarity, i.e., that effective collaboration unlocks substantial value unachievable by either agent alone. This approach requires detailed audit of feature/decision usage and often involves Shapley-value decomposition across all observed features [2411.10463].

## 3. HAC as Group Process: Common Ground and Dialogue

An alternative paradigm constructs HAC from linguistic theory, focusing on joint action and common ground maintenance during interactive problem solving [2602.21337]. In this framework, collaboration is quantified not only by task outcomes but also by referential and conversational coordination. Using structured collaborative puzzle benchmarks, four principal components are normalized and aggregated:

1. **Task Success ($S^*$):** Average puzzle completion rate.
2. **Efficiency ($E^*$):** Inverse normalized word count per trial.
3. **Referential Coordination ($R^*$):** Ratio of shared noun-phrase vocabulary between human and AI.
4. **Grounding Engagement ($G^*$):** Rate of clarification and repair acts.

The composite HAC is
$$
\mathrm{HAC} = 0.4\, S^* + 0.2\, E^* + 0.2\, R^* + 0.2\, G^*
$$
This structure prioritizes outcome but penalizes inefficiency, ambiguous references, and lack of repair. Empirically, higher HAC values correlate with more robust alignment, efficient negotiation, and deeper mutual understanding between human and AI agents in controlled collaborative tasks [2602.21337].

## 4. Multi-Affordance and Sub-Index Decomposition

Cukurova et al. [2606.15509] advance the field by defining HAC through a vector of affordance-sensitive sub-indices, each measuring a specific cognitive or system function critical for collaboration:

| Sub-Index              | Definition/Example Metric                                                      | Formula (where given)                             |
|------------------------|--------------------------------------------------------------------------------|----------------------------------------------------|
| Grounding Accuracy (G) | Match between system’s user representation and actual user intent              | $G = \frac{1}{N}\sum_{i=1}^{N} \mathbf{1}[repr_i = intent_i]$      |
| Goal Negotiation (N)   | Alignment of system/user goal vectors post-negotiation                        | $N = 1 - \|g_{sys} - g_{user}\|_2 / (\|g_{sys}\|_2 + \|g_{user}\|_2)$ |
| Adaptation Responsiveness (A_r) | Speed of incorporating explicit corrections                   | $A_r = \frac{1}{M} \sum_{i=1}^M \exp(-\alpha \Delta t_i)$    |
| Update Fidelity (F)    | Correlation between user-requested and actual model change magnitude           | $F = \mathrm{corr}(\{u_i\},\{m_i\})$               |
| Mutual Modelling (M_d) | Deep inference of user knowledge, preferences, strategies                      | $M_d = |A_{correct}| / |A_{total}|$                |
| Shared Regulation (R)  | Success rate of joint planning/monitoring/meta-acts                            | $R = C_{joint} / C_{prop}$                         |
| Co-Reasoning Synergy (S) | Density and success of joint argumentation                           | $S = (A_s / T) \cdot (C_{res} / A_s)$              |

These indices are aggregated, commonly as
$$
HAC = \sum_{i} w_i \cdot S_i
$$
or by level-specific aggregation reflecting the five-level taxonomy. The sub-index methodology allows targeted diagnosis of collaborative deficits, supporting refinement of AI affordances and interaction protocols [2606.15509].

## 5. Empirical Measurement and Evaluation Protocols

HAC computation depends on both high-fidelity interaction data and reliable process annotation pipelines:

- **Interaction logs:** Transcript-level capture of exchanges, system states, and correction events.
- **User annotations:** Explicit labeling of intent, goals, and satisfaction.
- **Expert coding:** Ground-truthing of conversational acts or model updates.
- **Statistical analysis:** Mixed-effects modeling, interrater reliability checks, and validation against external benchmarks (e.g., learning gains, user satisfaction).
- **Benchmark tasks:** Task designs that demand referential coordination, negotiation, and repair are essential for valid HAC discrimination [2602.21337].

Weighting of sub-indices or components requires either normative justification or outcome-driven tuning (e.g., via regression on performance data), and must be transparent to permit interpretability and cross-system comparisons.

## 6. Comparisons, Limitations, and Relationship to Adjacent Constructs

Not all frameworks using "collaboration" terminology in human-AI contexts address the stringent requirements formalized in HAC. Many systems historically labeled as collaborative exhibit only transactional or consultative affordances and fail to meet the criteria of shared, negotiable goals, mutual modeling, or symmetric regulation [2606.15509]. Conceptually adjacent aggregates such as the Artificial Intelligence Quotient (AIQ) [2503.16438] propose multidimensional profiling of human capacity for AI interaction but do not provide a formal sub-index or explicit aggregation formula for HAC. The relationship between such broad frameworks and rigorous HAC quantification remains an open area for further psychometric and empirical development.

## 7. Future Directions and Open Challenges

Current HAC frameworks face several limitations:

- **Validation:** Few published empirical studies have validated HAC formulas against longitudinal real-world collaboration outcomes.
- **Task generality:** Most metrics are benchmark/task-specific; transferability across domains is not assured.
- **Ethical, privacy, and data governance:** Systematic measurement requires detailed logging and content analysis, raising user consent and privacy issues [2606.15509].
- **Rapid AI evolution:** Collabative affordances and necessary measurement criteria shift as system capabilities change, necessitating continual updating of HAC definitions and benchmarks.

Nevertheless, HAC provides a rigorous scaffold for analyzing, diagnosing, and improving human–AI teaming across decision-making, education, creative synthesis, and other domains where collaborative intelligence offers the potential for true hybrid performance gains [2411.10463, 2606.15509, 2602.21337].

Source: https://www.emergentmind.com/topics/human-ai-collaboration-index-hac