---
title: Skill Automation Feasibility Index (SAFI)
url: https://www.emergentmind.com/topics/skill-automation-feasibility-index-safi
type: topic
---

# Skill Automation Feasibility Index (SAFI)

Searching arXiv for the cited paper and closely related work to ground the article.
The **Skill Automation Feasibility Index (SAFI)** is a **0–100 index** that quantifies how well frontier large language models perform **text-based tasks aligned with each of the 35 O\*NET skills**, averaging performance across four models and using a standardized task-and-rubric pipeline. It is explicitly defined as a measure of **automation feasibility in text form**, not of full occupational automation, and is used together with the Anthropic Economic Index to build an **AI Impact Matrix** that situates skills at the intersection of model capability and real-world AI exposure [2604.06906].

## 1. Definition, scope, and mathematical form

SAFI is defined for a skill \(s\) as:

\[
\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}
\]

where \(M\) is the set of four evaluated LLMs, \(T_s\) is the set of benchmark tasks associated with skill \(s\), and \(\text{score}(t,m)\) is the task-level score from 0 to 10 for model \(m\) on task \(t\). The index is therefore a normalized average of model performance on a skill’s associated text tasks, scaled to the interval \([0,100]\) [2604.06906].

The scale is linear and interpretable. A value of 0 means no measurable performance on the benchmark’s text tasks for that skill; a value of 100 means perfect performance by all four models on every task for that skill. The paper repeatedly emphasizes that SAFI measures **LLM performance on text-based representations of skills**, and should be read as an **upper-bound proxy for what LLMs can do with the text-amenable part of a skill**, not as a direct statement about whether the real-world skill, or the occupations using it, is fully automatable [2604.06906].

This scope restriction is central. SAFI does **not** capture physical, embodied, real-time, or rich interpersonal aspects of skill execution, and it does not measure the organizational, regulatory, or economic conditions that determine actual displacement. A high SAFI for a skill such as Operation and Control therefore means that models can discuss or reason about the skill in text form; it does not mean that they can physically execute the corresponding activity. This suggests that SAFI is best understood as a **capability measure** for text-compatible components of skill rather than a labor-market forecast [2604.06906].

## 2. Benchmark design and computation

The benchmark covers **263 text-based tasks** spanning **all 35 O\*NET skills**, which are grouped into seven categories: Content, Process, Social, Complex Problem Solving, Technical, Systems, and Resource Management. Each task is authored to instantiate one specific O\*NET skill according to its formal definition, so the mapping \(T_s\) is fixed by construction. Tasks are presented in three difficulty levels—easy, medium, and hard—and are designed to capture the cognitive or communicative essence of the target skill, such as contradiction identification for Reading Comprehension, multi-party trade-off scenarios for Negotiation, and code writing or debugging for Programming [2604.06906].

Every model receives identical prompts with temperature fixed at **0.3**. Across **263 tasks** and **4 models**, the study executes **1,052 responses** with a **0% failure rate**. Each response is scored not by another LLM but by a **multi-signal heuristic engine** with four components: **Response Completeness (0–3)**, **Response Depth (0–3)**, **Reasoning Quality (0–2)**, and a **Difficulty-Adjusted Bonus (0–2)**. Skill-specific adjustments add bonuses for correct calculations in Mathematics and for functional, syntactically correct code in Programming [2604.06906].

The task-level scores are aggregated first to model-specific skill scores and then to model-agnostic SAFI values. For a given model \(m\), the per-skill score is:

\[
\text{SAFI}_m(s) = 100 \cdot \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}
\]

and the overall skill-level index is:

\[
\text{SAFI}(s) = \frac{1}{|M|} \sum_{m \in M} \text{SAFI}_m(s)
\]

Because every task is scored on the same 0–10 rubric and every skill is evaluated over the same four-model set, the resulting scores are explicitly treated as comparable across skills and across models [2604.06906].

## 3. Models evaluated and empirical skill profile

The evaluated models are **LLaMA 3.3 70B**, **Mistral Large**, **Qwen 2.5 72B**, and **Gemini 2.5 Flash**. The selection is described as spanning different geographies, architectural or training regimes, and licensing models: Meta and Google in the United States, Mistral AI in France, and Alibaba in China; open-source and closed-source systems are both represented [2604.06906].

At the model-wide level, average scores across all skills lie in a narrow range: **Mistral Large 60.0**, **LLaMA 3.3 70B 58.2**, **Qwen 2.5 72B 56.7**, and **Gemini 2.5 Flash 56.4**. The paper treats the resulting **3.6-point spread** as evidence of strong cross-model convergence, arguing that text-based automation feasibility appears **more skill-dependent than model-dependent** across current frontier systems [2604.06906].

The most automatable skills in the benchmark are **Mathematics (73.2)** and **Programming (71.8)**. The lowest-scoring are **Active Listening (42.2)** and **Reading Comprehension (45.5)**, followed by **Speaking (48.5)**, **Writing (51.0)**, and **Social Perceptiveness (51.5)**. These lower-ranked skills are concentrated in Content and Social categories and are also described as very important across many occupations [2604.06906].

| Skill pattern | Skills | SAFI |
|---|---|---:|
| Highest | Mathematics; Programming | 73.2; 71.8 |
| Lowest | Active Listening; Reading Comprehension | 42.2; 45.5 |
| Other low scores | Speaking; Writing; Social Perceptiveness | 48.5; 51.0; 51.5 |

At the category level, **Technical Skills** have the highest mean SAFI at **62.1** with within-category \(\sigma = 0.99\), followed by **Systems Skills** at **60.0** and **Complex Problem Solving** and **Resource Management** at **59.2**. **Content Skills** have the lowest mean at **53.1** and the highest variance, which the paper attributes to the coexistence of highly structured skills such as Mathematics and weakly captured communicative skills such as Active Listening within the same O\*NET category [2604.06906].

## 4. SAFI, the Anthropic Economic Index, and the AI Impact Matrix

SAFI is linked to real-world AI use through the **Anthropic Economic Index (AEI)**, which contributes **756 occupations** with AI exposure scores, **17,998 O\*NET tasks** with task penetration rates, and **3,364 tasks** classified into AI–human interaction modes: directive, feedback loop, task iteration, validation, and learning. The paper computes, for each of the 35 skills, the **Pearson correlation** between O\*NET skill importance across occupations and AEI exposure scores, thereby creating a second axis that complements SAFI’s capability measure [2604.06906].

These two axes define the **AI Impact Matrix**. Each skill \(s\) is positioned as:

\[
(x_s,y_s)=\left(\text{CorrExposure}(s),\text{SAFI}(s)\right)
\]

where the x-axis is the skill’s exposure correlation and the y-axis is SAFI. The matrix is divided into four qualitative quadrants: **Higher Displacement Risk**, **AI-Augmented**, **Upskilling Window**, and **Lower Displacement Risk**. The paper is explicit that this is a **heuristic interpretive framework**, not a predictive model of employment loss [2604.06906].

The canonical **Higher Displacement Risk** example is **Programming**, with **SAFI 71.8** and exposure correlation **+0.455**. By contrast, **Reading Comprehension (+0.372)**, **Writing (+0.369)**, and **Active Listening (+0.338)** are concentrated in AI-exposed occupations but have relatively low SAFI scores, placing them in the **AI-Augmented** quadrant. This mismatch is summarized as a **“capability-demand inversion”**: skills most concentrated in AI-exposed jobs are often the skills on which the benchmarked LLMs perform least well [2604.06906].

The inversion is quantified as a negative cross-skill relationship between SAFI and exposure correlation: **Pearson \(r=-0.196\), \(p=0.26\), \(n=35\)** and **Spearman \(\rho=-0.300\), \(p=0.08\), \(n=35\)**. The paper notes that these correlations are not statistically significant at conventional levels, but both point in the same negative direction. It interprets the pattern as evidence that current AI adoption is concentrated in jobs where AI is primarily used to **augment** difficult human skills rather than to automate them away [2604.06906].

This interpretation is reinforced by the AEI interaction data: **78.7%** of observed AI interactions are classified as **augmentation**, including **32.5% feedback loops**, **29.9% learning interactions**, and other collaborative modes such as task iteration and validation; only **21.3%** are **directive automation**. A plausible implication is that SAFI is most informative when read jointly with observed interaction modes rather than as a standalone replacement index [2604.06906].

## 5. Conceptual significance and relation to adjacent research

SAFI belongs to a broader research shift from occupation-level automation narratives toward skill- and task-level analysis. The paper’s focus on O\*NET skill units closely aligns with the **“skill automation view”**, which argues that automation substitutes for skills rather than occupations and studies how removing automatable skills changes the structure of the occupation network [2411.03605]. Within that perspective, SAFI provides a direct capability-oriented ranking of the text-based components of the 35 O\*NET skills rather than a probability of occupation disappearance.

Related work also sharpens the distinction between **automation** and **augmentation**. The **2ACT** framework models six human–AI usage patterns—Directive AI, Feedback Loop AI, Validation AI, Task Iteration AI, Learning AI, and Thinking Fraction—and finds that automation-focused usage patterns predict lower job-zone placement, whereas augmentative patterns interact positively with cognitive and scientific capabilities in specific “skill bridges” [2505.07914]. This suggests that SAFI’s AI Impact Matrix is best read as one layer in a larger account of occupational change: a high SAFI indicates technical feasibility for text-based execution, but actual career consequences depend on usage mode.

Other adjacent work defines automation exposure in more theory-driven terms. A Moravec-inspired exposure index scores O\*NET tasks on **performance variance**, **data abundance**, **tacit knowledge**, and **algorithmic efficiency gap**, explicitly targeting **fundamental automatability** rather than current capability snapshots [2510.13369]. SAFI differs in being benchmark-based and current-model-based, but both approaches converge on a non-routine pattern in which many cognitive and digital tasks are highly exposed while embodied and manual tasks remain relatively resistant.

A further extension comes from workflow theory. A model of AI automation as **chains** of contiguous AI-executed steps argues that comparative-advantage logic can fail once firms bundle adjacent steps into AI chains, and reports empirical evidence that AI-executed steps co-occur in chains, that dispersion lowers job-level AI execution, and that adjacency to AI-executed steps increases the likelihood of execution by AI [2606.15960]. This suggests that SAFI’s skill-level values may not be sufficient for organizational analysis unless they are also situated within workflow structure.

## 6. Limitations, uses, and methodological extensions

The paper identifies seven main limitations. First, SAFI is a **text-only representation of skills** and should be treated as an upper bound on text-amenable components. Second, the heuristic scoring engine does **not verify factual correctness** or deep domain accuracy and may under- or mis-estimate nuanced Content and Social skills. Third, **263 tasks across 35 skills** is broad but still limited relative to real-world variation, and the number of tasks per skill varies. Fourth, the AEI adoption data are derived from **Claude** and predominantly English-language contexts. Fifth, the results are a **cross-sectional snapshot** of frontier models in early 2026. Sixth, SAFI is **not employment-weighted**. Seventh, the AI Impact Matrix is interpretive rather than predictive [2604.06906].

These limitations define the proper use of the index. For policymakers, the paper proposes using SAFI together with the AI Impact Matrix to differentiate retraining strategies by quadrant, rather than deploying a single “AI readiness” program. For employers, it recommends identifying roles concentrated in Quadrant I skills for redeployment strategies, consistent with JPMorgan’s **“displace and redeploy”** approach, and formalizing co-pilot usage and output-verification practices in Quadrant II roles. For workers and educators, it recommends treating the near-term shift as movement from **first-draft production to last-mile judgment**, with curricular emphasis on skills that combine high labor demand and low SAFI, alongside AI-emergent skills such as prompting, output evaluation, and workflow design [2604.06906].

The broader literature points toward several methodological extensions. Capability-driven skill generation in industrial automation treats capabilities as contracts for skill implementations and uses retrieval-augmented generation to produce executable skills across interfaces and languages, emphasizing the role of structured capability descriptions and interface reuse [2505.03295]. Skill-centered agent evaluation frameworks such as **SkillAudit** measure utility, efficiency or cost, and safety of deployable skills through baseline comparison and task generation, finding that over **7%** of scanned real-world skill packages were in risky status [2606.22613]. A survey of agent skill evaluation and evolution similarly argues for multi-dimensional scoring across performance, safety, generalization, efficiency, human oversight, and ecosystem compatibility [2606.11435]. Together, these results suggest that SAFI’s benchmark-based capability score could plausibly be extended into a broader multi-axis family of indices rather than remaining a single scalar.

A final implication emerges from work on skill-level evaluation outside labor-market measurement. An annotation study on schema-guided behavioral profile labeling argues that automatic annotation should be evaluated in terms of **skill feasibility rather than task-level automation**, distinguishing directly operable, recoverable, and structurally underspecified skills [2604.14843]. A software scripting study likewise treats skills as verified reusable scripts discovered through offline simulation, showing higher automation success rates and lower runtime cost than runtime code generation [2504.20406]. These findings reinforce the central SAFI proposition: automation feasibility is often heterogeneous at the skill level, and task- or occupation-level summaries obscure that heterogeneity.

SAFI therefore occupies a specific methodological position. It is a reproducible, model-agnostic benchmark score for the **text-based execution of O\*NET skills**; it becomes most informative when paired with observed AI use, workflow structure, and transition analysis; and it is least informative when reinterpreted as a direct measure of job loss, physical substitutability, or deterministic labor-market displacement [2604.06906].

Source: https://www.emergentmind.com/topics/skill-automation-feasibility-index-safi