Skill Automation Feasibility Index (SAFI)

• SAFI is a 0–100 index that aggregates LLM scores on 263 text-based benchmark tasks across 35 O*NET skills, reflecting text-enabled automation capability.
• It uses a standardized scoring pipeline and multi-model evaluation to provide a normalized measure focused on the text-amenable part of each skill.
• Integrated with the Anthropic Economic Index into an AI Impact Matrix, SAFI reveals a capability-demand inversion that informs strategies for workforce upskilling and augmentation.

Searching arXiv for the cited paper and closely related work to ground the article. The Skill Automation Feasibility Index (SAFI) is a 0–100 index that quantifies how well frontier LLMs perform text-based tasks aligned with each of the 35 O*NET skills, averaging performance across four models and using a standardized task-and-rubric pipeline. It is explicitly defined as a measure of automation feasibility in text form, not of full occupational automation, and is used together with the Anthropic Economic Index to build an AI Impact Matrix that situates skills at the intersection of model capability and real-world AI exposure (Jadhav et al., 8 Apr 2026).

1. Definition, scope, and mathematical form

SAFI is defined for a skill $s$ as:

$$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$

where $M$ is the set of four evaluated LLMs, $T_s$ is the set of benchmark tasks associated with skill $s$, and $\text{score}(t,m)$ is the task-level score from 0 to 10 for model $m$ on task $t$. The index is therefore a normalized average of model performance on a skill’s associated text tasks, scaled to the interval $[0,100]$ (Jadhav et al., 8 Apr 2026).

The scale is linear and interpretable. A value of 0 means no measurable performance on the benchmark’s text tasks for that skill; a value of 100 means perfect performance by all four models on every task for that skill. The paper repeatedly emphasizes that SAFI measures LLM performance on text-based representations of skills, and should be read as an upper-bound proxy for what LLMs can do with the text-amenable part of a skill, not as a direct statement about whether the real-world skill, or the occupations using it, is fully automatable (Jadhav et al., 8 Apr 2026).

This scope restriction is central. SAFI does not capture physical, embodied, real-time, or rich interpersonal aspects of skill execution, and it does not measure the organizational, regulatory, or economic conditions that determine actual displacement. A high SAFI for a skill such as Operation and Control therefore means that models can discuss or reason about the skill in text form; it does not mean that they can physically execute the corresponding activity. This suggests that SAFI is best understood as a capability measure for text-compatible components of skill rather than a labor-market forecast (Jadhav et al., 8 Apr 2026).

2. Benchmark design and computation

The benchmark covers 263 text-based tasks spanning all 35 O*NET skills, which are grouped into seven categories: Content, Process, Social, Complex Problem Solving, Technical, Systems, and Resource Management. Each task is authored to instantiate one specific O*NET skill according to its formal definition, so the mapping $T_s$ is fixed by construction. Tasks are presented in three difficulty levels—easy, medium, and hard—and are designed to capture the cognitive or communicative essence of the target skill, such as contradiction identification for Reading Comprehension, multi-party trade-off scenarios for Negotiation, and code writing or debugging for Programming (Jadhav et al., 8 Apr 2026).

Every model receives identical prompts with temperature fixed at 0.3. Across 263 tasks and 4 models, the study executes 1,052 responses with a 0% failure rate. Each response is scored not by another LLM but by a multi-signal heuristic engine with four components: Response Completeness (0–3), Response Depth (0–3), Reasoning Quality (0–2), and a Difficulty-Adjusted Bonus (0–2). Skill-specific adjustments add bonuses for correct calculations in Mathematics and for functional, syntactically correct code in Programming (Jadhav et al., 8 Apr 2026).

The task-level scores are aggregated first to model-specific skill scores and then to model-agnostic SAFI values. For a given model $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$0, the per-skill score is:

$$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$1

and the overall skill-level index is:

$$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$2

Because every task is scored on the same 0–10 rubric and every skill is evaluated over the same four-model set, the resulting scores are explicitly treated as comparable across skills and across models (Jadhav et al., 8 Apr 2026).

3. Models evaluated and empirical skill profile

The evaluated models are LLaMA 3.3 70B, Mistral Large, Qwen 2.5 72B, and Gemini 2.5 Flash. The selection is described as spanning different geographies, architectural or training regimes, and licensing models: Meta and Google in the United States, Mistral AI in France, and Alibaba in China; open-source and closed-source systems are both represented (Jadhav et al., 8 Apr 2026).

At the model-wide level, average scores across all skills lie in a narrow range: Mistral Large 60.0, LLaMA 3.3 70B 58.2, Qwen 2.5 72B 56.7, and Gemini 2.5 Flash 56.4. The paper treats the resulting 3.6-point spread as evidence of strong cross-model convergence, arguing that text-based automation feasibility appears more skill-dependent than model-dependent across current frontier systems (Jadhav et al., 8 Apr 2026).

The most automatable skills in the benchmark are Mathematics (73.2) and Programming (71.8). The lowest-scoring are Active Listening (42.2) and Reading Comprehension (45.5), followed by Speaking (48.5), Writing (51.0), and Social Perceptiveness (51.5). These lower-ranked skills are concentrated in Content and Social categories and are also described as very important across many occupations (Jadhav et al., 8 Apr 2026).

Skill pattern	Skills	SAFI
Highest	Mathematics; Programming	73.2; 71.8
Lowest	Active Listening; Reading Comprehension	42.2; 45.5
Other low scores	Speaking; Writing; Social Perceptiveness	48.5; 51.0; 51.5

At the category level, Technical Skills have the highest mean SAFI at 62.1 with within-category $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$3, followed by Systems Skills at 60.0 and Complex Problem Solving and Resource Management at 59.2. Content Skills have the lowest mean at 53.1 and the highest variance, which the paper attributes to the coexistence of highly structured skills such as Mathematics and weakly captured communicative skills such as Active Listening within the same O*NET category (Jadhav et al., 8 Apr 2026).

4. SAFI, the Anthropic Economic Index, and the AI Impact Matrix

SAFI is linked to real-world AI use through the Anthropic Economic Index (AEI), which contributes 756 occupations with AI exposure scores, 17,998 O*NET tasks with task penetration rates, and 3,364 tasks classified into AI–human interaction modes: directive, feedback loop, task iteration, validation, and learning. The paper computes, for each of the 35 skills, the Pearson correlation between O*NET skill importance across occupations and AEI exposure scores, thereby creating a second axis that complements SAFI’s capability measure (Jadhav et al., 8 Apr 2026).

These two axes define the AI Impact Matrix. Each skill $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$4 is positioned as:

$$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$5

where the x-axis is the skill’s exposure correlation and the y-axis is SAFI. The matrix is divided into four qualitative quadrants: Higher Displacement Risk, AI-Augmented, Upskilling Window, and Lower Displacement Risk. The paper is explicit that this is a heuristic interpretive framework, not a predictive model of employment loss (Jadhav et al., 8 Apr 2026).

The canonical Higher Displacement Risk example is Programming, with SAFI 71.8 and exposure correlation +0.455. By contrast, Reading Comprehension (+0.372), Writing (+0.369), and Active Listening (+0.338) are concentrated in AI-exposed occupations but have relatively low SAFI scores, placing them in the AI-Augmented quadrant. This mismatch is summarized as a “capability-demand inversion”: skills most concentrated in AI-exposed jobs are often the skills on which the benchmarked LLMs perform least well (Jadhav et al., 8 Apr 2026).

The inversion is quantified as a negative cross-skill relationship between SAFI and exposure correlation: Pearson $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$6, $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$7, $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$8 and Spearman $$\text{SAFI}(s) = \frac{100}{|M|} \sum_{m \in M} \frac{1}{|T_s|} \sum_{t \in T_s} \frac{\text{score}(t, m)}{10}$$9, $M$0, $M$1. The paper notes that these correlations are not statistically significant at conventional levels, but both point in the same negative direction. It interprets the pattern as evidence that current AI adoption is concentrated in jobs where AI is primarily used to augment difficult human skills rather than to automate them away (Jadhav et al., 8 Apr 2026).

This interpretation is reinforced by the AEI interaction data: 78.7% of observed AI interactions are classified as augmentation, including 32.5% feedback loops, 29.9% learning interactions, and other collaborative modes such as task iteration and validation; only 21.3% are directive automation. A plausible implication is that SAFI is most informative when read jointly with observed interaction modes rather than as a standalone replacement index (Jadhav et al., 8 Apr 2026).

5. Conceptual significance and relation to adjacent research

SAFI belongs to a broader research shift from occupation-level automation narratives toward skill- and task-level analysis. The paper’s focus on O*NET skill units closely aligns with the “skill automation view”, which argues that automation substitutes for skills rather than occupations and studies how removing automatable skills changes the structure of the occupation network (Lee et al., 2024). Within that perspective, SAFI provides a direct capability-oriented ranking of the text-based components of the 35 O*NET skills rather than a probability of occupation disappearance.

Related work also sharpens the distinction between automation and augmentation. The 2ACT framework models six human–AI usage patterns—Directive AI, Feedback Loop AI, Validation AI, Task Iteration AI, Learning AI, and Thinking Fraction—and finds that automation-focused usage patterns predict lower job-zone placement, whereas augmentative patterns interact positively with cognitive and scientific capabilities in specific “skill bridges” (Mullens et al., 12 May 2025). This suggests that SAFI’s AI Impact Matrix is best read as one layer in a larger account of occupational change: a high SAFI indicates technical feasibility for text-based execution, but actual career consequences depend on usage mode.

Other adjacent work defines automation exposure in more theory-driven terms. A Moravec-inspired exposure index scores O*NET tasks on performance variance, data abundance, tacit knowledge, and algorithmic efficiency gap, explicitly targeting fundamental automatability rather than current capability snapshots (Schaal, 15 Oct 2025). SAFI differs in being benchmark-based and current-model-based, but both approaches converge on a non-routine pattern in which many cognitive and digital tasks are highly exposed while embodied and manual tasks remain relatively resistant.

A further extension comes from workflow theory. A model of AI automation as chains of contiguous AI-executed steps argues that comparative-advantage logic can fail once firms bundle adjacent steps into AI chains, and reports empirical evidence that AI-executed steps co-occur in chains, that dispersion lowers job-level AI execution, and that adjacency to AI-executed steps increases the likelihood of execution by AI (Demirer et al., 14 Jun 2026). This suggests that SAFI’s skill-level values may not be sufficient for organizational analysis unless they are also situated within workflow structure.

6. Limitations, uses, and methodological extensions

The paper identifies seven main limitations. First, SAFI is a text-only representation of skills and should be treated as an upper bound on text-amenable components. Second, the heuristic scoring engine does not verify factual correctness or deep domain accuracy and may under- or mis-estimate nuanced Content and Social skills. Third, 263 tasks across 35 skills is broad but still limited relative to real-world variation, and the number of tasks per skill varies. Fourth, the AEI adoption data are derived from Claude and predominantly English-language contexts. Fifth, the results are a cross-sectional snapshot of frontier models in early 2026. Sixth, SAFI is not employment-weighted. Seventh, the AI Impact Matrix is interpretive rather than predictive (Jadhav et al., 8 Apr 2026).

These limitations define the proper use of the index. For policymakers, the paper proposes using SAFI together with the AI Impact Matrix to differentiate retraining strategies by quadrant, rather than deploying a single “AI readiness” program. For employers, it recommends identifying roles concentrated in Quadrant I skills for redeployment strategies, consistent with JPMorgan’s “displace and redeploy” approach, and formalizing co-pilot usage and output-verification practices in Quadrant II roles. For workers and educators, it recommends treating the near-term shift as movement from first-draft production to last-mile judgment, with curricular emphasis on skills that combine high labor demand and low SAFI, alongside AI-emergent skills such as prompting, output evaluation, and workflow design (Jadhav et al., 8 Apr 2026).

The broader literature points toward several methodological extensions. Capability-driven skill generation in industrial automation treats capabilities as contracts for skill implementations and uses retrieval-augmented generation to produce executable skills across interfaces and languages, emphasizing the role of structured capability descriptions and interface reuse (Silva et al., 6 May 2025). Skill-centered agent evaluation frameworks such as SkillAudit measure utility, efficiency or cost, and safety of deployable skills through baseline comparison and task generation, finding that over 7% of scanned real-world skill packages were in risky status (Yu et al., 21 Jun 2026). A survey of agent skill evaluation and evolution similarly argues for multi-dimensional scoring across performance, safety, generalization, efficiency, human oversight, and ecosystem compatibility (Ding et al., 9 Jun 2026). Together, these results suggest that SAFI’s benchmark-based capability score could plausibly be extended into a broader multi-axis family of indices rather than remaining a single scalar.

A final implication emerges from work on skill-level evaluation outside labor-market measurement. An annotation study on schema-guided behavioral profile labeling argues that automatic annotation should be evaluated in terms of skill feasibility rather than task-level automation, distinguishing directly operable, recoverable, and structurally underspecified skills (Wu, 16 Apr 2026). A software scripting study likewise treats skills as verified reusable scripts discovered through offline simulation, showing higher automation success rates and lower runtime cost than runtime code generation (Xu et al., 29 Apr 2025). These findings reinforce the central SAFI proposition: automation feasibility is often heterogeneous at the skill level, and task- or occupation-level summaries obscure that heterogeneity.

SAFI therefore occupies a specific methodological position. It is a reproducible, model-agnostic benchmark score for the text-based execution of O*NET skills; it becomes most informative when paired with observed AI use, workflow structure, and transition analysis; and it is least informative when reinterpreted as a direct measure of job loss, physical substitutability, or deterministic labor-market displacement (Jadhav et al., 8 Apr 2026).