Structured Capabilities Model Overview

• Structured Capabilities Model is an approach that represents capability as explicit, organized components rather than a single undifferentiated score.
• It underpins advancements in LLM evaluation, human-AI collaboration, ML engineering, and cybersecurity by providing clear, measurable specifications.
• The model enhances explanation and operational decision-making by decomposing performance into latent factors, typed profiles, and behavior-specific metrics.

Searching arXiv for recent and directly relevant papers on structured capabilities models and related capability-based frameworks. A structured capabilities model is an approach to representing capability as an explicit structure rather than as a single undifferentiated score, label, or benchmark average. In recent arXiv work, that structure appears in several forms: latent, correlated factors for LLMs; typed partitions of user expertise for professional human-LLM collaboration; fine-grained behavioral specifications for ML engineering; weighted capability domains with tiered practices and metrics for cybersecurity maturity; and formal capability descriptions of security controls for policy refinement and incident management (Burnell et al., 2023, Yang et al., 14 May 2026, Yang et al., 2022, Liyanage et al., 2 Apr 2025, Basile et al., 2024). Across these formulations, the common premise is that performance, oversight, or operational fitness is better understood when capability is decomposed into organized components rather than treated as monolithic.

1. Core idea and major formulations

The concept is motivated by a recurring limitation of coarse evaluation. In the LLM setting, aggregate benchmark averages do not explain why a model performs differently across tasks, whereas latent factors extracted from task covariation can supply explanatory and predictive structure (Burnell et al., 2023). In professional human-LLM collaboration, style adaptation, preference alignment, context memory, and persona imitation do not model whether a user can actually evaluate the system’s reasoning in a given domain, which motivates capability-aware interaction governance rather than style-based personalization (Yang et al., 14 May 2026). In ML engineering, accuracy or F1 can hide systematic blind spots, and capabilities are therefore framed as behavior-centric and scenario-specific specifications (Yang et al., 2022). In cybersecurity, capability maturity models are recast around tailored domains, stratified practices and metrics, and weighted maturity scoring, while security controls themselves are modeled by the capabilities they offer for enforcing security policies (Liyanage et al., 2 Apr 2025, Basile et al., 2024).

Work	Structured unit	Immediate purpose
(Burnell et al., 2023)	Reasoning, comprehension, core language modeling	Explain task-performance covariance across LLMs
(Yang et al., 14 May 2026)	Strong, mixed, weak domains	Condition intervention behavior on user evaluative capability
(Yang et al., 2022)	Fine-grained behavior specifications	Unite design, debugging, evaluation, and maintenance
(Liyanage et al., 2 Apr 2025)	Core and elective domains; tiered practices and metrics	Assess cybersecurity capability maturity
(Basile et al., 2024)	Conditions, actions, events, evaluation, resolution, default actions	Compare controls and translate policy into configuration

A plausible implication is that the phrase “structured capabilities model” does not denote a single formalism. It denotes a family of modeling choices that make capability machine-processable, analyzable, and operationally actionable, with the choice of structure depending on whether the target is a model, a human collaborator, an engineering workflow, or an organizational or security-control artifact.

2. Latent capability structure in LLMs

A direct psychometric formulation appears in work that analyzes data from 29 different LLMs across 27 cognitive tasks from the HELM benchmark dataset and argues that LLM capabilities are not monolithic (Burnell et al., 2023). The framework treats models as the “subjects” of factor analysis and uses the standard exploratory factor-analysis model

$$\mathbf{x} = \boldsymbol{\Lambda}\mathbf{f} + \boldsymbol{\epsilon}$$

with covariance

$\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$

A frequentist maximum-likelihood exploratory factor analysis is paired with the Hull method for factor-count selection and oblimin rotation; varimax gives similar results. A Bayesian exploratory factor analysis, using the method of Conti et al. (2014), is then used as a robustness check. The scree plot and eigenvalues, the Hull method, the Bayesian posterior over factor count, and the interpretability of loadings all converge on a 3-factor solution (Burnell et al., 2023).

The three factors are interpreted as comprehension, core language modeling, and reasoning. Comprehension loads strongly on tasks such as NaturalQuestions, XSUM, and NarrativeQA and is described as the ability to process and extract meaning from input text. Core language modeling is defined by next-token prediction tasks such as ICE, TwitterAAE, and The Pile. Reasoning loads strongly on GSM8K, the two synthetic reasoning tasks, BBQ, and TruthfulQA; LSAT is moderately associated with this factor in the frequentist model, whereas LSAT is the main discrepancy in the Bayesian model (Burnell et al., 2023).

The frequentist exploratory factor analysis reports variance explained of 33% for the first factor, 31% for the second, and 17% for the third, for a cumulative 82% (Burnell et al., 2023). The model-size correlations are positive for all three factors—0.70 for comprehension, 0.49 for language modeling, and 0.51 for reasoning—while instruction tuning is negatively correlated with language modeling at -0.50, positively correlated with reasoning at 0.44, and weak or not significant with comprehension at 0.23 with a wide interval. The authors do not find clear correlations between factors and training tokens, but emphasize that the range of training lengths in the dataset is limited (Burnell et al., 2023).

Several caveats are integral to the model’s interpretation. The sample of 29 models is small for factor analysis; fit statistics are not especially strong, with CFI = 0.70, TLI = 0.61, and RMSEA = 0.26 (Burnell et al., 2023). Factor naming remains partly interpretive, and some tasks do not map cleanly to expectations: bAbI and Dyck load weakly, HellaSwag loads on comprehension rather than reasoning, and some bias or truthfulness tasks load on reasoning. The paper’s stated conclusion is therefore neither a single general-capability account nor a highly fragmented account, but a multifaceted but compact capability space (Burnell et al., 2023).

3. Typed user capability profiles and capability-conditioned scaffolding

A second formulation moves from model capability to user evaluative capability. “Capability Conditioned Scaffolding” is introduced as a typed architectural framework for professional human-LLM collaboration that partitions expertise into strong, mixed, and weak domains and conditions intervention behavior on structured capability profiles (Yang et al., 14 May 2026). Its motivating problem is “Professional Domain Drift,” defined as the situation where AI-assisted work expands beyond the user’s reliably evaluable expertise into adjacent domains where the user may still feel professionally entitled or contextually able to judge, but actually cannot validate the model’s reasoning well. The concern is not only that the model may be wrong, but that oversight becomes procedural rather than critical and fluent AI outputs create a false sense of competence and legitimacy (Yang et al., 14 May 2026).

The typed profile is operationalized as: $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$4 and the routing policy is: $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$5 This routing mechanism reads the structured user capability profile, classifies the prompt/domain relation, compares prompt evidence to declared expertise, and chooses an intervention intensity. The framework therefore uses consistency-based guardrails that compare declared expertise against prompt-level evidence, especially in mixed domains (Yang et al., 14 May 2026).

The evaluation uses MMLU subsets as a controlled proxy for professional reasoning. The 10 selected subsets are Machine Learning, College Computer Science, Professional Psychology, Formal Logic, Econometrics, Clinical Knowledge, Medical Genetics, Professional Law, Philosophy, and European History. Each subset contributes 30 questions, and the paper reports 480 question-condition pairs after applying the conditions/profile filtering. Replication is carried out across four LLM substrates: Claude Sonnet 4.5, Claude Haiku 4.5, GPT-4.1, and gpt-5.5 (Yang et al., 14 May 2026).

Two profiles are tested. PCS-NLP treats Machine Learning and College Computer Science as strong; Professional Psychology, Formal Logic, and Econometrics as mixed; and Clinical Knowledge, Medical Genetics, Professional Law, Philosophy, and European History as weak. PCS-LitProf inverts the technical/humanistic axis: Philosophy and European History are strong; Professional Psychology and Formal Logic are mixed; and Machine Learning, College Computer Science, Clinical Knowledge, Medical Genetics, and Professional Law are weak (Yang et al., 14 May 2026). The evaluation shows profile swapping and categorical inversion: under PCS-NLP, intervention in strong ML/CS subsets is 3.3% (1/30), while under PCS-LitProf the same ML/CS questions are weak-domain questions and activation rises to 100% (30/30). Under PCS-NLP, medical/legal subsets are 100% and humanities subsets are 90%; under PCS-LitProf, humanities drop to 0% while medical/legal remain high at 73.3% in the listed table (Yang et al., 14 May 2026).

The mixed domains form the paper’s “PDD risk zone,” and activation is differentiated rather than binary. Under PCS-NLP, Professional Psychology is 73%, Formal Logic is 13%, and Econometrics is 3% (Yang et al., 14 May 2026). The reported statistics are Fisher’s exact tests with $p < 10^{-13}$ for key comparisons and a permutation test with $p < 10^{-4}$ with 10,000 permutations. The paper presents these results as evidence that capability-aware scaffolding conditions intervention behavior on the structured capability profile rather than on prompt topic alone (Yang et al., 14 May 2026).

4. Fine-grained behavioral capabilities in ML engineering

A third formulation treats capability as a specification artifact for the ML lifecycle. In “Capabilities for Better ML Engineering,” a capability is defined as “a fine-grained specification of behaviors expected of an ML model” (Yang et al., 2022). Capabilities are explicitly contrasted with hard software specifications. A software specification is a hard constraint for which a single violating case is a bug, whereas an ML capability is a soft lower-bound specification assessed by failure rates rather than binary pass/fail. The associated formal notions are qualitative rather than axiomatic: capability as a fine-grained behavior specification, instantiation as deriving test cases from a capability, and failure rate as the empirical measure of how often the model violates the capability (Yang et al., 2022).

The paper’s central engineering claim is that capabilities can act as a unifying abstraction across model design, requirements and collaboration, testing and debugging, external quality assurance, deployment, and maintenance (Yang et al., 2022). Capabilities are described as finer-grained than standard evaluation metrics, more holistic than a single benchmark score, and usable as a boundary object across stakeholders. The framework also assumes a hierarchical structure in which a broad capability may contain narrower sub-capabilities; the paper gives “understanding negation” and more specific forms such as double negation or words like “hardly” and “never” as an example (Yang et al., 2022).

Instantiation is the process of deriving test data from capabilities. The paper enumerates slicing existing data, transforming examples, generating templated examples, and targeted curation of new data, possibly with crowdsourcing (Yang et al., 2022). In a pedestrian-detection example, “Recognize pedestrians in wheelchairs” is instantiated by curating images with wheelchair pedestrians; “Robust to extreme weather” is instantiated by transforming sunny images to rainy; and “Detect pedestrians of all ages” is instantiated by slicing test data by pedestrian age. These examples illustrate how capabilities encode important outliers, anticipated distribution shift, and concept variation (Yang et al., 2022).

The preliminary experiment tests whether capabilities reflect generalizability. It uses 100 BERT sentiment models fine-tuned with different random seeds on Amazon-WILDS, with Home-and-kitchen reviews as the source domain and 10 other domains as target domains (Yang et al., 2022). Eight sentiment-analysis capabilities adapted from prior work on linguistic phenomena are instantiated by slicing validation data on keywords: negation, negation (v2), shifter, modality, comparative, mixed, reducer, and amplifier. For each target domain, a linear model predicts target accuracy from source domain accuracy plus capability test performance, and adjusted $R^2$ is compared with and without capability variables; approximate source-target distances are computed using the $\mathcal{A}$-distance proxy (Yang et al., 2022). The reported finding is that on 50% of target domains, adding capability tests significantly improves adjusted $R^2$, and that the farther the target domain is from the source, the more useful capability tests become for prediction (Yang et al., 2022).

The paper also identifies unresolved design issues. Different capabilities can be complementary, redundant, or conflicting; some are too broad to be informative; and simple keyword slicing is acknowledged as easy but often flawed (Yang et al., 2022). The open problems include identifying capabilities, deciding capability granularity, communicating capabilities across stakeholders, selecting instantiation strategies, and comparing information gain and cost-benefit trade-offs across capability suites.

5. Capability domains, maturity scoring, and formal control semantics in cybersecurity

In cybersecurity, structured capabilities models appear in both organizational maturity assessment and the formal semantics of security controls. The Cybersecurity Capability Maturity Framework is presented as a holistic, flexible, and measurable alternative to existing cybersecurity capability maturity models that are described as rigid, one-size-fits-all, complex, and lacking quantitative metrics (Liyanage et al., 2 Apr 2025). Its three main building blocks are Tailored Capability Domains, Stratified Practices and Metrics, and a Maturity Scoring System. Capability domains are divided into Core Domains and Elective Domains. The core domains are Risk Management; Asset & Configuration Management; Identity & Access Management; Data Security; Incident Response; Cybersecurity Culture, Awareness & Training; and Cybersecurity Governance. The elective domains are Network Security; Endpoint Security; Cloud Security; Application Security; Physical Security; Supply Chain & External Dependencies Management; Security Architecture & Design; Situational Awareness; Threat Intelligence & Monitoring; Business Continuity & Disaster Recovery; Workforce Management; Communication, Collaboration, & Information Sharing; Compliance & Legal; and Performance Evaluation & Improvement (Liyanage et al., 2 Apr 2025).

The scoring system combines a Practice Implementation Score, a Metric Achievement Score, domain scores, weighted domain scores, an Overall Maturity Score, and maturity-level derivation (Liyanage et al., 2 Apr 2025). Each practice is rated on a 3-level Likert scale—Not Implemented = 0, Partially Implemented = 1, Fully Implemented = 2—while each metric is scored on a 0–3 scale using quantitative thresholds or qualitative rubrics. The overall score is given as

$OMS = \sum_{i=1}(W_i \times DS_i),$

where $W_i$ is the weight for domain $i$ and $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$0 is the domain score for domain $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$1 (Liyanage et al., 2 Apr 2025). Domain weights are derived through a Weighted Sum Model using Risk Impact, Compliance Requirement, Business Impact, and Interdependency, each scored from 1–3 and then normalized. The maturity levels are Initial for $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$2, Managed for $\Sigma = \Lambda \Phi \Lambda^\top + \Psi.$3, and Optimized for the upper interval above 66 (Liyanage et al., 2 Apr 2025). The framework is grounded in Design Science Research and implemented in a web-based tool, with planned validation through Delphi expert review, user testing, and a field study (Liyanage et al., 2 Apr 2025).

A different cybersecurity formulation, the Security Capability Model, formalizes the capabilities of security controls so that tools can reason about them independently of specific vendors and translate abstract security intent into device-specific configuration (Basile et al., 2024). The model has two layers: an Information Model and a Data Model. The Information Model covers rules and policies in terms of conditions, actions, events, condition evaluation, resolution strategies, and default actions. The principal capability families are ConditionCapability, ActionCapability, EventCapability, EvaluationCapability, ResolutionStrategyCapability, and DefaultActionCapability (Basile et al., 2024). The model is implemented using UML, follows the decorator pattern, and includes classes such as NSF, SecurityCapability, NSFCatalogue, HasSecurityCapabilityDetails, NSFPolicyDetails, CapabilityTranslationDetails, and ResolutionStrategyDetails (Basile et al., 2024).

The Data Model instantiates these abstractions for controls such as IpTables, XFRM, strongSwan, and Squid (Basile et al., 2024). The paper reports a catalogue with seven security controls and, across them, 270 condition capabilities and 167 action capabilities, plus one common event capability, one common resolution strategy (FMR), and two evaluation forms (CNF, DNF). From each control description, the system automatically generates an abstract XML Schema-based language containing only the constructs and capabilities the target control supports. Translation metadata in HasSecurityCapabilityDetails and CapabilityTranslationDetails supplies command names, conditional variants, dependencies, range or set handling, and rule-expansion behavior. The model is validated in applications to policy refinement, security control comparison and migration, and incident response, including use in the FISHY and PALANTIR scenarios and interoperability with CACAO and MISP reports (Basile et al., 2024). A plausible implication is that the structured capabilities model here functions not as a maturity taxonomy but as a semantics-preserving interoperability layer between abstract policy and operational enforcement.

6. Interpretation, limits, and recurring controversies

Across these works, a central misconception is that “capability” names a single scalar property. The LLM factor-analysis paper argues explicitly against a single undifferentiated “general capability” view and also against a highly fragmented view in which every benchmark probes a wholly unrelated skill (Burnell et al., 2023). The professional collaboration paper similarly rejects the assumption that a user is adequately modeled as a stylistic preference holder or a generic professional persona, insisting instead on uneven domain competence and domain-specific evaluative capacity (Yang et al., 14 May 2026). The ML engineering paper rejects the sufficiency of coarse aggregate metrics, while the cybersecurity papers reject either control checklists without effectiveness measurement or vendor-specific configuration languages without formal capability descriptions (Yang et al., 2022, Liyanage et al., 2 Apr 2025, Basile et al., 2024).

The limits are equally recurrent. The LLM latent-factor model is preliminary because only 29 models are analyzed, fit statistics are not ideal, and factor interpretation depends partly on human judgment (Burnell et al., 2023). Capability Conditioned Scaffolding is described as only a pilot validation: it is single-turn only, measures no real user outcomes, provides no longitudinal trust calibration, and uses benchmark prompts rather than full professional workflows (Yang et al., 14 May 2026). The ML engineering framework is promising but immature, with capability discovery, granularity, communication, and instantiation still open research problems, and with simple keyword slicing acknowledged as flawed (Yang et al., 2022). The Cybersecurity Capability Maturity Framework is a short paper or proposal that describes a validation roadmap rather than full empirical validation (Liyanage et al., 2 Apr 2025). The Security Capability Model currently focuses mainly on traffic filtering and channel protection, only partially captures stateful packet filters, and does not treat reverse translation from device configuration to abstract policy as a major design goal (Basile et al., 2024).

Taken together, these works suggest that a structured capabilities model is best understood as a modeling strategy rather than a settled ontology. Its unifying feature is the replacement of monolithic evaluation with an explicit capability structure that can support explanation, routing, debugging, benchmarking, maturity assessment, or policy translation. Its unresolved question is not whether capability should be structured, but which structure is appropriate for a given target system, stakeholder set, and operational objective.