SkillOpt-Lite: Minimal Skill Optimization

• The paper introduces SkillOpt-Lite, a minimal, validation-gated framework that optimizes external skill files via zeroth-order methods on frozen large language models.
• SkillOpt-Lite is defined by an iterative process that uses file-based trajectory exploration and consensus attribute mining to diagnose and patch recurring failure patterns.
• Empirical evaluations on benchmarks like LiveMath and SpreadsheetBench demonstrate significant performance gains, underscoring the efficiency of a lightweight, frozen-backbone adaptation strategy.

SkillOpt-Lite is a minimal skill-optimization framework for autonomous agents with frozen base models, introduced as a reduction of more elaborate self-evolution pipelines to the smallest set of components the authors regard as theoretically or empirically necessary (Shen et al., 3 Jul 2026). In its primary sense, a “skill” is an editable text artifact—typically Markdown or prompt-like code—that encodes domain-specific behavior, while the model itself remains fixed and all adaptation occurs through iterative edits to skills and, in an extended form, to harness code. In a broader 2026 usage, the label “SkillOpt-Lite” is also applied to lightweight relatives of skill optimization, including train-free hypothesis-validated skill curation and compact latent skill compression, both of which preserve the frozen-backbone premise while simplifying the optimization object or training loop (Shang et al., 21 Jun 2026, Tao et al., 18 Jun 2026).

1. Definition and scope

The central object in SkillOpt-Lite is the external skill file. The paper formalizes skill optimization as

$$f(s) = \mathbb{E}_{z \sim \mathcal{D}}[R(H(M, z, s))],$$

where $M$ is a frozen LLM, $H$ is the harness, $z$ is a task sampled from a distribution $\mathcal{D}$, $s$ is a text skill artifact, and $R$ is the task reward or performance metric (Shen et al., 3 Jul 2026). The objective is therefore not weight adaptation but improvement of an external textual policy through interaction data and self-editing.

This distinguishes SkillOpt-Lite from full SkillOpt, which treats the skill as a trainable external state but uses a more elaborate optimization stack, including mini-batch reflection pooling, learning-rate-like edit schedules, rejected-edit buffers, and multi-epoch meta refinement (Yang et al., 22 May 2026). SkillOpt-Lite asks which of those mechanisms are genuinely required. Its answer is a minimal loop built around trajectory logs, file-based inspection, small patches, and validation gating.

The phrase also acquired a secondary, looser meaning in adjacent work. HDSO describes itself as interpretable as a “SkillOpt-Lite” variant because it optimizes only an external repository of textual skill packages through paired validation, with both curator and executor frozen (Shang et al., 21 Jun 2026). SoftSkill is likewise presented as naturally readable as a “SkillOpt-Lite” because it starts from a Markdown skill and compresses it into a short latent prefix while leaving the backbone frozen (Tao et al., 18 Jun 2026). This suggests that the term denotes not a single representation format, but a methodological commitment to lightweight adaptation of external behavioral state.

2. Theoretical framing

SkillOpt-Lite is grounded in a zeroth-order formulation. Because the skill $s$ is discrete text and $H \circ M$ is non-differentiable, the framework treats optimization as a zeroth-order problem in which only function values can be observed through rollouts. The paper maps classical zeroth-order operators to prior agent-skill methods: single-trajectory reflection corresponds to a 1-point estimator; contrastive success-failure analysis resembles central difference; and localized edits resemble coordinate descent (Shen et al., 3 Jul 2026).

The principal conceptual departure from classical zeroth-order optimization is that agent rollouts are not blind perturbation queries. Each rollout yields rich semantic traces—plans, tool calls, errors, and logs—which the paper characterizes as interpretable debugging feedback rather than mere scalar evaluations. Skills are therefore treated less as opaque prompts and more as editable program-like artifacts whose failure modes can be diagnosed from execution traces.

The generalization argument is PAC-style. The paper emphasizes expected on-average stability and presents the bound

$$\epsilon(\mathcal{S}) \le \hat{\epsilon}_D(\mathcal{S}) + \mathcal{O}\left(\beta_{\exp} + \sqrt{\frac{\ln(1/\delta)}{N}}\right),$$

with stability controlled by avoiding updates that overfit single trajectories (Shen et al., 3 Jul 2026). Its operational consequence is “consensus attribute mining”: edits should target cross-trajectory invariants rather than idiosyncratic cases. The companion validation argument uses a disjoint hold-out set, yielding a bound in which the stability term disappears when validation is independent. This makes independent validation gating not a convenience but a constitutive part of the method.

3. Pipeline architecture

The SkillOpt-Lite pipeline collapses skill optimization to six recurring operations. First, the agent is run on a batch of training tasks with the current skill library. Second, each trajectory is dumped to disk as a separate text file containing plans, actions, tool calls, observations, errors, and final result. Third, an optimizer model, implemented as a coding agent, explores those files using ordinary file-system tools rather than receiving all logs in a single prompt. Fourth, it infers recurring failure patterns and proposes minimal patches to skill files. Fifth, the patched skill is evaluated on an independent validation set. Sixth, the patch is accepted only if the validation metric improves; otherwise it is rejected and the current best skill is retained (Shen et al., 3 Jul 2026).

Three design principles govern that loop. The first is file-system-based trajectory exploration, derived from the “everything is a file” or Claude Code philosophy. The second is consensus attribute mining, which replaces bespoke map-reduce reflection infrastructures with direct inspection of multiple trajectory files for repeated failure attributes. The third is independent validation gating, which rejects the widespread practice of validating on modified training failures or subsets of training tasks.

The implementation is deliberately ordinary. Skills are plain code or Markdown files, trajectories are text logs, and the system tracks a best_skill artifact on disk. The optimizer is not a special-purpose controller but a coding-capable LLM. The paper describes integration with VSCode Copilot through a slash-command interface: the command runs a training batch, logs trajectories, invokes the optimizer on those files, applies patches, evaluates on validation data, and reports updated metrics (Shen et al., 3 Jul 2026).

4. Empirical performance

SkillOpt-Lite is evaluated on six benchmarks—SearchQA, SpreadsheetBench, ALFWorld, LiveMath, DocVQA, and OfficeQA Pro (offline)—across GPT-4o, GPT-5.4-nano, GPT-5.4-mini, GPT-5.4, and GPT-5.5, with aligned optimizer models across SkillOpt and SkillOpt-Lite (Shen et al., 3 Jul 2026). For LiveMath and OfficeQA, the validation split was enlarged from 2:1:7 to 2:2:6 because the original validation sets of 10–20 examples produced high variance.

The headline results are reported against full SkillOpt. On LiveMath, GPT-5.5 improves from 64.8 under SkillOpt to 73.6 under SkillOpt-Lite, a gain of +8.8 points; GPT-5.4-nano improves from 30.3 to 55.7, a gain of +25.4 points (Shen et al., 3 Jul 2026). On SpreadsheetBench, GPT-5.4 moves from 61.5 to 79.4, GPT-5.5 from 76.2 to 79.7, and GPT-5.4-nano from 51.6 to 66.2. SearchQA is described as slightly better or comparable, ALFWorld as showing slight improvements that often push to 100%, OfficeQA as consistently better by roughly 2–5 points, and DocVQA as showing small but consistent gains.

The convergence behavior is central to the paper’s claim of minimal sufficiency. Over 10 batches, SkillOpt-Lite improves more rapidly in the first 2–3 iterations on LiveMath and Spreadsheet, and its final ceilings are higher or equal to those of SkillOpt (Shen et al., 3 Jul 2026). The reported efficiency gain derives not from a new inference mechanism but from removing multi-epoch tree reductions, mini-batch patch merging, rejected-edit buffers, and slow-damping machinery.

A pilot experiment illustrates both the promise and the limit of minimalism. A single-batch coding-agent pass over raw trajectory files, with no intermediate validation, beat full SkillOpt after 4 epochs on LiveMath and DocVQA, but regressed below the initial skill on SpreadsheetBench (Shen et al., 3 Jul 2026). The paper treats this as evidence that simple file-level reasoning by strong models is already powerful, while also showing that validation gating remains indispensable.

5. HarnessOpt and full harness optimization

SkillOpt-Lite extends beyond prompt or skill text to full harness optimization under the name HarnessOpt. In this setting, the editable object includes tool wrappers, control flow logic, retry policies, observation formatting, and state memory. The optimization loop remains zeroth-order: propose harness or skill changes, evaluate them on held-out tasks, and retain only improvements (Shen et al., 3 Jul 2026).

HarnessOpt begins with a “Round-0 harness” diagnostic phase in which logs are analyzed for systemic architectural issues along five axes: tool inventory, prompt context, loop policy, codegen shape, and state memory. Structural proposals are reviewed by a human inside VSCode. Thereafter, automated continuous harness evolution permits the coding agent to patch harness scripts directly, subject to allowlist constraints, compile checks, smoke tests, full validation, reversible git-based rollback, and feature-flagged deployment controls.

The main quantitative case study is SpreadsheetBench. There, HarnessOpt enables GPT-5.4-nano to achieve 0.7758 accuracy, outperforming GPT-5.5 running standard pipelines at 0.7620 (Shen et al., 3 Jul 2026). The broader HarnessOpt-with-skill setting reaches up to 0.8577 accuracy for GPT-5.5. Qualitative analyses attribute gains to changes such as expanded spreadsheet previews, explicit final-output introspection, recognition of repetitive states, and fallback policies when tool-driven reasoning loops stall.

In deployment terms, the paper’s “one line of vibe” slogan refers to the user interface rather than to a one-shot algorithm. The human issues a single command in the IDE, but under that command lies a complete loop of rollout, logging, diagnosis, patching, validation, and update.

6. Relation to other lightweight skill paradigms

A broader view of the 2026 literature shows several distinct ways to make skill optimization “lite.”

System	Optimization object	Admission or use mechanism
SkillOpt-Lite	Editable skill files; in HarnessOpt, harness code	Minimal patches with independent validation gating
HDSO	External repository $M$0 of structured skill packages	Paired control/treatment validation and progressive disclosure
SoftSkill	Soft prefix $M$1	NTP-trained prefix with frozen backbone and validation-selected checkpoints
OpenSkillEval	Skill-augmented systems and skills as evaluation targets	Dynamic task construction, trajectory analysis, and artifact analysis

HDSO is the most conservative of these variants. It treats each persistent skill update as a falsifiable hypothesis with explicit applicability conditions, risks, and a validation plan; candidate skills are accepted only after staged paired control/treatment evaluation, and deployed through progressive disclosure so that the executor-only path is preserved when no skill is selected (Shang et al., 21 Jun 2026). On ALFWorld, it improves Qwen3-8B by +6.9 Avg. SR points and Qwen3.6-27B by +4.0 points, while preserving a +7.1-point gain under 20% randomly flipped success/failure feedback during optimization.

SoftSkill moves in a different direction. It compresses a natural-language skill into a short sequence of virtual embeddings, typically a length-32 prefix, parameterized as $M$2, and trained with next-token prediction while the backbone remains frozen (Tao et al., 18 Jun 2026). In single-round settings on Qwen3.5-4B, it improves over no-skill prompting by 8.3 points on SearchQA, 42.1 points on LiveMath, and 1.3 points on DocVQA; relative to SkillOpt, it improves accuracy by 5.2 points on SearchQA and 12.5 points on LiveMath. The same paper, however, treats agentic execution as a harder boundary case, where sparse trajectory imitation does not yet robustly compress long-horizon procedural behavior.

OpenSkillEval is not an optimizer, but it supplies an important empirical counterpoint. Across more than 600 dynamically generated task instances and 30 open-source skills, it finds that skill availability does not guarantee effective skill usage, that benefits depend strongly on model and framework, and that many popular skills do not consistently outperform no-skill baselines (Ying et al., 22 May 2026). That evaluation perspective is directly relevant to any SkillOpt-Lite deployment, because it places selection and invocation under measurable cost–performance trade-offs rather than assuming that more skill content is always better.

7. Limitations and interpretation

SkillOpt-Lite is constrained by the same premise that defines it: the foundation model remains frozen, and improvement is sought only through skills and harnesses. Its success therefore depends on informative logs, a meaningful held-out validation set, and safe editability of the surrounding agent code (Shen et al., 3 Jul 2026). For HarnessOpt in particular, the paper stresses allowlists, sandboxes, compile checks, smoke tests, and reversible changes.

Several misconceptions are explicitly contradicted by the current literature. Minimal skill optimization is not equivalent to unrestricted self-editing: the SpreadsheetBench pilot shows that removing validation can produce regressions even when a strong coding agent diagnoses logs correctly (Shen et al., 3 Jul 2026). Nor does the mere presence of skills guarantee their use: OpenSkillEval reports that, in default settings, skills are explicitly read only about 48% of the time across agents, and only about 20% of the time for Claude Opus; under force-using instructions, read rate rises to about 94%, with earlier access on average (Ying et al., 22 May 2026). Likewise, lightweight skill mechanisms are not uniformly portable across tasks. HDSO notes cross-model curation fragility when curator diagnosis, executor capability, and validation evidence do not align, while SoftSkill reports that agentic trajectory imitation remains brittle in long-horizon settings (Shang et al., 21 Jun 2026, Tao et al., 18 Jun 2026).

Taken together, these results position SkillOpt-Lite as a methodological program rather than a single engineering trick. In its narrow sense, it is a file-centric, coding-agent-based, validation-gated reduction of skill evolution to a minimal zeroth-order loop (Shen et al., 3 Jul 2026). In its broader sense, it names a class of frozen-backbone adaptation schemes that optimize lightweight external behavioral state—textual skills, validated repositories, or latent prefixes—under stronger controls than unconstrained reflection or memory accumulation (Shang et al., 21 Jun 2026, Tao et al., 18 Jun 2026).