How2Everything: Mining the Web for How-To Procedures to Evaluate and Improve LLMs

Abstract: Generating step-by-step "how-to" procedures is a key LLM capability: how-to advice is commonly requested in chatbots, and step-by-step planning is critical for reasoning over complex tasks. Yet, measuring and improving procedural validity at scale on real-world tasks remains challenging and understudied. To address this, we introduce How2Everything, a scalable framework to evaluate and improve goal-conditioned procedure generation. Our framework includes How2Mine, which mines 351K procedures from 980K web pages across 14 topics and readily scales to larger corpora. From this pool we build How2Bench, a 7K-example evaluation set balanced across topics. To reliably score model outputs, we develop How2Score, an evaluation protocol that uses an LLM judge to detect whether a generation contains any critical failure that would prevent achieving the goal. For low-cost, reproducible evaluation, we distill a frontier model into an open 8B model, achieving 80.5% agreement with human annotators. How2Bench reveals clear scaling trends across model sizes and training stages, providing signal early in pretraining. Finally, RL using How2Score as a reward improves performance on How2Bench by >10 points across three models without systematic regressions on standard benchmarks, with gains robust to superficial source-document memorization or format compliance. Taken together, How2Everything shows how pretraining web data can support a closed loop of capability evaluation and improvement at scale.

• The paper introduces a web-scale framework (How2Everything) that mines, benchmarks, and optimizes procedural generation in LLMs.
• It utilizes multi-stage filtering and RL-based optimization with LLM-as-a-judge to ensure procedural validity and detect critical failures.
• Results indicate measurable improvements across model scales and diverse domains, enhancing real-world applicability of step-by-step instructions.

Systematic Evaluation and Improvement of Procedural Generation in LLMs with How2Everything

Motivation and Problem Setting

The production of reliable step-by-step procedures is a critical capability for LLMs, tightly coupled to real-world usability in domains ranging from troubleshooting and education to practical planning. Procedure generation exhibits unique evaluation challenges: accuracy demands correct completion of multi-step dependencies, domain coverage, execution feasibility, and avoidance of critical errors. Prior datasets and benchmarks for procedural generation suffer from constrained topical scope (often limited to domains like cooking) and unreliable evaluation metrics (string overlap or single-reference likelihood), limiting robust capability analysis and scalable optimization.

The How2Everything Framework

How2Everything operationalizes a web-scale pipeline to mine, evaluate, and optimize goal-conditioned procedural knowledge:

• How2MINE: Large-scale extraction of procedures from 980K web documents, stratified across 14 diverse topics, yielding a pool of 351K high-quality procedure instances. Multi-stage filtering (LLM-based and heuristics) ensures elimination of entity-specific, non-sequential, trivial, or nonsensical procedures and standardizes the format for evaluation and training.
• How2BENCH: Evaluation corpus constructed with balanced topical coverage (7K examples, 500/topic) from How2MINE, designed for benchmarking procedural validity across scales, from small (1B) to frontier models.
• How2TRAIN: Remaining mined examples serve as a heterogeneous, large-scale procedural training resource.
• How2SCORE and How2JUDGE: Critical-failure detection protocol leveraging LLM-as-a-judge, formalizing evaluation as binary classification of "has critical failure" vs. "no failure," anchored to procedural reference. How2JUDGE, an open 8B model distilled from GPT 5 annotations, yields 80.5% agreement with human annotators, enabling reproducible, cost-effective evaluation and RL reward signals.

Evaluation Methodology

Procedural validity is robustly defined via detection of omissions, contradictions, severe vagueness, and extraneous actions that preclude goal achievement or render instructions unusable, with anchor reference procedures serving as evaluation targets. Human validation of critical failure detection achieves moderate inter-annotator agreement ($\alpha=0.593$), supporting use as a reliable RL reward and ranking metric. LLM judges exhibit high agreement with human labels (up to 83%) and minimal self-preference bias vis-à-vis model family, supporting cross-family evaluations and performance comparisons.

Benchmarking Procedural Capability

How2BENCH demonstrates strong scaling properties:

• Performance correlates with model size and training progression, with measurable gains emerging early in pretraining (5% FLOPs for 1B models) and continuing through subsequent stages.
• Score distributions by reference step count (length) reveal monotonic increases in difficulty with more required steps. Logistic regression analyses confirm reference step count as the dominant predictor of failure rates, superseding resource count or residual verbosity.
• Benchmark ordering is not reducible to reference-step perplexity; Spearman rank correlation between How2Score and perplexity weakens in large models, indicating that procedural validity is distinct from reference-likelihood optimization.

Optimization via RL

How2Everything enables closed-loop optimization with scalable, low-cost RL:

• RL using How2SCORE as reward, evaluated by How2JUDGE, produces consistent improvements: e.g., Qwen3-8B-Instruct improves by $+10.1$ points on How2Bench post-RL with no systematic out-of-domain regressions, including robustness to external judges.
• Length-control during RL prevents gaming through verbosity, keeping average generation-token ratios tightly bounded.
• RL gains depend on pretraining maturity and broad topical training coverage, confirming that improvements are not driven solely by format imitation or data contamination.

Robustness to Confounds

Targeted analyses invalidate alternative explanations for observed improvements:

• Format Compliance: Gains increase substantially only with strong pretraining checkpoints and sufficient topic diversity, indicating content-level (not format-level) enhancement.
• Source Memorization: Aggressive document contamination during midtraining results in sharp drops in perplexity, but How2Score only modestly increases ($+3$ for 7B, $+6.1$ for 32B models), eliminating memorization as the dominant factor.
• Residual Verbosity Bias: Positive association between generation/reference token ratio and no_failure odds, even under explicit length control, supports reporting average token counts with all scores.

Qualitative Diagnostics

Large-scale qualitative analysis reveals canonical failure patterns: critical omissions of prerequisite actions, vague or underspecified operational parameters, unsafe actions, internal contradictions, and refusals. Successful generations exhibit actionable precision, correct sequencing, and minimal critical deviations, even when phrasing diverges from the reference.

Practical and Theoretical Implications

How2Everything establishes a closed-loop paradigm for procedural capability development: web-mined data enables rigorous evaluation and optimization, bypassing the need for expensive human annotation or unreliable overlap-based metrics. The framework provides practical scalability to model selection, ranking, and forecasting downstream performance. Procedural skill is shown to be a core generalizable capability (observable early in training), making it a promising auxiliary RL signal with tangible effects across unrelated domains.

Potential risks include the propagation of societal biases or unsafe procedural content from web-mined data, and the impact of applying LLM-evaluated metrics as proxies for real-world correctness in safety-critical applications. Mitigation strategies include topic-level safety filtering and explicit documentation of evaluation limitations. Release of evaluation splits, prompt templates, and judge model weights supports transparent, reproducible benchmarking.

Conclusion

How2Everything addresses the longstanding challenges of procedural generation evaluation and optimization in LLMs through web-scale corpus mining, validity-oriented LLM-as-a-judge protocols, and systematic RL-based improvement. The framework unlocks scalable analysis and enhancement of goal-conditioned procedure generation, facilitating domain-agnostic benchmarks, robust training signals, and practical advances for user-facing LLM applications (2602.08808).