WebShop: UML-F Web Framework and E-commerce Benchmark
- WebShop denotes two distinct software artifacts: a 2014 e-commerce framework for demonstrating UML and UML-F, and a 2022 simulated e-commerce environment for training language agents.
- The 2014 WebShop framework uses object-oriented patterns and focuses on variable aspects such as payments, promotions, and catalog contents, eliminating features like authentication and encryption.
- The 2022 WebShop benchmark is a realistic e-commerce environment with 1.18 million products, used to evaluate various learning, planning, and error-correction methods for language agents.
WebShop denotes two distinct software artifacts in the research literature. The first is an object-oriented e-commerce framework introduced to demonstrate UML and the UML-F profile for documenting and adapting Web applications; it is explicitly not a full-fledged commercial store platform and omits security features (Fontoura et al., 2014). The second is a simulated e-commerce environment and benchmark for grounded language agents, containing more than 1.18 million real-world products, 12,087 crowd-sourced shopping instructions, and multi-step search, navigation, product-configuration, and purchase tasks (Yao et al., 2022). The benchmark has subsequently been used to study offline reinforcement learning, error correction, hierarchical control, skill memory, uncertainty-aware rewards, credit assignment, clarification seeking, and cross-environment generalization.
1. Terminological distinction and historical development
The 2014 WebShop framework and the 2022 WebShop benchmark share an e-commerce theme but serve different research purposes. The framework is a reusable object-oriented basis for deriving Web-store configurations from product information and store configuration. Its conceptual pipeline is:
1 |
Product information & store configuration → WebShop → Web-site with product catalog and shopping facilities |
It was developed primarily to demonstrate UML and UML-F rather than to provide a production-ready online store. The framework concentrates on variability in payment mechanisms, promotions, administrative reports, and catalog contents. A particular application can supply products, select or implement payment mechanisms, define promotional policies, and configure or extend reporting.
The benchmark, by contrast, is an interactive environment for training and evaluating agents that ground natural-language instructions in Web content and actions. It was designed to address limitations of static language benchmarks and interactive environments that use artificial language, short episodes, low-level mouse actions, or human-in-the-loop evaluation. Its environment combines realistic product language, long-horizon interaction, high-level semantic actions, and automatically computable rewards (Yao et al., 2022).
The benchmark’s subsequent use has broadened the scope of WebShop research. Studies have treated it as a testbed for generalization to unseen instructions (Mediratta et al., 2023), proactive error correction (Zeng et al., 23 Sep 2025), trajectory-search rollouts (Djuhera et al., 12 Feb 2026), uncertainty-aware reward design (Zhang et al., 24 Feb 2026), hierarchical planning (Jin et al., 1 Mar 2026), strategic trajectory abstraction (Xue et al., 7 May 2026), graph-structured skills (Li et al., 12 May 2026), selective hindsight distillation (Li et al., 19 May 2026), training-free skill adaptation (Yu et al., 31 May 2026), cross-environment reranking (Shao, 1 Jun 2026), self-internalizing skills (He et al., 1 Jun 2026), clarification seeking (Matsnev, 17 Jun 2026), progress-oriented credit assignment (Fan et al., 5 Jul 2026), milestone-based policy optimization (Qian et al., 20 Aug 2026), record–state reliability failures (Khezresmaeilzadeh et al., 19 Oct 2026), and evolving user intent (Zhou et al., 24 Oct 2026).
2. The UML-F WebShop framework
The framework’s central domain object is ShoppingCart. A new shopping-cart object is created for each client access to a Web store. It manages product selection and checkout and is associated with multiple Product objects and one TransLog object. The principal operations are:
1 2 3 4 5 6 7 |
ShoppingCart.addProduct() ShoppingCart.removeProduct() ShoppingCart.changeQuantity() ShoppingCart.checkout() TransLog.addLog() Product.getName() Product.getValue() |
The cart updates the transaction log through TransLog.addLog(). The log supports customer-relationship activities, promotions, and reports; it is therefore intended as historical business information rather than merely an audit artifact.
The user-facing navigation begins with a Shopping page displaying available products. Customers can add products to a personal shopping cart and change quantities. At checkout, they select a payment method and provide payment information. The system verifies the information and either processes the transaction or reports an error. An administrative site can separately expose reports through a report list from which an administrator selects a report and views its result.
The framework identifies four principal variation areas:
- Payment options: stores may use credit cards, electronic money, or entirely new electronic payment methods.
- Promotions: business rules may depend on purchase value, shopping frequency, or historical sales volume. Examples include sending a gift when annual sales exceed a threshold or upgrading delivery for a high-value order.
- Administrative reports: organizations may require best-customer rankings, product or product-group sales figures, or preferred-payment statistics.
- Catalog contents: the offered products and their associated names and values are configurable.
Payment and report variation use the Separation construction principle. The Payment interface defines processPayment(), and concrete payment classes implement that operation. ShoppingCart.checkout() acts as a template method, while Payment.processPayment() is a hook method. Concrete payment classes such as CreditCard and EMoney can be dynamically loaded through the «adapt-dyn» mechanism.
The framework passes payment information as one formatted string, for example:
1 |
number = '5534453567144532'; expdate = '10/2002'; name = 'John V. Lee' |
Each concrete payment class defines the attributes it expects, parses the string, and processes the result. XML is mentioned as an alternative representation. The printed source fragment uses Class.forName(paymentClassName) and newInstance() to load and instantiate the selected payment class. The conceptual design is that concrete implementations provide the payment-processing hook, although the printed material contains a naming inconsistency between processPayment(), verifyPayment(...), and paymentInfo versus payInfo.
Promotions use the Chain-of-Responsibility pattern. ShoppingCart invokes the first promotion, and each promotion checks its conditions before forwarding the request to its successor. The example chain contains Over1000, which applies a discount to transactions exceeding US$1,000, followed by BaseDiscount, which provides a 10-percent discount. The design is acknowledged to be incomplete for sophisticated history-dependent promotions that require payment data, user data, and the transaction log.
Reports use a dynamically selected implementation analogous to payments. Report.report() loads an implementation and invokes generateReport(). ListLog lists the complete transaction log, while ByProduct lists transactions associated with a particular product. The report result is an HTML string.
The paper generalizes payment and report dispatch into the domain-specific Web Request pattern. A stable request-facing interface invokes a dynamically loaded concrete handler. UML-F tags identify template methods, hook methods, dynamically adapted classes, Chain-of-Responsibility roles, and Web Request loading and handling operations. The associated UML-F Web site provides Java source files and sample WebShop adaptations.
The framework is deliberately limited. The presented version lacks security features and is not a complete e-commerce product. The supplied description does not document authentication, authorization, encryption, secure payment gateways, injection protection, class-loading protection, session security, inventory management, shipping, tax calculation, order fulfillment, persistence, fault tolerance, scalability, or production-grade transaction guarantees. These omissions are consistent with its demonstrative scope rather than evidence that the framework was intended as a deployable commercial platform (Fontoura et al., 2014).
3. The WebShop benchmark environment
The benchmark contains 1,181,436 products scraped from Amazon across fashion, makeup and beauty, electronics, furniture, and food. Product text has an average length of 262.9 words, and the corpus contains 224,041 words occurring more than ten times. Across products there are 842,849 unique options and 670 mined product attributes, with approximately 3.1 attributes per product and 0.67 options per product on average.
A product is represented through aggregated textual content, price, available buying options, images, and hidden attributes. Attributes are mined from product titles and descriptions using TF-IDF over bigrams, manually filtered for interpretability, and consolidated into a pool of 670 phrases. Examples include “waterproof,” “soft sole,” and “good for dry skin.” The hidden attributes are used for task construction and reward computation but are not exposed to the agent.
The environment has four principal page types:
- Search page, containing a search bar.
- Results page, displaying products returned by a search.
- Item page, presenting a product, description, price, image, and buying options.
- Item-detail page, providing additional description or overview information.
A search returns the top 50 products over five result pages with ten products per page. Retrieval uses a deterministic BM25 index implemented with Pyserini. Indexed text concatenates product titles, descriptions, overviews, and customization options. Literal use of the entire instruction is insufficient: the target appears in the first ten results only about one-third of the time and is absent from the top 50 more than half of the time. Query formulation and reformulation are therefore central components of the task.
WebShop supports two observation modes. HTML mode renders a browser-like interface and is used for human data collection. Simple mode strips unnecessary HTML metadata and presents a compact textual observation for model training and evaluation. The separation enables realistic human interaction while avoiding the requirement that agents learn low-level visual clicking.
Actions are high-level and text-based:
1 2 |
search[query] choose[button] |
Representative transitions include searching, selecting a product, changing result pages, selecting an option, opening a description or overview, returning to a product page, returning to search, and buying. The action argument is itself language: the agent must match instruction phrases such as “red,” “size 9,” or “cordless bottom-up blackout white” to potentially noisy button or option strings.
The benchmark is modeled as a partially observable Markov decision process:
where is the state space, the action space, the deterministic transition function, the reward, the instruction space, and the observation space. An instruction specifies desired attributes, desired option–value pairs, and a price constraint. The target product is not necessarily named explicitly.
The terminal reward combines product-type matching, attribute matching, option matching, and price satisfaction. Success does not require selecting the exact target product: any product satisfying all required criteria can receive the maximum reward. Task Score is reported as $100$ times expected reward, while Success Rate is the percentage of episodes with maximum reward. The automatic reward correlates with manual judgments at approximately 0.856 for average-worker trajectories and 0.773 for expert trajectories, although exact lexical matching can fail to recognize valid synonyms.
Human instructions were collected through Amazon Mechanical Turk. Workers wrote 12,087 instructions with an average length of 15.9 words and a vocabulary of 9,036 words. Human demonstrations include 1,012 training trajectories, 54 additional development demonstrations, and trajectories over all 500 test instructions. Human experts achieve approximately 59.6% success, while the best original learned agent achieves approximately 29% (Yao et al., 2022).
4. Agent learning and generalization
The original benchmark evaluates rule-based heuristics, imitation learning, reinforcement learning, and image-LLMs. The rule baseline searches the exact instruction, selects the first result, and buys without selecting options. Its task score is 45.6 with a 9.6–10% success rate. The main imitation-learning architecture separates search generation from page-level choice: BART generates search queries, while a BERT-based model scores available actions. Optional image features use a pretrained ResNet-50.
The strongest original system is initialized from imitation learning and fine-tuned with online policy-gradient reinforcement learning. BART supplies candidate search queries, and the BERT choice model selects among queries and page actions. The reported imitation-learning score is 59.9 with approximately 29.1% success, while imitation learning plus reinforcement learning reaches a score of 62.4 and approximately 28.7% success. Human experts achieve a score of 82.1 and 59.6% success. The gap is associated with compositional instruction following, query reformulation, noisy webpage language, option selection, navigation, backtracking, and strategic exploration.
Ablation analyses identify pretrained language representations as important: replacing pretrained BERT with a randomly initialized Transformer reduces success by nearly two-thirds. Replacing learned query generation with the exact instruction reduces performance by roughly three points. Naively adding one previous observation and the last five actions reduces score from approximately 59.9 to 57.3. Direct reinforcement learning from a pretrained BERT model performs worse than the heuristic baseline, suggesting that imitation warm-starting is important under sparse terminal rewards.
Reinforcement learning can also alter behavior in undesirable ways. In one analysis, RL shortens trajectories from approximately 9.4 states for imitation learning to 4.5–4.8 states after RL fine-tuning. The agent visits fewer products and performs fewer searches. Attribute, type, and price scores improve, but option score falls from 45.2 to 38.9, indicating a shift toward quickly exploiting partial matches rather than exploring for the correct configuration.
The benchmark has also been used to study offline reinforcement learning. In one study, WebShop generalization means zero-shot performance on unseen natural-language instructions while the website dynamics remain fixed and the desired product, relevant attributes, initial state, and reward function change. The experiments focus on the choice task, using a pretrained BART search model to generate the search query. On a human-demonstration dataset of 452 replayable trajectories, behavioral cloning outperforms BCQ and CQL on test instructions. Scaling lower-quality imitation-policy data shows substantial improvement up to approximately 500 episodes, followed by diminishing returns. The study concludes that diversity of instructions is more important than simply increasing the number of redundant transitions, although the WebShop data-size experiment confounds dataset size and instruction diversity (Mediratta et al., 2023).
WebShop also exhibits limited but measurable sim-to-real transfer. Agents trained in the simulator were evaluated without fine-tuning on Amazon and eBay. Real pages were scraped, converted into WebShop-style observations, and paired with high-level actions. On Amazon, imitation learning plus reinforcement learning reaches a score of 65.9 and 25% success, compared with 45.8 and 19% for the rule baseline. On eBay, the corresponding figures are 62.3 and 21%, compared with 31.7 and 7% for the rule baseline. Human performance remains higher, but the result demonstrates transfer across products, HTML, search engines, and website dynamics (Yao et al., 2022).
Later work has shown that within-environment reinforcement fine-tuning can raise WebShop performance substantially, while cross-environment transfer depends on semantic priors and action interfaces. SearchQA-to-WebShop transfer is particularly useful because both tasks require query formulation and information extraction. For Qwen2.5-7B, SearchQA training raises WebShop performance from 28.59 to 47.07 in one evaluation. Conversely, BabyAI-to-WebShop transfer falls from 28.59 to 10.25, which is associated with BabyAI’s explicit valid-action lists and a mismatch with WebShop’s open search-and-click interface (Xi et al., 12 Mar 2026).
5. Methods for improving WebShop agents
Research after the original benchmark has introduced several distinct mechanisms for improving search, planning, credit assignment, and reliability.
Error correction and reflection. REBACT inserts a reflection step before each action. The model examines previous action–observation pairs, decides whether an earlier action was wrong, optionally proposes a corrected action, and then produces the next action in one LLM call. On 100 WebShop instructions using Claude 3.5 Sonnet, REBACT achieves 61% success, compared with 37% for ADAPT and 29% for ReAct. Its reported WebShop score is 76.18. The comparison is not a controlled head-to-head evaluation with the 500-instruction human benchmark, but it indicates that operationally revising earlier actions can reduce error accumulation (Zeng et al., 23 Sep 2025).
Trajectory-search rollouts. TSR moves search from inference-time action selection into training-time rollout generation. It samples candidate actions at each turn and ranks partial trajectories using milestone progress and instruction–page similarity. Best-of-, shallow lookahead, and beam search are paired with GRPO. With Qwen2.5-3B, beam-based TSR raises WebShop success from 70.3% with instance filtering to 85.3%, reduces average response length from 519 to 453 tokens, and reduces average turns from 6.8 to 5.8. The additional training cost is substantial: reported WebShop training time rises from 6.24 hours for instance filtering to 14.94 hours for beam search (Djuhera et al., 12 Feb 2026).
Uncertainty-aware rewards. SELAUR combines entropy, least-confidence, and margin-based token uncertainty and reshapes rewards on failed trajectories. With Qwen2.5-1.5B, SELAUR increases WebShop success from 0.6757 for the strongest listed baseline to 0.7656. With Qwen2.5-7B, the improvement is smaller, from 0.7929 to 0.7968. The complete combination of entropy, least confidence, and margin performs best in the reported ablation, although the paper does not report confidence intervals or complete reproducibility details (Zhang et al., 24 Feb 2026).
Hierarchical planning and strategy abstraction. HiMAC separates a macro-policy that generates natural-language sub-goals from a micro-policy that executes atomic actions. Sub-goal completion is indicated by <sub_done>, and macro and micro updates alternate to reduce non-stationarity. On WebShop, HiMAC reaches 83.4% success with Qwen2.5-1.5B and 84.1% with Qwen2.5-7B, compared with 67.4% and 75.2% for the corresponding GiGPO baselines. Removing hierarchy reduces 7B WebShop success from 84.1% to 66.1%; removing iterative co-evolution reduces it to 74.8%; removing adaptive termination reduces it to 79.8% (Jin et al., 1 Mar 2026).
StraTA uses a single trajectory-level strategy generated at the beginning of an episode and conditions subsequent actions on it. It additionally uses diverse strategy rollout and critical self-judgment. On WebShop, StraTA reaches 82.5% success with Qwen2.5-1.5B and 84.2% with Qwen2.5-7B, compared with 65.0% and 72.8% for GiGPO. Its ablation indicates that critical self-judgment gives a larger isolated WebShop improvement than diverse strategy rollout, while their combination is stronger than either alone (Xue et al., 7 May 2026).
Skill memory and graph structure. SKILLGRAPH represents skills as nodes with prerequisite, enhancement, and co-occurrence edges. It retrieves ordered skill subgraphs rather than isolated semantically similar entries and evolves the graph through skill insertion, merging, splitting, deprecation, and edge reinforcement. On WebShop, SKILLGRAPH reaches a task score of 91.5 and 84.4% success, compared with 66.1% for vanilla GRPO and 72.7% for SkillRL. Removing graph evolution reduces success to 70.3%, while removing graph structure reduces it to 72.7% (Li et al., 12 May 2026).
SkillAdaptor performs training-free external skill maintenance with a frozen LLM. It localizes the first actionable fault in a failed trajectory, attributes responsibility to retrieved skills, revises or generates a skill, and accepts the modification only if re-execution does not reduce performance. On 1,624 training instances and 200 test instructions, SkillAdaptor improves WebShop success over the no-skill baseline for Kimi-K2.5, GLM-5, and GPT-5.2. Its strongest matched-baseline improvement is 1.7 percentage points in success over EvoSkill for Kimi-K2.5. Removing the localization and linking stages reduces Kimi WebShop success from 33.0% to 28.6%, while removing the qualification gate reduces it to 26.3% (Yu et al., 31 May 2026).
SIRI uses skills only during training. It first warms up with GiGPO, mines skills from successful skill-free trajectories, validates them through paired skill-augmented and skill-free rollouts, and distills only beneficial executable action tokens into the plain policy. The skill bank is discarded at inference. On WebShop with Qwen2.5-7B, SIRI raises success from 72.8% for GiGPO to 81.3%; without Phase 0 warm-up, success falls to 71.1%. The method therefore separates skill discovery from skill internalization and avoids runtime retrieval overhead (He et al., 1 Jun 2026).
Credit assignment. ProGPO preserves exact-prefix action comparisons while adding rollout-derived state potentials, semantic expansion, inverse-variance fusion, and transition credit. On WebShop with Qwen2.5-1.5B, ProGPO reaches 71.5% success, compared with 67.6% for GiGPO and 65.6% for HGPO. It uses no additional rollouts, critic network, reward model, or auxiliary model (Fan et al., 5 Jul 2026).
MileGPO extends graph-based policy optimization with milestone discovery, failure-trap detection, reliability-calibrated shaping, and progress-contrastive calibration. WebShop has substantial recurring-state coverage: approximately 73.9% of transitions originate from states shared across sampled rollouts, while 72.7% of same-state action pairs are tied under final-goal-distance credit. MileGPO resolves some of these ambiguities through local progress and same-state branch evidence. It achieves a score of 90.29 and 78.58% success, compared with 88.24 and 74.80% for reproduced GraphGPO. Removing PCC reduces success by 1.43 points; removing both PCC and RCS produces an approximately 4.8-point loss (Qian et al., 20 Aug 2026).
Selective hindsight and action-level feedback. SERL uses environmental feedback to reweight reward-directed updates while preventing privileged hindsight from independently determining behavior. It gives the teacher access to immediate feedback, next observations, future trajectories, or successful trajectories, but keeps such information hidden from the student. On WebShop, SERL reaches a score of 89.5 and 80.1% success, compared with 73.1 and 64.1% for GRPO. Immediate feedback is highly competitive, while a successful trajectory becomes more useful when grounded by immediate feedback (Li et al., 19 May 2026).
6. Reliability, clarification, evolving intent, and unresolved issues
WebShop research has increasingly examined failures that are not captured by final success alone.
A clarification-augmented version removes attributes and option clauses from 50% of instructions while preserving product type and price feasibility. Agents may emit request_clarification, after which the original fully specified goal is revealed and the episode continues. A prompt-based uncertainty decomposition separates action confidence from request uncertainty. The proposed method achieves the highest clarification F1 on every tested backbone, with an approximate mean F1 of 0.455 compared with 0.177 for ReAct with uncertainty estimation and 0.212 for uncertainty-aware memory. However, standard WebShop success is not improved uniformly, and the additional uncertainty fields can cause capability dilution (Matsnev, 17 Jun 2026).
A separate reliability study examines record–state mismatch: the action record claims that an option was selected, while the WebShop observation still shows it as unselected. Across four decision models, agents purchase in 53–100% of such cases. Linear probes can distinguish successful and failed option-selection states with AUROC 1.00 on the reported WebShop diagnostic set, showing a decodability gap between internal representations and final decisions. EviGate reads an intermediate activation and vetoes buy now when the preceding action probably failed. In a live WebShop setting with injected silent option-click failures, target-calibrated EviGate raises task success from 8.5% to 13.5%, a five-percentage-point improvement. Calibration on offline counterfactual states transfers poorly and reduces success to 2.5%, demonstrating that deployment-specific calibration is important (Khezresmaeilzadeh et al., 19 Oct 2026).
IntentDelta extends WebShop to evolving user requirements. Its WebShop portion contains 334 trajectories and 1,820 turns. Each turn includes a user utterance, a cumulative structured intention state, and a human reference action. Constraints are classified as must-have, preferred, or optional, and updates include addition, relaxation, override, and reprioritization. Reprioritization accounts for 46.6% of WebShop change events, while 66.5% of change turns involve compound changes. Across eleven models, average WebShop intention detection is 83.1%, but full intention recovery is only 44.7%; hard success is 50.2% and Action Score is 41.0%. The results show that recognizing a constraint is substantially easier than preserving its content and priority across multiple turns (Zhou et al., 24 Oct 2026).
The benchmark’s limitations remain consequential. WebShop is English- and US-centric in its original construction and is based on a limited set of product categories. Its attributes are mined and manually filtered, its reward relies heavily on lexical matching, and its images are not central to the original task. Product reviews, seller reliability, shipping, stock, and temporal price changes are not modeled. Many later studies use different WebShop wrappers, catalog versions, action grammars, episode limits, evaluation subsets, or task scores, so numerical comparisons across papers are not always controlled.
The benchmark also separates several notions that are often conflated:
- Task score versus success: partial product matching can yield a high score without satisfying every requirement.
- Action selection versus complete-task completion: a reranker can select expert actions in candidate sets without solving candidate generation or full episodes.
- Clarification detection versus useful clarification: emitting
request_clarificationcan be scored correctly without evaluating the linguistic quality or informativeness of the question. - Fault detection versus recovery: detecting a failed click does not guarantee that the agent will select the missing option.
- Skill retrieval versus skill internalization: a skill may improve behavior while present but fail to transfer into the policy without retrieval.
- In-domain success versus generalization: performance on unseen instructions within the same site does not establish transfer to new websites, catalogs, or action interfaces.
Taken together, WebShop functions as both a software-framework example and a benchmark for grounded, long-horizon agent research. The 2014 framework demonstrates explicit variability modeling through UML-F, object-oriented patterns, dynamic adaptation, and configurable e-commerce mechanisms. The later benchmark exposes the difficulties of natural-language grounding, retrieval, option selection, long-horizon planning, sparse-reward credit assignment, reliability monitoring, user-intent maintenance, and cross-environment transfer. Its research significance lies less in reproducing a complete commercial store than in providing a controlled environment in which these interacting capabilities can be isolated, measured, and improved.