---
title: 'WorthBuying Dataset: Purchase Reason Benchmark'
url: https://www.emergentmind.com/topics/worthbuying-dataset
type: topic
---

# WorthBuying Dataset: Purchase Reason Benchmark

The WorthBuying Dataset—described in "Unlocking the ‘Why’ of Buying: Introducing a New Dataset and Benchmark for Purchase Reason and Post-Purchase Experience" [2402.13417]—is a large-scale resource focused on purchase reason prediction, offering systematic and explicit differentiation between pre-purchase motivation and post-purchase sentiment in user product reviews. It enables AI models, particularly LLMs, to benchmark explanation generation tasks for personalized recommendation, marketing analytics, and user modeling.

## 1. Dataset Definition and Structure

The dataset, termed the *purchase reason explanation dataset*, is drawn from the Amazon product review 5-core dataset (75 million reviews; users/items with minimum five reviews). The current version comprises a random sample of 10,000 reviews, spanning major product categories: Electronics, Books, Fashion (Clothing, Shoes & Jewelry), Home & Kitchen, and Sports & Outdoors.

Each record contains:
- **Product Information:** Title and description
- **User Review:** Original text
- **Explanations** (LLM-generated JSON fields)
  - *Explicit Purchase Reason*: Reason(s) directly stated in the review
  - *Implicit Purchase Reason*: Inferred motivations deducible from context
  - *Purchase Reason Explanation*: Rationale for identification of the above
  - *Post-Purchase Experience*: User sentiment or factual experience after purchase

All explanations are concise, short-text, and categorized as personal or generic.

| Field                    | Example Value                                                         | Type                     |
|--------------------------|----------------------------------------------------------------------|--------------------------|
| explicit_purchase_reason | "To help aging parents who need a clock ..."                         | Short Text               |
| implicit_purchase_reason | "Multiple alarm options (everyday, weekday only, weekend only)"      | Short Text               |
| purchase_reason_explanation| "Emphasizes readability and usability for elderly parents..."      | Short Text               |
| post_purchase_experience | "Parents love the clock, met expectations ..."                       | Short Text               |

## 2. Data Generation and Annotation Methodology

### Source and Sampling
- **Source:** Amazon 5-core reviews
- **Sample size:** 10,000 reviews (limited for LLM computational cost)

### LLM-based Generation
- **Model:** Gemini Ultra (Google)
- **Prompting:** 
  - Simultaneous request for purchase reasons and experiences, with clear distinction
  - Explicit prompts for both direct and inferred (implicit) motivations
  - Instructions to avoid hallucination and provide supporting evidence
  - Output enforced in JSON for machine readability

### Evaluation Protocol
- **Automated LLM Rater**: Explanations rated by a secondary LLM along four axes:
  1. Hallucination
  2. Correctness (categorization)
  3. Completeness
  4. Personalization (specificity)

- **Human Annotation:** 100 sample double-annotations compared against LLM, with high agreement on hallucination/correctness; LLM stricter on completeness but more generous on personalization.

**Coverage/Quality:** 
- Purchase reasons found in 96.4% of reviews (explicit: 61.8%, implicit: 70.9%)
- Post-purchase experiences found in 88.2%
- Personalization rates: 70.1% (reasons), 71.18% (experiences)
- Hallucination <0.5%, Correctness >99%, Completeness ~77%

## 3. Benchmark Tasks and Evaluation Metrics

The WorthBuying Dataset supports multiple benchmarking tasks for personalized natural language explanation generation in recommender systems:

### Task Definitions
1. **Task 1:** Generate explanation for recommending a specific item (user and item info provided)
2. **Task 2:** Generate explanation including user's rating
3. **Task 3:** Joint prediction of user rating and recommendation explanation

### Representations
- **User:** Past reviews (*UserReview*) or LLM-summarized profiles (*UserProfile*)
- **Item:** Metadata, past reviews (*ItemReview*), or summary (*ItemProfile*)
- Review/history truncation: max 10 most recent, max 8k tokens

### Evaluation
Standard NLG metrics:
- **BLEU** (formula: $ \text{BLEU} = BP \cdot \exp \left( \sum_{n=1}^N w_n \log p_n \right) $)
- **ROUGE-1, ROUGE-2, ROUGE-Lsum**

## 4. Experimental Results and Data Characteristics

### Overall Performance
- Gemini Ultra (zero-shot) benchmarks:
  - **Best representations:** User’s raw past reviews + item metadata
    - *Purchase Reason*: BLEU 6.46, ROUGE-1 22.14, ROUGE-2 8.45, ROUGE-Lsum 20.38
    - *Post-Purchase Experience*: BLEU 3.66, ROUGE-1 21.35, ROUGE-2 5.34, ROUGE-Lsum 16.21
  - Summarizing user reviews attenuates personal detail
  - Adding item reviews can introduce noise
  - Variations in benchmark task (e.g., rating presence, joint tasks) produce minimal change in zero-shot LLM scores

### Category-wise Analysis
- **Electronics, Home & Kitchen**: Highest metric scores
- **Books, Fashion**: Lower scores, attributably to more personal language/use cases and less explicit reasoning
- Performance highest when purchase reasoning is strongly reflected in product metadata

## 5. Downstream Applications and Utility

Demonstrated applications include:
- **Recommendation Explanation Generation:** Personalized and context-aware justifications for recommendations, both pre- and post-purchase
- **Justification of Recommendations:** Enables systems to articulate *why* an item is recommended, preceding the purchase event
- **Marketing Analysis:** Facilitates granular analysis of purchase intent versus satisfaction, supporting sophisticated segmentation and targeting
- **Recommender Systems:** Potential for more credible, persuasive recommendation outputs by grounding explanations in both explicit and implicit user motives
- **User Modeling:** Enables richer, context-dependent profile construction by capturing motivational and experiential information

## 6. Advances, Limitations, and Research Impact

### Advances over Prior Work
- Previous datasets relied on post-purchase or generic sentiment extraction, missing context and personal motivation
- The WorthBuying Dataset is the first large-scale resource to annotate pre-purchase reasons and post-purchase experiences distinctly, including rationale fields
- Enables direct benchmarking of pre-purchase explanation tasks, facilitating research that bridges the gap between generic recommendation rationales and personalized decision drivers

### Potential Impact
- **Purchase Reason Prediction**: Supports development and assessment of models capable of generating fine-grained, user-specific rationales
- **Explainable AI**: Introduces rigorous criteria for explanation relevance and completeness beyond generic NLG fluency
- **Marketing Science**: Provides infrastructure to model and understand factors influencing purchase intent, improving campaign efficacy
- **Personalization**: Advances explainable recommendation research towards contextual, individualized justifications

## 7. Summary

The WorthBuying Dataset constitutes the inaugural benchmark for systematic, large-scale annotation and modeling of both explicit/implicit purchase reasons and post-purchase experiences. Leveraging high-fidelity LLM generation and evaluation, the resource enables robust explanation-based benchmarking, recommendation justification, and comprehensive marketing/user modeling research. It transitions the explainable recommendation field from post-hoc satisfaction explanation to pre-purchase motivational articulation, marking a significant methodological advance for recommendation science and marketing analytics.

Source: https://www.emergentmind.com/topics/worthbuying-dataset