- The paper introduces AIGP, a novel LLM-based framework that integrates SFT and DPO to align dynamic pricing with long-term business objectives.
- The approach leverages chain-of-thought reasoning, knowledge distillation, and an LTVE model to deliver robust and interpretable pricing decisions.
- Empirical results show significant gains in GMV, ROI, and stability, outperforming traditional RL and heuristic-based models.
Authoritative Summary of AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing
Motivation and Problem Setting
Dynamic pricing in high-volume e-commerce platforms presents intricate challenges, especially in achieving long-term business objectives such as cumulative Gross Merchandise Value (GMV), Return on Investment (ROI), and milestone achievement while maintaining interpretability and leveraging both structured and unstructured information. Conventional methods, including rule-based heuristics and demand/elasticity-driven mathematical models, primarily optimize for short-term sales, fail to incorporate rich textual context (e.g., product reviews), and lack actionable transparency. Recent reinforcement learning (RL) approaches allow direct optimization of long-term objectives but struggle with reward sparsity, poor interpretability, and distribution shift—particularly problematic for cold-start and out-of-distribution SKUs.
LLMs have demonstrated superior reasoning and capacity for integrating heterogeneous data streams. However, generic LLMs lack domain alignment and long-term preference supervision, producing suboptimal actions unsuitable for production deployment in e-commerce pricing.
AIGP: Framework Composition
AIGP (Artificial Intelligence Generated Pricing) is introduced as a comprehensive solution that combines LLM-based reasoning, supervised fine-tuning (SFT) with knowledge distillation, and direct preference optimization (DPO) underpinned by a Long-Term Value Estimator (LTVE). Its architecture integrates chain-of-thought (CoT) prompting, domain-adaptive instruction tuning, and offline RL for reward modeling.
- Policy Parameterization: The pricing policy is an LLM (Qwen3-30B-A3B), fed structured signals and business context via carefully designed prompts. Unlike standard RL agents, the LLM outputs both a compliant action and interpretive reasoning trace.
- Deployment Pipeline: SFT is performed via teacher-student distillation, where a larger teacher model (Qwen3-235B-A22B) generates high-quality output. Responses are filtered by an LLM-as-Judge module for accuracy, coherence, and compliance, ensuring reliable training signals.
Long-Term Value Modeling (LTVE)
AIGP formulates dynamic pricing as an MDP with reward structure balancing milestone achievement and sustained ROI. The LTVE is a double-critic model trained via offline RL on millions of production transitions:
- State and Action Representation: Contextual features encompass product statistics, temporal attributes, historical trajectory, and incremental discount adjustments normalized across categories.
- Reward Design: Relative category-normalized rewards isolate the impact of pricing strategies from intrinsic product popularity and demand characteristics.
- Critic-Only TD Training: LTVE employs expectile regression for robust value estimation and bootstraps TD targets from the state-value function, mitigating distributional shift and extrapolation error.
LTVE ensures the policy is aligned not just for immediate gain but for robust cumulative performance across business-relevant horizons.
Preference Alignment via DPO
Supervised fine-tuning alone is insufficient for aligning actions to long-horizon rewards; thus, AIGP integrates DPO. Candidate pricing actions are sampled in decision-only ([ACT_ONLY]) mode, scored via LTVE, and paired for preference learning. This concentrates optimization signal on the action tokens, preventing dilution by reasoning traces and ensuring robust learning of decision quality.
Evaluation and Empirical Results
Comprehensive evaluation encompasses both offline (decision quality, expert alignment, LTVE-independent metrics) and online (GMV, ROI, MAR) metrics:
- Offline Alignment: Full AIGP (SFT+DPO) achieves highest Q-Score (2.836), reduction in MAE (0.062), and superior Expert Action Matching Accuracy (EAMA 82.51%). Both EAMA-R and KIPS, which are independent of the value estimator, confirm authentic gains in operational fidelity and expert trajectory overlap.
- Reasoning Quality: SFT and DPO considerably close the performance gap between the compact 30B student and the much larger 235B teacher, as measured by LLM-as-Judge across data accuracy, completeness, coherence, and consistency.
- Online A/B Tests: AIGP delivers significant sustained improvements: +13.21% GMV, +7.59% ROI, +8.20% MAR over 14 days, outperforming both RL-based and neural elasticity baselines.
- Pricing Stability: AIGP generates smaller-magnitude daily adjustments, reducing volatility and abrupt price swings—critical for merchant planning and operational trust.
Core Component Ablations
Ablation studies demonstrate critical contributions of each module:
- DPO boosts long-term value alignment beyond SFT alone.
- Removal of [ACT_ONLY] during DPO training leads to diluted optimization and lower Q-Score/EAMA.
- Unstructured inputs (diagnosis report, knowledge base) are essential for cold-start robustness and effective utilization of qualitative product signals; removal of domain knowledge disproportionately degrades performance.
- Alignment with strong open-source LLMs reveals that scaling alone does not substitute for domain-specific preference alignment.
Robustness and Cold-Start Adaptation
AIGP's combined LLM+LTVE pipeline exhibits superior robustness in:
- Cold-start scenarios (≤3 days on-shelf), with AIGP achieving 94.6% Counterfactual Discrimination Accuracy.
- Boundary actions (rare, large adjustments), where AIGP compensates for OOD suppression common in RL.
- Interpretability and auditability are preserved via explicit CoT traces in production outputs.
Practical and Theoretical Implications
AIGP demonstrates that integrating domain-adaptive LLMs with offline RL-based value estimation and DPO leads to interpretable, scalable, and business-aligned pricing policies. The synergy enables compact models to internalize operational heuristics, outperforming generic large models and traditional RL. Practical deployment on massive SKU volumes is feasible with rigorous operational constraints, efficient processing, and real-time evaluative monitoring.
Theoretically, this hybrid architecture points toward future decision-making frameworks in e-commerce and beyond: combining symbolic reasoning, preference alignment, and offline temporal modeling. The explicit utilization of unstructured information and domain-specific knowledge transfer overcomes cold-start and OOD fragilities pervasive in standard RL and neural elasticity models.
Conclusion
AIGP synthesizes LLM-based reasoning, supervised fine-tuning, and RL-based value preference alignment for dynamic pricing, achieving substantively improved long-term business outcomes and operational stability with transparent and auditable outputs. Its principled preference alignment and domain knowledge condensation enable compact models to match or exceed performance of larger LLMs. The demonstrated robustness and interpretability render AIGP suitable for large-scale e-commerce deployment. Future directions include extending this framework to broader decision tasks in e-commerce and developing unified alignment strategies bridging symbolic and strategic AI reasoning.
