CPC-big: Scalable CPC-Constrained Optimization

• CPC-big is a family of scalable methods for optimizing cost-per-click in real-time online advertising through bid optimization, budget allocation, and auction dynamics.
• It employs dual formulations (LP/KKT, primal-dual, stochastic control) and dynamic feedback systems like PID loops to maintain CPC targets under high throughput.
• The framework integrates hierarchical reinforcement learning and multi-agent models for cross-channel bidding, achieving over 90% CPC satisfaction and up to 13% click uplift.

CPC-big refers to a family of scalable algorithmic and system approaches for cost-per-click (CPC) constrained optimization at industrial scale, predominantly in real-time online advertising ecosystems. The CPC-big paradigm encompasses methods for bid optimization, budget allocation, and auction dynamics that systematically enforce expected or empirical CPC targets, even in highly dynamic and high-throughput environments such as Taobao and Meituan. These systems leverage dual mathematical formulations (LP/KKT, primal-dual control, stochastic control, hierarchical reinforcement learning), robust feedback loops, and cross-channel coordination to guarantee CPC constraints at a scale of tens of millions to billions of requests per day.

1. Formalizing the CPC-Constrained Optimization Problem

CPC-big mechanisms start from the empirical requirement that advertisers maximize value (clicks, conversions, or other performance metrics) while satisfying explicit CPC constraints. For a set of $N$ ad opportunities, the canonical linear program is: $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$ where $CTR$, $CVR$, $wp$ (winning price), and $C$ (target CPC) are defined per request, $B$ is the budget, and $x_i$ is the selection variable. This admits a KKT-based primal-dual solution, yielding a bid function: $$bid_i = \frac{1}{p+q}\,CTR_i\,CVR_i + \frac{q}{p+q}\,CTR_i\,C$$ with dual prices $p,q$ controlling spend and CPC, respectively (Yang et al., 2019). Bid shading and multipliers are optimal in both one-shot and repeated auctions under cost-per-action constraints (Heymann, 2018).

2. Dynamic Control Systems and Feedback for CPC Tracking

CPC-big implementations must maintain tight CPC control under time-varying auction volumes, click rates, and spend patterns. To this end, PID (proportional-integral-derivative) control loops are deployed, one each for spend ($$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$0) and CPC ($$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$1): $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$2

$$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$3

where $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$4 is the error between target and realized CPC. Cross-effects are compensated by a lightweight model-predictive (MP) correction (Yang et al., 2019). This ensures that even as traffic composition fluctuates, empirical CPC remains within $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$5 of the target for $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$6 of campaigns at scale.

3. Hierarchical and Multi-Agent RL for Cross-Channel CPC Control

In multi-channel and cross-channel advertising, CPC-big is extended to hierarchical structures. Both HiBid (Wang et al., 2023) and HMMCB (He et al., 2024) model two levels:

• High-level: Allocates budget slices across channels under joint budget and CPC constraints, typically via deep RL methods (MCQ, diffusion policy) and auxiliary losses to avoid channel crowding.
• Low-level: Executes per-channel, per-request bidding (e.g., ratio scaling vs. CPC target), employing either actor-critic RL with value decoupling or efficient data augmentation across constraint multipliers.

Crucially, both methods implement explicit CPC-guided action selection: every candidate bid is filtered by evaluating the predicted end-of-day

$$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$7

and only actions keeping $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$8 are allowed (Wang et al., 2023). This mechanism guarantees hard satisfaction of cross-channel CPC constraints in both offline simulators and online production.

4. Online Platform-Scale Deployment and System Integration

CPC-big approaches are integrated into high-throughput ad serving architectures, with deployments documented on platforms such as Taobao and Meituan. Standard system modularization is as follows:

• Front-ends collect and route requests.
• Strategy/bidding layers apply CPC-big logic (dual control or RL-based budgeting/bidding).
• Selection/search modules run greedy reranking or RL-based action selection.
• Data nodes fetch creatives; responses are delivered within strict (<50 ms) real-time constraints.

CPC-big-style systems (e.g., OCPC (Zhu et al., 2017), HiBid, HMMCB) report the following platform properties:

• Day-level planner retrains nightly, low-level executor can retrain hourly.
• CPC target satisfaction ratio (CSR) exceeds $$\begin{aligned} & \max_{x_1,\dots,x_N} \sum_{i=1}^N x_i\,CTR_i\,CVR_i \ & \text{s.t.} \quad \sum_{i=1}^N x_i\,wp_i \leq B, \ & \qquad \;\;\; \frac{\sum_{i=1}^N x_i\,wp_i}{\sum_{i=1}^N x_i\,CTR_i} \leq C, \ & \qquad \;\;\; 0 \le x_i \le 1 \end{aligned}$$9 for large advertiser populations.
• Value/revenue delivered remains within $CTR$0-$CTR$1 of the realized unconstrained optimum, as measured in replay and live A/B (Yang et al., 2019, Wang et al., 2023, He et al., 2024).
• 99.9th percentile latency remains well below the production standard (e.g., $CTR$234 ms for 19k QPS) (Wang et al., 2023).

5. Algorithmic and Empirical Properties

Theoretical analysis and large-scale experiments demonstrate that CPC-big methods:

• Are internally stable over large ranges of $CTR$3, with anti-windup and monotonicity guarantees on dual variables.
• Outperform single-loop, greedy, and cost-min baselines in both CPC satisfaction and delivered clicks, with typical click uplift $CTR$4-$CTR$5 and CPC reduction $CTR$6 to $CTR$7 on Meituan-scale data (Wang et al., 2023, He et al., 2024).
• Avoid unhealthy channel crowding and maintain revenue stability by auxiliary batch constraints or explicit "capacity guards."
• Scale to tens of thousands of advertisers, billions of daily requests, and multi-agent environments without the need for continuous custom retraining per new allocation (via $CTR$8-generalization and centralized training/decentralized execution).

6. Extensions, Limitations, and Future Directions

CPC-big is a generalizable optimization framework: any per-request business metric $CTR$9 can be incorporated into the composite objective, provided accurate predictions are available (Zhu et al., 2017). Limitations include:

• Dependence on accurate CTR/CVR prediction; miscalibration can bias bid adaptation.
• Feedback delay can induce minor overshoot; practical systems cap error at $CVR$0 by design.
• Fully cooperative equilibrium may not capture all real-world auction externalities (e.g., competing platforms, starvation in low-traffic channels).
• Sufficiently dynamic environments may require more expressive controllers (e.g., RL with explicit budgets as state), an area addressed by recent multi-agent methods (Wang et al., 2023, He et al., 2024).

Continuous progress in RL architectures, contextual control, and real-time market data integration is extending CPC-big methods to cover more sophisticated constraints (e.g., ROI, CPM, retention, fairness) and to operate under nonstationary and adversarial conditions.

---

Key Publications:

• "Optimized Cost per Click in Taobao Display Advertising" (Zhu et al., 2017)
• "Bid Optimization by Multivariable Control in Display Advertising" (Yang et al., 2019)
• "Cost Per Action Constrained Auctions" (Heymann, 2018)
• "HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning" (Wang et al., 2023)
• "Hierarchical Multi-agent Meta-Reinforcement Learning for Cross-channel Bidding" (He et al., 2024)