Quality-Controllable Policy (QCP) Overview

• Quality-controllable policies (QCPs) are frameworks where policy behavior can be adjusted according to explicit quality, risk, performance, or execution-preference variables, such as network QoS, safety risk, return-distribution tails, or robotic execution attributes.
• QCP mechanisms include parameterized optimization, quality-conditioned imitation, and policy interpolation, each offering different ways to control policy outcomes, such as adjusting safety margins, convergence, and task reward through parameter updates, altering execution regimes with quantized trajectory-quality labels, or blending student and teacher distributions for trajectory improvement.
• Adaptation procedures in QCP range from real-time measurement-based feedback in networking to staged calibration in conformal and quantile-based reinforcement learning, emphasizing both empirical performance and theoretical guarantees.

Quality-Controllable Policy (QCP) denotes a policy whose behavior can be deliberately adjusted according to an explicit quality, risk, performance, or execution-preference variable. Depending on the application, the control variable may regulate network QoS, safety risk, return-distribution tails, content coverage and coherence, robot execution attributes, precision–recall asymmetry, teacher intervention, or policy deviation from a trusted reference. QCP is not a single universally standardized formalism. In the research surveyed here, it appears as a family of architectures sharing four elements: a parameterized policy, measurable quality dimensions, a mechanism that maps quality requirements to policy behavior, and an adaptation or calibration procedure linking observed outcomes back to policy control.

1. Conceptual scope and formal structure

A QCP can be represented abstractly as a policy conditioned on state and a quality-control variable:

$\pi_\theta(a\mid s,\mathbf{c}),$

where $\mathbf{c}$ specifies desired quality attributes, operating constraints, risk tolerance, or policy preferences. The policy produces actions or outputs whose quality is evaluated by a signal $Q$, which may be a scalar, vector, distributional statistic, constraint loss, or trajectory-level measurement.

A general QCP therefore contains:

1. Quality specification: a target, preference, constraint, or operating point.
2. Policy control interface: a parameter, label, coefficient, threshold, or vector that changes behavior.
3. Quality measurement: an observable metric such as bitrate, delay, return quantile, recall, trajectory smoothness, or false-negative/false-positive ratio.
4. Policy adaptation or calibration: optimization, feedback control, conformal calibration, relabeling, or inference-time guidance.
5. Deployment semantics: a relationship between the requested quality and the resulting policy behavior.

The studies differ in which of these elements are formalized. CNQF provides an implemented measurement-based QoS controller but does not define a formal QCP model (Yerima et al., 2013). A differentiable CBF-QP controller adapts policy parameters to improve reward while maintaining recursive feasibility, but lacks an explicit user-facing quality command (Parwana et al., 2021). Quantile-based RL controls a selected point of the return distribution, whereas CFGRL controls a policy-improvement coefficient at inference time (Jiang et al., 2022, Frans et al., 29 May 2025). The robot-policy work explicitly defines QCP as a quality-conditioned policy with iterative hindsight relabeling (Kang et al., 27 Oct 2026).

The term “quality” is consequently application-dependent. It may mean:

• Network service quality: bandwidth, utilization, delay, loss, bitrate, service availability, or blocking.
• Control quality: task reward, convergence, safety margin, feasibility horizon, smoothness, speed, or path efficiency.
• Distributional RL quality: a lower quantile of cumulative return.
• Generation quality: coverage, coherence, goal achievement, or teacher–student learnability.
• Moderation quality: precision, recall, Jaccard overlap, and the FN/FP operating point.
• Statistical reliability: expected constraint loss, conformal coverage, or false acceptance risk.
• Verification quality: the degree and form of user-controlled verification information, although the QCP acronym has a different expansion in the C-verification system “Qualified C Programming Verifier” (Wu et al., 19 May 2025).

A strong QCP interpretation requires more than tunable parameters. It requires that the parameters have interpretable effects, that quality be measurable, and that the relationship between control settings and outcomes be sufficiently characterized for deployment. Several papers provide only partial versions of these requirements.

2. Architectural mechanisms for quality control

Closed-loop policy-driven control

The Converged Networks QoS Management Framework (CNQF) separates policy administration, policy decision, measurement, control, and adaptation. Its architecture comprises the Resource Management Subsystem, Measurement and Monitoring Subsystem, and Context Management and Adaptation Subsystem, organized into the Policy Administrative Layer, Policy Decision Layer, and Measurement, Control and Adaptation Layer (Yerima et al., 2016).

High-level policies are interpreted by Resource Brokers, including Wireless Access Resource Brokers, Fixed Access Resource Brokers, and Core Network Resource Brokers. Resource Controllers translate abstract decisions into technology-specific operations at routers, gateways, switches, and wireless access nodes. Network Monitors and the Central Measurement and Monitoring entity collect bandwidth, utilization, delay, loss, flow-state, and context measurements.

The principal control loop is:

$$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$

In the prototype, policies are mapped to DiffServ marking, packet classification, queuing, policing, bandwidth allocation, and admission control. An example policy marks traffic with DSCP $0x2e$ and gives it Expedited Forwarding treatment through Linux traffic-control mechanisms. The architecture therefore supplies practical QCP components, but it does not define formal quality contracts, utility functions, conflict-resolution semantics, or proofs of SLA compliance.

Parameterized optimization-based control

The safety-critical controller in (Parwana et al., 2021) uses a quadratic program whose parameters determine the strength of Control Barrier Function and Control Lyapunov Function constraints. For a safe-set barrier $h_i$, the discrete-time constraint is:

$h_i(t+1,x_{t+1})\geq (1-\alpha_i)h_i(t,x_t).$

The parameter vector is

$$\theta = \begin{bmatrix} \alpha_0 & \alpha_1 & \cdots & \alpha_N \end{bmatrix}^{\!\top}.$$

The lower-level QP generates the action, while an upper-level optimizer adjusts $\theta$ to maximize horizon reward or extend the feasible horizon. Differentiation through the QP uses KKT sensitivity analysis, implicit differentiation, and backpropagation through system dynamics.

The method is QCP-like because explicit parameters alter safety margins, convergence, task reward, and recursive feasibility. Its Recursive-Feasibility-Guided Gradient Descent restricts parameter updates to directions satisfying linearized feasibility conditions. However, the guarantees are local and conditional: linearized feasibility need not imply exact nonlinear feasibility, and QP degeneracy or active-set changes can invalidate ordinary derivatives.

Quality-conditioned imitation and policy interpolation

The robot QCP conditions action generation on quantized trajectory-quality labels. A trajectory metric $m(\tau)$ is transformed so that larger values represent better quality, then partitioned into $\mathbf{c}$0 empirical equal-frequency bins:

$\mathbf{c}$1

At inference, a user supplies a target label $\mathbf{c}$2. The policy can consequently request different execution regimes, such as faster, smoother, more stable, or more path-efficient behavior. Multiple attributes receive separate labels and embeddings (Kang et al., 27 Oct 2026).

Interpolated Policy Distillation uses a different control interface. Its rollout policy is a token-level mixture of student and teacher distributions:

$\mathbf{c}$3

Here $\mathbf{c}$4 yields student/on-policy rollouts, $\mathbf{c}$5 yields teacher/off-policy behavior, and intermediate values trade trajectory quality against student learnability. The mixture is realized exactly using speculative sampling, with the student as proposal model and the interpolated distribution as target (Shi et al., 29 Sep 2026).

CFGRL instead defines a guided policy through a reference policy and a quality-conditioned policy:

$\mathbf{c}$6

The guidance coefficient $\mathbf{c}$7 controls the trade-off between behavior-policy adherence and advantage-oriented improvement. The theoretical product-policy result establishes monotonic policy improvement under boundedness, measurability, nonnegativity, and monotonicity assumptions, but learned diffusion models and approximate advantage signals do not inherit an unconditional monotonicity guarantee (Frans et al., 29 May 2025).

3. Quality objectives and control variables

Distributional return quality

Quantile-Based Policy Optimization replaces expected return with a selected quantile of cumulative reward:

$\mathbf{c}$8

The parameter $\mathbf{c}$9 identifies the region of the return distribution treated as quality. Lower-tail optimization emphasizes conservative performance and reduces the probability of poor outcomes. QPO jointly estimates the quantile and updates policy parameters on separate timescales, with $Q$0 so that quantile estimation is faster than policy adaptation. QPPO adds importance sampling and PPO clipping (Jiang et al., 2022).

This is implicit quality control rather than an explicit constraint. The method optimizes one quantile; it does not directly solve a requirement such as $Q$1, does not control all quantiles simultaneously, and does not guarantee a prescribed out-of-sample reliability level.

Precision–recall quality

Adaptive Tversky Policy Optimization defines multi-label video safety detection as a QCP in which the target FN/FP ratio controls precision–recall behavior. For category $Q$2:

$Q$3

The Adaptive Tversky Reward maintains exponential moving averages of false positives and false negatives, computes the logarithmic error relative to the target, and updates a controller state that determines the Tversky coefficients. Category-aware control uses independent $Q$4, while global control uses one coefficient pair for all categories (Yang et al., 1 Oct 2026).

The global experiments show monotonic changes in precision, recall, and empirical FN/FP ratio across tested targets. Category-aware control can favor precision for misinformation while favoring recall for extreme content. The target ratio is a training-time control target, not an inference-time switch, and does not numerically guarantee the final operating point.

Coverage and coherence

The multi-document summarization system uses a sentence-extraction policy whose score is:

$Q$5

The coefficients $Q$6 and $Q$7 control coverage and local coherence. The selected sentences are passed to FLAN-T5-XL for rewriting, and the extractor is trained with ROUGE and S-BERT similarity rewards (Kurisinkel et al., 2023).

Increasing $Q$8 changes human-rated coherence from 3% at $Q$9 to 70% at $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$0 in the reported evaluation. This demonstrates a controllable attribute, but not a complete coverage–coherence Pareto characterization. The controls are scalar weights rather than target values or formal constraints.

Risk and uncertainty

Conformal Policy Control uses a safe reference policy $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$1 and an optimized policy $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$2. A policy-deviation parameter $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$3 defines:

$$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$4

Small $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$5 produces strong clipping and behavior near the safe policy; large $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$6 approaches the optimized policy. Conformal calibration selects the largest $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$7 whose conservative weighted risk remains below a user-declared tolerance $$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$8. Under bounded losses, replace-one stability, and Lipschitz assumptions, the method provides a finite-sample marginal expected-risk bound of the form:

$$\text{quality requirement} \rightarrow \text{policy decision} \rightarrow \text{resource enforcement} \rightarrow \text{measurement} \rightarrow \text{policy adaptation}.$$9

CPC therefore offers a direct mapping from a semantically meaningful risk target to a calibrated policy-control parameter. Its guarantee is population-level and marginal, not per-context or per-action, and depends on the safety of the reference policy and the validity of calibration (Prinster et al., 2 Mar 2026).

Conformal QoT applies a related separation between prediction, uncertainty set, and operational decision. A QoT regressor predicts received OSNR; conformal calibration constructs a prediction interval; and a policy decides whether a lightpath is feasible. The conformal significance level $0x2e$0 controls nominal interval coverage, while the policy parameter $0x2e$1 controls the operational decision rule (Rezaei et al., 10 Jun 2026).

The distinction between interval coverage and decision reliability is essential. A $0x2e$2–$0x2e$3 conformal coverage result is not itself a guarantee of a specified false-acceptance rate for accepted lightpaths. The reported risk-averse policy accepts when the lower interval bound exceeds the threshold, whereas the efficiency-oriented policy uses the upper bound and a margin.

Execution quality

The robot QCP defines trajectory-level metrics such as speed, action smoothness, path efficiency, completion time, joint path length, jerk, tray tilt, and end-effector speed. The policy is trained with quality-conditioned behavior cloning or flow matching, while iterative hindsight relabeling adds successful online rollouts and recomputes empirical quality labels (Kang et al., 27 Oct 2026).

The method preserves successful trajectories for task learning rather than filtering exclusively for the best demonstrations. This permits quality control without discarding task coverage. It also enables quality-range expansion when newly generated successful trajectories exceed the initial demonstration quality.

4. Adaptation, calibration, and learning procedures

QCP systems use several distinct adaptation paradigms.

Measurement-based feedback: CNQF observes network state and triggers resource reconfiguration. Its runtime experiment uses rising utilization, an $0x2e$4 trigger, DSCP marking, and priority queuing. The reported 1 Mbit/s test flow recovers or maintains its bitrate under increasing background traffic, but the experiment does not establish formal delay, jitter, loss, or QoE guarantees (Yerima et al., 2013).

Differentiable parameter adaptation: the CBF-QP method differentiates through the policy optimization layer and system dynamics. Reward gradients update policy parameters, while feasibility-direction constraints protect the previously achieved feasible horizon. Recursive feasibility is maintained only under the assumptions and approximations used by the method (Parwana et al., 2021).

Two-timescale stochastic optimization: QPO estimates the current policy quantile through a Robbins–Monro recursion and updates neural policy parameters more slowly. Its convergence theorem is for QPO rather than QPPO and requires smooth quantiles, Lipschitz sensitivities, bounded policy scores, compact convex parameters, suitable step sizes, and an additional strict-convexity assumption whose use for global maximization requires scrutiny (Jiang et al., 2022).

Policy-gradient reward adaptation: ATPO updates Tversky coefficients from observed category-level errors and uses GRPO for policy optimization. Exponential moving averages reduce noise, while KL regularization and clipped policy ratios constrain policy updates (Yang et al., 1 Oct 2026).

Offline calibration: CPC holds out calibration data from policy improvement, reweights safe-policy observations under candidate constrained policies, and selects the largest conformally controlled deviation parameter. Its procedure handles non-monotonic losses by controlling an empirical risk envelope over the full path from the safe endpoint (Prinster et al., 2 Mar 2026).

Iterative hindsight relabeling: robot QCP collects rollouts under the highest quality label, retains successful trajectories, measures actual quality, recomputes empirical quantile labels for the entire accumulated dataset, and retrains. This changes the absolute meaning of labels across iterations while preserving the relative ordering of quality regimes (Kang et al., 27 Oct 2026).

Inference-time guidance: CFGRL and IPD expose quality controls without retraining for each operating point. CFGRL varies $0x2e$5 during diffusion or flow sampling; IPD varies $0x2e$6 during rollout generation. The former emphasizes advantage or goal achievement, while the latter controls teacher intervention and student learnability (Frans et al., 29 May 2025, Shi et al., 29 Sep 2026).

5. Empirical evidence and application domains

QCP mechanisms have been evaluated across networking, reinforcement learning, generation, moderation, optical networking, and robotics.

In converged-network experiments, CNQF validates policy-to-device enforcement on Linux routers using DiffServ, SNMP/SNMP4J, Java RMI, and Linux tc. The fuzzy allocation extension uses voice, video, and data classes, and compares adaptive allocation with fixed-policy and class-agnostic admission control. The reported results include improved service availability, lower blocking for video and data, and approximately 4% voice blocking under FRB compared with approximately 15% for the class-agnostic case at $0x2e$7 (Yerima et al., 2013).

In quantile-based RL, QPO and QPPO outperform mean-based REINFORCE and PPO on a zero-mean toy environment, simulated stock trading, and Dow Jones trading under lower-tail quantile objectives. QPPO converges faster in several reported settings, but the evidence is based on learning curves rather than universal finite-sample quality guarantees (Jiang et al., 2022).

In controllable summarization, reinforcement learning improves ROUGE over the non-RL extractor, and the proposed extract–rewrite system outperforms the reported extract–rewrite baselines. On WCEP, the RL system reports ROUGE-1 40.71, ROUGE-2 18.34, and ROUGE-L 31.58. Human evaluation favors the proposed method for coherence, although the evaluation uses four postgraduate linguistics students and 45 summary sets (Kurisinkel et al., 2023).

In diffusion-based policy improvement, CFGRL improves over behavioral cloning, goal-conditioned behavioral cloning, and AWR on many offline RL and goal-conditioned tasks. Its experiments also show degradation at excessive guidance, demonstrating that the quality-control parameter is useful over an operating range rather than universally monotone in practical return (Frans et al., 29 May 2025).

In interpolated policy distillation, IPD outperforms student-only and teacher-only endpoints, conventional SFT-then-OPD, and heuristic segment-interleaving baselines across reported text-only and multimodal reasoning settings. Performance generally peaks at an interior coefficient, with $0x2e$8 effective in the main experiments. This supports the existence of a quality–learnability trade-off rather than a universal preference for maximal teacher intervention (Shi et al., 29 Sep 2026).

In multi-label video safety detection, ATPO-G improves SafeWatch-Bench-Real Jaccard from 40.66 for the Qwen2.5-VL-7B zero-shot baseline to 75.44, while ATPO-C obtains 75.15. The global sweep provides empirical control over precision–recall behavior, and category-level targets separately shift misinformation and extreme-content operating points (Yang et al., 1 Oct 2026).

In conformal QoT estimation, conformal bands achieve approximately 90%–91% reported coverage, while model-uncertainty bands are narrow but poorly calibrated under the described domain mismatch. The reported policy layer improves empirical classification accuracy in the experimental settings, but the paper distinguishes these accuracy results from formal conformal coverage (Rezaei et al., 10 Jun 2026).

In robot manipulation, QCP improves execution quality and success across Robomimic, RoboEval, and real-world tasks. Reported deployments include increased conveyor end-effector speed and throughput, and reduced tray tilt with improved wine-glass transport success. The method also controls multiple attributes jointly, although incompatible or geometrically fixed attributes must be excluded (Kang et al., 27 Oct 2026).

6. Guarantees, limitations, and unresolved issues

QCP mechanisms differ substantially in their guarantees.

Some results are formal but conditional. CFGRL proves policy improvement and monotonicity for ideal product policies constructed from a reference policy and a bounded nondecreasing advantage transformation. CPC provides finite-sample marginal expected-risk bounds under safe-reference, bounded-loss, stability, Lipschitz, and calibration assumptions. Conformal QoT provides marginal prediction-interval coverage under exchangeability. QPO provides an almost-sure asymptotic convergence claim under restrictive stochastic-approximation assumptions.

Other systems provide empirical or architectural evidence rather than formal guarantees. CNQF demonstrates a working measurement-and-enforcement loop but not SLA compliance, stability, fairness, or bounded reaction time. ATPO demonstrates controllable empirical operating points but does not guarantee that a target FN/FP ratio will be realized after training. Robot QCP demonstrates ordered and expanding quality distributions but depends on successful exploration, quality metrics, and repeated retraining. IPD provides exact sampling from its interpolated token policy, but not a guarantee of monotonic downstream task accuracy. QCP summarization demonstrates controllable coherence, but not factuality, global discourse quality, or reference-free deployment quality.

Common limitations include:

• Quality ambiguity: quality may be reward, risk, coverage, coherence, safety, throughput, or calibration; these are not interchangeable.
• Marginal versus conditional validity: conformal and risk guarantees generally apply to population averages, not every context, subgroup, action, or trajectory.
• Calibration dependence: quality-control parameters often require representative data, empirical calibration, or deployment-specific tuning.
• Model and distribution shift: learned policies may leave the support of training data, especially under strong guidance, aggressive optimization, or high-quality labels.
• Non-monotonic outcomes: increasing a control parameter may improve quality over one range and degrade it beyond that range.
• Multi-objective conflict: precision and recall, speed and stability, coverage and coherence, or reward and safety may compete.
• Delayed and noisy feedback: SNMP monitoring, trajectory-level quality measurement, quantile estimation, and category-level error statistics can all introduce latency and variance.
• Formalization gaps: many systems lack explicit policy grammars, conflict-resolution rules, utility functions, rollback behavior, stability analysis, or end-to-end contract composition.
• Operational cost: iterative retraining, teacher inference, active probing, target-domain calibration, or repeated rollout evaluation can be substantial.
• Failure handling: quality requests may be infeasible, yet several systems do not fully specify abstention, degradation, fallback, or safe refusal semantics.

A mature QCP framework would therefore need an explicit quality contract, a calibrated control interface, multi-objective semantics, uncertainty and distribution-shift handling, policy-conflict resolution, feasibility and safety guarantees, monitoring and recalibration, and a clear distinction between empirical operating-point control and formal statistical or dynamical guarantees.

7. Relation to adjacent uses of the QCP acronym

The acronym QCP is not semantically uniform across the literature. In the principal quality-control sense, it denotes Quality-Controllable Policy. However, “QCP” in “NEAT: QCP” means Qualified C Programming Verifier, a separation-logic-based verification tool rather than a policy framework (Wu et al., 19 May 2025).

The verifier provides user-controllable verification mechanisms through partial assertions, multiple specifications, which implies, multi-branch assertions, branch selection, multi-branch loop invariants, Stellis entailment strategies, SMT automation, and interactive Rocq proofs. Its controllability concerns the amount and form of verification information rather than a measurable quality command. The system reports 111 verified functions, 2,795 annotation lines, 1,055 automatically solved verification conditions, 483 manually solved verification conditions, and 44.65 ms average verification time per function. These results concern verification automation and proof effort, not the policy-quality control mechanisms described by the other QCP studies.

Accordingly, QCP should be disambiguated by context. In control, reinforcement learning, generation, networking, moderation, optical networking, and robotics, it refers to policies whose behavior can be adjusted according to explicit quality or risk objectives. In C program verification, it refers to a specific verifier whose user-controlled annotations and proof strategies regulate the verification process.