Resource Fairness Counter in LLM Serving

• Resource Fairness Counter is defined as an operator fairness metric that uses predicted throughput and normalized GPU utilization to drive efficient LLM request scheduling.
• RFC is updated per request batch via ωf · TPS · Util_GPU, rewarding efficient, batched scheduling without applying decode weighting.
• RFC integrates with the User-Fairness Counter in Equinox to balance operator efficiency and user QoS, achieving higher throughput and lower latency.

Searching arXiv for the main paper and closely related fairness work to ground the article. Resource Fairness Counter (RFC) is an operator-side fairness metric in LLM serving that quantifies how scheduling decisions affect server efficiency, specifically through throughput and GPU utilization, rather than user-perceived quality of service. In Equinox, RFC forms one half of a dual-counter framework together with the User-Fairness Counter (UFC): UFC tracks user-side service using weighted tokens and latency, while RFC tracks equitable operator allocation using GPU utilization and throughput without decode weighting, because throughput bottlenecks arise in prefill (Wei et al., 19 Aug 2025). Within that framework, RFC is not an isolated telemetry value but a scheduling primitive: it is predicted before execution, accumulated as requests are admitted, normalized for heterogeneous hardware, and combined with UFC into a unified Holistic Fairness score that governs proactive fairness-aware scheduling (Wei et al., 19 Aug 2025).

1. Conceptual role in fairness-aware LLM serving

In Equinox’s dual-counter framework, RFC represents the operator’s objective: equitable and efficient allocation of GPU resources. It is explicitly designed to bias scheduling toward configurations that maximize system throughput and keep GPUs highly utilized, favoring batches and requests that push tokens per second and utilization upward (Wei et al., 19 Aug 2025). This operator perspective is distinct from the user perspective encoded by UFC, which measures user-side Quality of Service by combining weighted token usage with user-perceived latency (Wei et al., 19 Aug 2025).

The distinction between RFC and UFC is structurally important. UFC applies a strong decode weighting and latency compensation to protect users from starvation, whereas RFC does not weight decode against prefill, because the operator objective is to maintain efficient pipeline operation and throughput bottlenecks arise in prefill (Wei et al., 19 Aug 2025). The paper summarizes the split as follows: “The UFC ensures equal tenant Quality of Service (QoS) by combining weighted tokens with user perceived latency... The RFC ensures equitable operator allocation using GPU utilization and throughput, without weighting since throughput bottlenecks arise in prefill” (Wei et al., 19 Aug 2025).

This separation places RFC within a broader lineage of multi-resource fairness research, but with a serving-specific interpretation. Classical cluster schedulers formulate fairness over CPU, memory, or storage shares; proportional fairness, for example, is framed as a better efficiency–fairness compromise than dominant resource fairness in stochastic environments (Bonald et al., 2014). Equinox adapts the same general tension—fairness versus efficiency—to token-serving systems, where the relevant operator-side resources are not abstract shares alone but predicted throughput and GPU utilization (Wei et al., 19 Aug 2025). This suggests that RFC is best understood as a serving-native fairness counter: it is fairness over operational efficiency states rather than over raw resource entitlements.

2. Formal definition and update rule

RFC is updated after processing each request batch and integrates two per-batch critical performance metrics: system throughput in tokens per second and GPU utilization percentage. For every client, the update operation is

$$RFC \leftarrow RFC + \omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU},$$

where $\omega_f$ is the per-user priority weight, $\text{TPS}$ is the estimated request throughput in tokens per second in the GPU batch, and $\text{Util}_{GPU}$ is the estimated GPU utilization percentage scaled from 0 to 1 (Wei et al., 19 Aug 2025).

The use of $\omega_f$ aligns RFC with UFC at the level of per-user weighting, but the counter itself remains operator-centric because it multiplies throughput by utilization rather than token volume by latency. RFC therefore aggregates cumulatively, batch by batch, by summing $\omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU}$ for each admitted request of user $f$ (Wei et al., 19 Aug 2025).

Two design choices are central. First, GPU utilization is normalized to $[0,1]$, making RFC portable across heterogeneous GPU setups (Wei et al., 19 Aug 2025). Second, RFC does not apply a decode multiplier analogous to the $4\times$ weighting used in UFC for output tokens (Wei et al., 19 Aug 2025). This asymmetry is deliberate: RFC is intended to reflect operator efficiency, not user-perceived cost asymmetry.

The following table summarizes the two counters in Equinox’s framework.

Counter	Update rule	Operational meaning
UFC	$$UFC \leftarrow UFC + \omega_f \frac{\mathrm{Tokens}_{req}^{in} + 4 \cdot \mathrm{Tokens}_{req}^{out}}{1 + \delta \cdot (\mathrm{WaitTime}_{req} + \mathrm{PredictTime}_{req})}$$	User QoS via weighted tokens and latency
RFC	$\omega_f$0	Operator efficiency via throughput and GPU utilization

Because RFC is multiplicative in TPS and utilization, it rewards requests that are efficient in batched GPU context rather than merely large or short. A plausible implication is that RFC behaves as a compact proxy for hardware-effective service contribution: high TPS without high utilization, or high utilization with poor TPS, both yield smaller increments than jointly favorable requests.

3. Throughput and utilization as operator fairness primitives

Throughput in RFC is defined as system throughput in tokens per second per request as estimated from historical mappings and MoPE’s predicted output tokens, then attached to the request before scheduling (Wei et al., 19 Aug 2025). The paper notes that throughput is compute-bound during prefill and exhibits non-monotonic patterns across token ranges, but within RFC it is represented as an estimated tokens-per-second value for the request in the batched GPU context (Wei et al., 19 Aug 2025).

GPU utilization is defined as GPU utilization percentage scaled from 0 to 1, estimated per request or batch from offline profiling and historical mapping of token length to GPU utilization behavior (Wei et al., 19 Aug 2025). Equinox models utilization as exhibiting step-like plateaus driven by batch-refresh dynamics, and computes a predicted $\omega_f$1 prior to scheduling via an offline-learned mapping (Wei et al., 19 Aug 2025).

This pairing of throughput and utilization is notable because it differs from classical dominant-share formulations. Dominant Resource Fairness equalizes the largest normalized resource share of each job, while proportional fairness maximizes a sum of logarithmic utilities over feasible multi-resource allocations (Bonald et al., 2014). RFC instead treats operational efficiency itself as the relevant fairness substrate. It does not ask whether two tenants consume equal fractions of a GPU; it asks whether admitted requests contribute equitably to operator-valued efficiency under the system’s chosen weighting scheme (Wei et al., 19 Aug 2025).

That design also differs from dependency-aware DRF variants for interdependent resources, which equalize active dominant shares on congested resources while explicitly modeling dependency constraints (Zeidan et al., 24 Jun 2026). RFC does not encode dependency functions over resources. Instead, it compresses serving behavior into throughput and normalized utilization, which are already downstream consequences of batching, memory pressure, and token-length interactions (Wei et al., 19 Aug 2025). This suggests a different abstraction boundary: RFC is a counter over observed or predicted serving outcomes, not over declarative demand vectors.

4. The scheduling paradox and predictive reconstruction

A central difficulty in using RFC operationally is that its constituent metrics are reliably known only after execution. Actual output tokens, latency, throughput, and GPU utilization are post-execution quantities, yet a scheduler must choose whom to schedule before execution. Equinox identifies this as a scheduling paradox: the scheduler needs exactly the metrics that only become available after dispatch and completion (Wei et al., 19 Aug 2025).

Equinox resolves the paradox through a deterministic Mixture of Prediction Experts (MoPE) that predicts user-perceived latency, output tokens, throughput, and GPU utilization before GPU dispatch (Wei et al., 19 Aug 2025). MoPE comprises multiple specialized predictors coordinated by a routing layer that dynamically assigns incoming requests to the most suitable expert, handling diverse prompt types and output token lengths (Wei et al., 19 Aug 2025). The router learns to classify prompts using input length thresholds and automatically identified keywords indicative of output length classes, while each expert is a specialized regression BERT-base model fine-tuned for prediction accuracy (Wei et al., 19 Aug 2025).

The reported trade-off favors three experts: L1 error decreases from 80 with one expert to 33 with three experts and 25 with five experts, but resource usage grows substantially beyond three experts (Wei et al., 19 Aug 2025). Router accuracy peaks around 80% at approximately 110k LMSYS samples and converges after roughly 50k samples (Wei et al., 19 Aug 2025). MoPE introduces router overhead of 0.02 ms and total inference time of 4.5 ms, less than 1% of the average prompt latency of 2,400 ms (Wei et al., 19 Aug 2025).

Online, the router rapidly classifies an incoming prompt and directs it to the relevant expert, incorporating the target LLM identity during preprocessing; the selected expert then generates the token length prediction (Wei et al., 19 Aug 2025). Historical offline data are used to model token-latency-throughput-GPU utilization relationships, extending a latency-validated offline modeling method to GPU utilization and throughput (Wei et al., 19 Aug 2025). Predicted TPS and $\omega_f$2 are then plugged directly into RFC before execution, while predicted output tokens and latency feed UFC (Wei et al., 19 Aug 2025).

This mechanism makes RFC actionable. Without prediction, RFC would be retrospective telemetry. With MoPE and historical mapping, RFC becomes a pre-execution control variable.

5. Integration into Holistic Fairness and scheduler control

RFC is not used alone. Equinox defines a Holistic Fairness score for each user $\omega_f$3 as a weighted linear combination of normalized UFC and RFC values:

$\omega_f$4

with $\omega_f$5 (Wei et al., 19 Aug 2025). The system asserts that end-user experience is the ultimate goal of the service and therefore sets $\omega_f$6 (Wei et al., 19 Aug 2025).

Hyperparameter analysis tests $\omega_f$7 from 0.5 to 0.9 with $\omega_f$8 and reveals a fairness–throughput trade-off: $\omega_f$9 yields peak latency fairness but 20% lower throughput than $\text{TPS}$0, while $\text{TPS}$1 yields maximum throughput but 23% lower fairness than $\text{TPS}$2 (Wei et al., 19 Aug 2025). The selected balance is $\text{TPS}$3, $\text{TPS}$4, which maintains 97% peak fairness while achieving 90% of maximum throughput (Wei et al., 19 Aug 2025).

The scheduling loop uses pre-execution predictions to update UFC and RFC and rank clients by HF. Requests are enqueued by current HF counter; the scheduler repeatedly selects the client with minimal HF, dequeues that client’s head request, and checks feasibility under batch size and GPU memory constraints using canSchedule(req, B, M, Lb) (Wei et al., 19 Aug 2025). If feasible, the request is added to the batch and updateCounter(req, c*) is called using predicted output tokens, latency, GPU utilization, TPS, and observed wait time, thereby invoking both UFC and RFC updates (Wei et al., 19 Aug 2025). After execution, HF is refreshed with actual metrics and the prediction mapping is updated to close the feedback loop (Wei et al., 19 Aug 2025).

RFC shapes this loop by adding $\text{TPS}$5 as requests are admitted, causing the scheduler to preferentially build batches whose predicted throughput and utilization are higher (Wei et al., 19 Aug 2025). Because HF combines RFC with UFC under tunable weights, the resulting policy is neither purely operator-optimal nor purely user-optimal. A plausible implication is that RFC acts as an operator-side regularizer on continuous batching, steering the scheduler away from user-fair but hardware-inefficient batches.

This differs from token-granular fair serving schemes such as Virtual Token Counter (VTC), which equalize service using weighted input and output tokens and prove a tight bounded difference for backlogged clients (Sheng et al., 2023). VTC is fairness over served work; RFC is fairness over operator efficiency contribution. Equinox’s contribution is to merge these axes via HF rather than privileging one alone (Wei et al., 19 Aug 2025).

6. Empirical behavior, bounded discrepancy, and limits

Empirically, RFC is the operator-side driver behind Equinox’s reported gains. On synthetic balanced load, Equinox delivers a higher total service rate than FCFS or VTC by $\text{TPS}$6 and maintains up to 60% lower response times than VTC while sustaining high GPU utilization (Wei et al., 19 Aug 2025). Under Poisson arrivals with heterogeneous requests, Equinox keeps total service rate virtually identical to FCFS but significantly reduces accumulated service difference, correcting VTC’s bias against long-decode requests through MoPE predictions and RFC/UFC balancing (Wei et al., 19 Aug 2025). On SGLang with ShareGPT, Equinox achieves improvements in P50/P90 TTFT of up to 30% and mild throughput improvements of up to 25% when request rate is high (Wei et al., 19 Aug 2025). On vLLM with ShareGPT, it yields up to 33% higher and more stable Jain’s fairness, slightly lower TTFT and end-to-end latency, and slightly higher per-client service rate (Wei et al., 19 Aug 2025). The abstract aggregates these findings as up to $\text{TPS}$7 higher throughput, 60% lower time-to-first-token latency, 13% higher fairness versus VTC, and 94% GPU utilization (Wei et al., 19 Aug 2025).

The paper states that Equinox proves fairness under bounded discrepancy across heterogeneous platforms, but formal theorem or lemma statements with proofs are not included in the body text (Wei et al., 19 Aug 2025). Instead, bounded discrepancy is evidenced empirically. In synthetic ablation, Equinox with MoPE substantially reduces maximum, average, and variance of accumulated absolute service difference relative to baselines, approaching an oracle variant (Wei et al., 19 Aug 2025). Cross-system Jain’s Fairness Index improves by up to 13% versus FCFS and VTC on S-LoRA, vLLM, and SGLang (Wei et al., 19 Aug 2025).

These observations support, but do not formally establish in the main text, the role of RFC as part of a bounded-discrepancy fairness mechanism. This suggests that the guarantee is empirical and contingent on sufficiently accurate predictions and stationary-enough mappings between token length, latency, throughput, and utilization over the profiling window (Wei et al., 19 Aug 2025).

RFC also has explicit limits. MoPE predictions remain imperfect, especially for very long generations; router accuracy peaks around 80%, and expanding beyond three experts raises memory and latency overhead substantially (Wei et al., 19 Aug 2025). Historical mappings for throughput and utilization are extended empirically rather than derived from a formal utilization theorem, so portability to non-standard models and workloads may require re-profiling (Wei et al., 19 Aug 2025). Under constant overload, fairness gains plateau: Equinox matches VTC’s fairness while still improving total service rate (Wei et al., 19 Aug 2025). Very long decode requests remain memory-bound and can depress throughput despite RFC’s incentives, while UFC’s latency compensation protects against starvation without eliminating architectural constraints (Wei et al., 19 Aug 2025). Multi-node deployment would require distributed UFC/RFC state and synchronization; current evaluations focus on single-node or single-cluster configurations such as A100-80GB and 8×A100-40GB (Wei et al., 19 Aug 2025).

In heterogeneous-platform settings, RFC’s normalization of GPU utilization to $\text{TPS}$8 is the main portability mechanism, and scalability experiments from 1 to 8 GPUs with proportional tensor parallelism indicate robust setup-agnostic behavior (Wei et al., 19 Aug 2025). This hardware normalization is narrower than broader multi-resource heterogeneity frameworks in GPU clusters, such as OEF, which optimize throughput under envy-freeness, sharing incentive, or strategy-proofness over type-dependent GPU speedups (Mo et al., 2024). Equinox instead treats heterogeneity through normalized utilization and predicted per-request TPS, keeping the fairness abstraction at serving time rather than at cluster-wide allocation (Wei et al., 19 Aug 2025).

RFC therefore occupies a specific place in the fairness literature. It is neither classical DRF nor PF, neither a sketch-based fairness counter nor a generic socio-technical parity metric. It is an operator-side fairness counter for LLM serving that accumulates $\text{TPS}$9 per admitted request, is made actionable through predictive modeling, and is ultimately subordinate to a broader Holistic Fairness objective that balances hardware efficiency against user-facing QoS (Wei et al., 19 Aug 2025).