---
title: Resource Fairness Counter in LLM Serving
url: https://www.emergentmind.com/topics/resource-fairness-counter
type: topic
---

# Resource Fairness Counter in LLM Serving

Searching arXiv for the main paper and closely related fairness work to ground the article.
Resource Fairness Counter (RFC) is an operator-side fairness metric in large language model (LLM) serving that quantifies how scheduling decisions affect server efficiency, specifically through throughput and GPU utilization, rather than user-perceived quality of service. In Equinox, RFC forms one half of a dual-counter framework together with the User-Fairness Counter (UFC): UFC tracks user-side service using weighted tokens and latency, while RFC tracks equitable operator allocation using GPU utilization and throughput without decode weighting, because throughput bottlenecks arise in prefill [2508.16646]. Within that framework, RFC is not an isolated telemetry value but a scheduling primitive: it is predicted before execution, accumulated as requests are admitted, normalized for heterogeneous hardware, and combined with UFC into a unified Holistic Fairness score that governs proactive fairness-aware scheduling [2508.16646].

## 1. Conceptual role in fairness-aware LLM serving

In Equinox’s dual-counter framework, RFC represents the operator’s objective: equitable and efficient allocation of GPU resources. It is explicitly designed to bias scheduling toward configurations that maximize system throughput and keep GPUs highly utilized, favoring batches and requests that push tokens per second and utilization upward [2508.16646]. This operator perspective is distinct from the user perspective encoded by UFC, which measures user-side Quality of Service by combining weighted token usage with user-perceived latency [2508.16646].

The distinction between RFC and UFC is structurally important. UFC applies a strong decode weighting and latency compensation to protect users from starvation, whereas RFC does not weight decode against prefill, because the operator objective is to maintain efficient pipeline operation and throughput bottlenecks arise in prefill [2508.16646]. The paper summarizes the split as follows: “The UFC ensures equal tenant Quality of Service (QoS) by combining weighted tokens with user perceived latency... The RFC ensures equitable operator allocation using GPU utilization and throughput, without weighting since throughput bottlenecks arise in prefill” [2508.16646].

This separation places RFC within a broader lineage of multi-resource fairness research, but with a serving-specific interpretation. Classical cluster schedulers formulate fairness over CPU, memory, or storage shares; proportional fairness, for example, is framed as a better efficiency–fairness compromise than dominant resource fairness in stochastic environments [1404.2266]. Equinox adapts the same general tension—fairness versus efficiency—to token-serving systems, where the relevant operator-side resources are not abstract shares alone but predicted throughput and GPU utilization [2508.16646]. This suggests that RFC is best understood as a serving-native fairness counter: it is fairness over operational efficiency states rather than over raw resource entitlements.

## 2. Formal definition and update rule

RFC is updated after processing each request batch and integrates two per-batch critical performance metrics: system throughput in tokens per second and GPU utilization percentage. For every client, the update operation is

\[
RFC \leftarrow RFC + \omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU},
\]

where $\omega_f$ is the per-user priority weight, $\text{TPS}$ is the estimated request throughput in tokens per second in the GPU batch, and $\text{Util}_{GPU}$ is the estimated GPU utilization percentage scaled from 0 to 1 [2508.16646].

The use of $\omega_f$ aligns RFC with UFC at the level of per-user weighting, but the counter itself remains operator-centric because it multiplies throughput by utilization rather than token volume by latency. RFC therefore aggregates cumulatively, batch by batch, by summing $\omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU}$ for each admitted request of user $f$ [2508.16646].

Two design choices are central. First, GPU utilization is normalized to $[0,1]$, making RFC portable across heterogeneous GPU setups [2508.16646]. Second, RFC does not apply a decode multiplier analogous to the $4\times$ weighting used in UFC for output tokens [2508.16646]. This asymmetry is deliberate: RFC is intended to reflect operator efficiency, not user-perceived cost asymmetry.

The following table summarizes the two counters in Equinox’s framework.

| Counter | Update rule | Operational meaning |
|---|---|---|
| UFC | \(UFC \leftarrow UFC + \omega_f \frac{\mathrm{Tokens}_{req}^{in} + 4 \cdot \mathrm{Tokens}_{req}^{out}}{1 + \delta \cdot (\mathrm{WaitTime}_{req} + \mathrm{PredictTime}_{req})}\) | User QoS via weighted tokens and latency |
| RFC | \(RFC \leftarrow RFC + \omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU}\) | Operator efficiency via throughput and GPU utilization |

Because RFC is multiplicative in TPS and utilization, it rewards requests that are efficient in batched GPU context rather than merely large or short. A plausible implication is that RFC behaves as a compact proxy for hardware-effective service contribution: high TPS without high utilization, or high utilization with poor TPS, both yield smaller increments than jointly favorable requests.

## 3. Throughput and utilization as operator fairness primitives

Throughput in RFC is defined as system throughput in tokens per second per request as estimated from historical mappings and MoPE’s predicted output tokens, then attached to the request before scheduling [2508.16646]. The paper notes that throughput is compute-bound during prefill and exhibits non-monotonic patterns across token ranges, but within RFC it is represented as an estimated tokens-per-second value for the request in the batched GPU context [2508.16646].

GPU utilization is defined as GPU utilization percentage scaled from 0 to 1, estimated per request or batch from offline profiling and historical mapping of token length to GPU utilization behavior [2508.16646]. Equinox models utilization as exhibiting step-like plateaus driven by batch-refresh dynamics, and computes a predicted $\text{Util}_{GPU}$ prior to scheduling via an offline-learned mapping [2508.16646].

This pairing of throughput and utilization is notable because it differs from classical dominant-share formulations. Dominant Resource Fairness equalizes the largest normalized resource share of each job, while proportional fairness maximizes a sum of logarithmic utilities over feasible multi-resource allocations [1404.2266]. RFC instead treats operational efficiency itself as the relevant fairness substrate. It does not ask whether two tenants consume equal fractions of a GPU; it asks whether admitted requests contribute equitably to operator-valued efficiency under the system’s chosen weighting scheme [2508.16646].

That design also differs from dependency-aware DRF variants for interdependent resources, which equalize active dominant shares on congested resources while explicitly modeling dependency constraints [2606.25540]. RFC does not encode dependency functions over resources. Instead, it compresses serving behavior into throughput and normalized utilization, which are already downstream consequences of batching, memory pressure, and token-length interactions [2508.16646]. This suggests a different abstraction boundary: RFC is a counter over observed or predicted serving outcomes, not over declarative demand vectors.

## 4. The scheduling paradox and predictive reconstruction

A central difficulty in using RFC operationally is that its constituent metrics are reliably known only after execution. Actual output tokens, latency, throughput, and GPU utilization are post-execution quantities, yet a scheduler must choose whom to schedule before execution. Equinox identifies this as a scheduling paradox: the scheduler needs exactly the metrics that only become available after dispatch and completion [2508.16646].

Equinox resolves the paradox through a deterministic Mixture of Prediction Experts (MoPE) that predicts user-perceived latency, output tokens, throughput, and GPU utilization before GPU dispatch [2508.16646]. MoPE comprises multiple specialized predictors coordinated by a routing layer that dynamically assigns incoming requests to the most suitable expert, handling diverse prompt types and output token lengths [2508.16646]. The router learns to classify prompts using input length thresholds and automatically identified keywords indicative of output length classes, while each expert is a specialized regression BERT-base model fine-tuned for prediction accuracy [2508.16646].

The reported trade-off favors three experts: L1 error decreases from 80 with one expert to 33 with three experts and 25 with five experts, but resource usage grows substantially beyond three experts [2508.16646]. Router accuracy peaks around 80% at approximately 110k LMSYS samples and converges after roughly 50k samples [2508.16646]. MoPE introduces router overhead of 0.02 ms and total inference time of 4.5 ms, less than 1% of the average prompt latency of 2,400 ms [2508.16646].

Online, the router rapidly classifies an incoming prompt and directs it to the relevant expert, incorporating the target LLM identity during preprocessing; the selected expert then generates the token length prediction [2508.16646]. Historical offline data are used to model token-latency-throughput-GPU utilization relationships, extending a latency-validated offline modeling method to GPU utilization and throughput [2508.16646]. Predicted TPS and $\text{Util}_{GPU}$ are then plugged directly into RFC before execution, while predicted output tokens and latency feed UFC [2508.16646].

This mechanism makes RFC actionable. Without prediction, RFC would be retrospective telemetry. With MoPE and historical mapping, RFC becomes a pre-execution control variable.

## 5. Integration into Holistic Fairness and scheduler control

RFC is not used alone. Equinox defines a Holistic Fairness score for each user $f$ as a weighted linear combination of normalized UFC and RFC values:

\[
\text{HF}_f = \alpha \cdot \text{UFC}_f + \beta \cdot \text{RFC}_f,
\]

with $\alpha + \beta = 1$ [2508.16646]. The system asserts that end-user experience is the ultimate goal of the service and therefore sets $\alpha > \beta$ [2508.16646].

Hyperparameter analysis tests $\alpha$ from 0.5 to 0.9 with $\beta = 1-\alpha$ and reveals a fairness–throughput trade-off: $\alpha=0.9$ yields peak latency fairness but 20% lower throughput than $\alpha=0.5$, while $\alpha=0.5$ yields maximum throughput but 23% lower fairness than $\alpha=0.9$ [2508.16646]. The selected balance is $\alpha=0.7$, $\beta=0.3$, which maintains 97% peak fairness while achieving 90% of maximum throughput [2508.16646].

The scheduling loop uses pre-execution predictions to update UFC and RFC and rank clients by HF. Requests are enqueued by current HF counter; the scheduler repeatedly selects the client with minimal HF, dequeues that client’s head request, and checks feasibility under batch size and GPU memory constraints using `canSchedule(req, B, M, Lb)` [2508.16646]. If feasible, the request is added to the batch and `updateCounter(req, c*)` is called using predicted output tokens, latency, GPU utilization, TPS, and observed wait time, thereby invoking both UFC and RFC updates [2508.16646]. After execution, HF is refreshed with actual metrics and the prediction mapping is updated to close the feedback loop [2508.16646].

RFC shapes this loop by adding $\omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU}$ as requests are admitted, causing the scheduler to preferentially build batches whose predicted throughput and utilization are higher [2508.16646]. Because HF combines RFC with UFC under tunable weights, the resulting policy is neither purely operator-optimal nor purely user-optimal. A plausible implication is that RFC acts as an operator-side regularizer on continuous batching, steering the scheduler away from user-fair but hardware-inefficient batches.

This differs from token-granular fair serving schemes such as Virtual Token Counter (VTC), which equalize service using weighted input and output tokens and prove a tight bounded difference for backlogged clients [2401.00588]. VTC is fairness over served work; RFC is fairness over operator efficiency contribution. Equinox’s contribution is to merge these axes via HF rather than privileging one alone [2508.16646].

## 6. Empirical behavior, bounded discrepancy, and limits

Empirically, RFC is the operator-side driver behind Equinox’s reported gains. On synthetic balanced load, Equinox delivers a higher total service rate than FCFS or VTC by $1.3\times$ and maintains up to 60% lower response times than VTC while sustaining high GPU utilization [2508.16646]. Under Poisson arrivals with heterogeneous requests, Equinox keeps total service rate virtually identical to FCFS but significantly reduces accumulated service difference, correcting VTC’s bias against long-decode requests through MoPE predictions and RFC/UFC balancing [2508.16646]. On SGLang with ShareGPT, Equinox achieves improvements in P50/P90 TTFT of up to 30% and mild throughput improvements of up to 25% when request rate is high [2508.16646]. On vLLM with ShareGPT, it yields up to 33% higher and more stable Jain’s fairness, slightly lower TTFT and end-to-end latency, and slightly higher per-client service rate [2508.16646]. The abstract aggregates these findings as up to $1.3\times$ higher throughput, 60% lower time-to-first-token latency, 13% higher fairness versus VTC, and 94% GPU utilization [2508.16646].

The paper states that Equinox proves fairness under bounded discrepancy across heterogeneous platforms, but formal theorem or lemma statements with proofs are not included in the body text [2508.16646]. Instead, bounded discrepancy is evidenced empirically. In synthetic ablation, Equinox with MoPE substantially reduces maximum, average, and variance of accumulated absolute service difference relative to baselines, approaching an oracle variant [2508.16646]. Cross-system Jain’s Fairness Index improves by up to 13% versus FCFS and VTC on S-LoRA, vLLM, and SGLang [2508.16646].

These observations support, but do not formally establish in the main text, the role of RFC as part of a bounded-discrepancy fairness mechanism. This suggests that the guarantee is empirical and contingent on sufficiently accurate predictions and stationary-enough mappings between token length, latency, throughput, and utilization over the profiling window [2508.16646].

RFC also has explicit limits. MoPE predictions remain imperfect, especially for very long generations; router accuracy peaks around 80%, and expanding beyond three experts raises memory and latency overhead substantially [2508.16646]. Historical mappings for throughput and utilization are extended empirically rather than derived from a formal utilization theorem, so portability to non-standard models and workloads may require re-profiling [2508.16646]. Under constant overload, fairness gains plateau: Equinox matches VTC’s fairness while still improving total service rate [2508.16646]. Very long decode requests remain memory-bound and can depress throughput despite RFC’s incentives, while UFC’s latency compensation protects against starvation without eliminating architectural constraints [2508.16646]. Multi-node deployment would require distributed UFC/RFC state and synchronization; current evaluations focus on single-node or single-cluster configurations such as A100-80GB and 8×A100-40GB [2508.16646].

In heterogeneous-platform settings, RFC’s normalization of GPU utilization to $[0,1]$ is the main portability mechanism, and scalability experiments from 1 to 8 GPUs with proportional tensor parallelism indicate robust setup-agnostic behavior [2508.16646]. This hardware normalization is narrower than broader multi-resource heterogeneity frameworks in GPU clusters, such as OEF, which optimize throughput under envy-freeness, sharing incentive, or strategy-proofness over type-dependent GPU speedups [2403.18545]. Equinox instead treats heterogeneity through normalized utilization and predicted per-request TPS, keeping the fairness abstraction at serving time rather than at cluster-wide allocation [2508.16646].

RFC therefore occupies a specific place in the fairness literature. It is neither classical DRF nor PF, neither a sketch-based fairness counter nor a generic socio-technical parity metric. It is an operator-side fairness counter for LLM serving that accumulates $\omega_f \cdot \text{TPS} \cdot \text{Util}_{GPU}$ per admitted request, is made actionable through predictive modeling, and is ultimately subordinate to a broader Holistic Fairness objective that balances hardware efficiency against user-facing QoS [2508.16646].

Source: https://www.emergentmind.com/topics/resource-fairness-counter