Queue Dynamic State Encoding (QDSE)

• QDSE is a dual-use concept that encodes microscopic queue dynamics in decentralized traffic control and computes scaled sojourn time in network queue management.
• In traffic control, it uses six lane-level indicators to predict vehicle inflow and enhance multi-agent reinforcement learning performance.
• In network management, QDSE adjusts packet sojourn times with backlog ratios to provide up-to-date congestion signals for active queue management.

Queue Dynamic State Encoding (QDSE) denotes two distinct technical constructs in the arXiv literature. In decentralized traffic signal control, QDSE is a lane-level state representation introduced in CoordLight to encode queueing dynamics for Multi-Agent Reinforcement Learning (MARL) at signalized intersections (Zhang et al., 25 Mar 2026). In queue management for data networks, the same label is used for “scaled sojourn time,” a congestion signal that scales packet sojourn time by the ratio of queue backlogs at dequeue and enqueue (Briscoe, 2019). The shared acronym reflects a common concern with queue dynamics, but the underlying objects, measurements, and control loops differ substantially.

1. Terminological scope and domain separation

In CoordLight, QDSE is described as “a novel state representation based on vehicle queuing models,” designed to strengthen agents’ capability to analyze, predict, and respond to local traffic dynamics in Adaptive Traffic Signal Control (ATSC) (Zhang et al., 25 Mar 2026). The representation replaces “traditional, myopic intersection-level state descriptors (e.g., raw vehicle counts or ‘pressures’)” with a compact encoding of microscopic queue evolution.

In the queue-management memo, QDSE is presented as “scaled sojourn time,” a metric for congestion signalling within an Active Queue Management (AQM) algorithm (Briscoe, 2019). Rather than describing traffic state for an RL agent, it modifies how congestion information is derived from packet backlog and packet residence time in a queue.

The overlap in acronym can create a misconception that the two uses are variants of a single framework. They are not. One is a per-lane observation tensor for decentralized MARL in road networks; the other is a dequeue-time congestion signal for packet queues. A plausible implication is that the acronym should always be interpreted in conjunction with its application domain.

2. QDSE in decentralized traffic signal control

CoordLight treats each intersection in discrete decision epochs $t$, each of duration $\Delta t$ “(5 s in our experiments),” and defines QDSE over the set of incoming lanes $L_{\rm in}$ (Zhang et al., 25 Mar 2026). The full state is

$$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$

where each row is a $|L_{\rm in}|$-dimensional vector of lane-specific values.

The queue update is given by

$$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$

Here $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$ are stopped-vehicle queue lengths per lane, while $\Delta_{\rm in}(t)$ and $\Delta_{\rm out}(t)$ are the vectors of vehicles joining and departing the queue during the current green phase. To estimate $\Delta_{\rm in}$, CoordLight predicts which moving vehicles will catch up to the back of the queue before the next decision step. For a vehicle $\Delta t$0 with instantaneous speed $\Delta t$1 and Intelligent Driver Model (IDM) acceleration $\Delta t$2, the estimated travel distance over $\Delta t$3 is

$\Delta t$4

If $\Delta t$5 denotes the measured distance from vehicle $\Delta t$6 to the current queue tail on its lane, then the predicted joiners for lane $\Delta t$7 are approximated by

$\Delta t$8

The six lane-level signals collected in QDSE are:

1. $\Delta t$9: current queue length.
2. $L_{\rm in}$0: predicted joiners.
3. $L_{\rm in}$1: vehicles departed during last phase.
4. $L_{\rm in}$2: total moving vehicles on the lane.
5. $L_{\rm in}$3: distance from queue tail to the nearest moving vehicle.
6. $L_{\rm in}$4: number of moving vehicles within $L_{\rm in}$5.

CoordLight states that this representation captures both “instantaneous congestion” through $L_{\rm in}$6 and $L_{\rm in}$7 and “incipient congestion dynamics” through $L_{\rm in}$8, $L_{\rm in}$9, $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$0, and $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$1 (Zhang et al., 25 Mar 2026). This suggests that QDSE is intended not merely as a descriptive state, but as a predictive summary of near-future queue formation.

3. Measurement model and per-step computation in CoordLight

CoordLight specifies how each QDSE component is measured or predicted from roadway sensing infrastructure (Zhang et al., 25 Mar 2026). Queue length $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$2 is counted via loop detectors or camera-based occupancy, with vehicles below a speed threshold deemed “stopped.” Departures $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$3 are measured by counting vehicles that cross the stop-bar during the green interval. The moving-vehicle set $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$4 contains all vehicles on lane $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$5 moving above the stop threshold, and its size gives $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$6. The distance feature $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$7 is obtained by locating the nearest moving vehicle upstream of the queue tail, “again from camera or LiDAR,” and $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$8 counts moving vehicles within that range. The predicted inflow $$S(t) \in \mathbb{R}^{6\times |L_{\rm in}|} = \begin{bmatrix} Q(t)\ N_{\rm in}(t)\ N_{\rm out}(t)\ N_r(t)\ D_{\rm fr}(t)\ N_{\rm fr}(t) \end{bmatrix},$$9 is computed from each moving vehicle’s current speed and IDM acceleration by comparing estimated travel distance over 5 s with distance to the queue.

The paper also provides a per-step computation sketch. For each incoming lane, the procedure counts stopped vehicles, counts departures during the last phase, identifies moving vehicles, computes $|L_{\rm in}|$0, determines the nearest moving vehicle distance, counts vehicles in the nearest range, and then iterates over moving vehicles to compute IDM-based $|L_{\rm in}|$1, evaluate $|L_{\rm in}|$2, and increment $|L_{\rm in}|$3 whenever $|L_{\rm in}|$4 exceeds distance to the queue (Zhang et al., 25 Mar 2026). After these steps, the per-intersection state is assembled as

$|L_{\rm in}|$5

Two implementation notes delimit the intended computation. First, “No explicit smoothing or historical averaging is applied—the GRU in the actor-critic network handles temporal aggregation of the recent QDSE sequence.” Second, each feature is clipped or normalized to a fixed range before embedding. These details indicate that QDSE is treated as an instantaneous structured observation, with temporal integration delegated to the downstream recurrent policy architecture.

4. Functional role in Neighbor-aware Policy Optimization

CoordLight couples QDSE with Neighbor-aware Policy Optimization (NAPO), its MARL algorithm for decentralized coordination (Zhang et al., 25 Mar 2026). For agent $|L_{\rm in}|$6, the observation is

$|L_{\rm in}|$7

where $|L_{\rm in}|$8 and $|L_{\rm in}|$9 denotes the four geographically adjacent neighbors.

The actor network linearly projects each neighbor state into $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$0-dimensional vectors and applies a multi-head attention block to compute attention weights $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$1 among neighboring states. The resulting weighted sum, combined with a GRU over the past $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$2 observations, yields the policy embedding $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$3, from which the action distribution $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$4 is produced. The critic uses the same QDSE embeddings together with an action-attention block that weights neighbors’ actions via $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$5 to produce a neighbor-aware value estimate $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$6. Generalized Advantage Estimation (GAE) is then computed using $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$7, and policy updates are driven in a PPO-style manner.

CoordLight explicitly states that QDSE is “the only perceptual input” to NAPO (Zhang et al., 25 Mar 2026). Consequently, the representational sufficiency of QDSE directly affects the attention modules’ ability to identify “influential” neighbors and the critic’s ability to assign credit. A plausible implication is that CoordLight’s coordination performance depends not only on attention or PPO-style optimization, but also on the predictive granularity embedded in the six QDSE signals.

5. Empirical evidence for the traffic-control formulation

The CoordLight paper reports an ablation over the high-demand Hangzhou map $$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$8 against five alternative state definitions: VC (vehicle counts), GP (general pressure), EP (efficient pressure), ATS (advanced traffic state), and DTSE (discrete grid encoding) (Zhang et al., 25 Mar 2026). Figure 1(a), as summarized in the provided material, shows that QDSE achieves the lowest average queue length, the highest average speed, the lowest average travel time, and the smallest queue-length variance, while converging faster and more stably than the alternatives.

On the same dataset, “CoordLight w/ QDSE outperforms CoordLight w/o QDSE by 10–15% in travel-time reduction,” which the paper states confirms that the predicted-inflow and proximity features materially improve decision quality (Zhang et al., 25 Mar 2026). In network-wide benchmarks over Jinan, Hangzhou, and New York, with “up to 196 intersections,” CoordLight “consistently beats state-of-the-art baselines by 5–15% in average travel time,” and the ablation results isolate QDSE’s contribution to “roughly half of that gain.”

These findings are specific to the CoordLight formulation. They do not establish that QDSE is universally optimal as a traffic-state encoding, but they do document that, within this framework and these datasets, a queue-dynamics representation outperforms several alternative state definitions. This suggests that microscopic inflow prediction and queue-tail proximity can materially alter policy quality in decentralized ATSC.

6. QDSE as scaled sojourn time in queue management

In the queue-management memo, QDSE refers to a different object: the “scaled sojourn time” metric for congestion signalling (Briscoe, 2019). A packet carries a timestamp when enqueued, and on dequeue its raw sojourn time is

$$Q(t+1)=Q(t)+\Delta_{\rm in}(t)-\Delta_{\rm out}(t).$$9

QDSE augments this by recording backlog in bytes at enqueue and dequeue. With $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$0 the backlog at enqueue and $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$1 the backlog at dequeue, the scaled sojourn time is defined as

$Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$2

The memo derives this quantity via average service rates. If $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$3, $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$4, and $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$5, then the average departure rate during the packet’s sojourn is

$Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$6

and the time required to drain the current backlog $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$7 at that same rate is

$Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$8

which recovers the scaled-sojourn expression.

The memo’s stated motivation is to reduce delay in congestion signalling rather than directly minimize queuing delay of data. Traditional queue-delay signals suffer “measurement delay,” because one only learns a packet’s sojourn time after it has traversed the queue, and “queue delay,” because a signal applied at enqueue must itself wait in the queue before being sent. By computing at dequeue and scaling by $Q(t)=[Q^l(t)]_{l\in L_{\rm in}}$9, the algorithm “immediately ‘extrapolate[s]’ an up-to-date estimate of the queue’s delay—even if the queue has grown or shrunk since the packet first entered” (Briscoe, 2019).

The implementation sketch maintains byte counters for enqueued and dequeued traffic, stores enqueue timestamp and enqueue backlog per packet, computes raw sojourn at dequeue, scales it by the backlog ratio when $\Delta_{\rm in}(t)$0, and marks or drops the packet if $\Delta_{\rm in}(t)$1 exceeds a delay target (Briscoe, 2019). The memo also describes an “optimized integer trick” using shifts based on $\Delta_{\rm in}(t)$2 backlog values, and proposes deterministic signalling in the network rather than in-line randomization.

A critical limitation is stated explicitly: “no empirical evaluation has yet been performed (‘These ideas … have not been evaluated either.’)” (Briscoe, 2019). Theoretical claims in the memo therefore concern signalling delay rather than experimentally validated end-to-end performance.

7. Comparative interpretation and recurrent points of confusion

The two QDSE formulations are linked by attention to queue dynamics, but they operate at different abstraction levels. CoordLight’s QDSE is a multivariate state tensor over incoming road lanes; the AQM QDSE is a scalar congestion signal associated with a packet at dequeue. CoordLight uses QDSE as input to a GRU-attention actor-critic architecture; the AQM memo uses QDSE to accelerate the control loop within queue management (Zhang et al., 25 Mar 2026, Briscoe, 2019).

A common misconception would be to interpret the traffic-control QDSE as a direct adaptation of scaled sojourn time. The supplied material does not support that claim. CoordLight defines QDSE entirely through six lane-level queue-dynamics signals grounded in stopped vehicles, moving vehicles, queue-tail distance, and IDM-based inflow prediction, whereas the memo defines QDSE through the ratio of queue backlogs multiplied by packet sojourn time.

Another point of contrast concerns empirical status. CoordLight reports benchmark and ablation results over real-world traffic datasets, including improvements in average travel time and queue metrics (Zhang et al., 25 Mar 2026). The queue-management memo explicitly reports no empirical evaluation (Briscoe, 2019). This asymmetry matters for interpretation: the former is an experimentally supported state encoding within a specific MARL system, whereas the latter is a theoretically motivated signalling metric awaiting validation.

Taken together, the literature uses Queue Dynamic State Encoding to denote methods that compress queue evolution into a control-relevant signal. In traffic signal control, that compression takes the form of six per-lane variables intended to expose immediate and incipient congestion to decentralized agents. In packet queue management, it takes the form of scaled sojourn time intended to expose fresher congestion information to the control loop. The commonality is the emphasis on queue dynamics; the substantive formulations are domain-specific and should not be conflated.