---
title: Discrete Traffic State Encoding
url: https://www.emergentmind.com/topics/discrete-traffic-state-encoding-dtse
type: topic
---

# Discrete Traffic State Encoding

A Discrete Traffic State Encoding (DTSE) is a formalism by which the state of a vehicular traffic system is represented as a collection of discrete units—either spatial, velocity, or phase variables—to enable efficient and information-dense state description for control, simulation, and analytic purposes. DTSEs have been developed and utilized in both deep reinforcement learning for traffic signal control and in kinetic theory models of vehicular traffic, providing a flexible and precise bridge between microscopic reality and macroscopic modeling. The following sections systematically detail DTSE’s foundations, mathematical definitions, construction procedures, and impact in recent research.

## 1. Formal Foundations of Discrete Traffic State Encoding

The core principle underlying DTSE is the partition of position and velocity space into finite, discrete sets, such that the state of traffic at any moment is fully specified by occupancy and vehicle properties in each cell and velocity class. In the reinforcement learning for traffic control context, [1611.01142] defines the instantaneous state as
\[
s_t = (B_t, V_t, p_t) \in \{0,1\}^{M \times n} \times [0,1]^{M \times n} \times \{0,1\}^{|A|}
\]
where:
- $B_t[i,j]$ indicates presence (1) or absence (0) of a vehicle in spatial cell $i$ of lane $j$,
- $V_t[i,j]$ is the normalized speed $v_{i,j} / v_{max}$ for occupied cells (else zero),
- $p_t$ is a one-hot phase vector representing the current traffic signal phase.

Analogously, [1210.7988] describes DTSE in a kinetic theory framework as placing each vehicle into one of finitely many $(x_i, v_k)$ pairs, where $x_i$ marks spatial cell, and $v_k$ labels velocity class. The state space is then
\[
S = \{s_{i,k} = (x_i, v_k) \mid i=1, \dots, N_x; \ k=1, \dots, N_v\}
\]
and the system is described by the distribution function $f_{i,k}(t)$.

This formalization enables systemic discretization of both space and velocity, such that all dynamical, control, or analytical processes are executed on a finite-dimensional space.

## 2. DTSE Construction Methodologies

DTSE construction proceeds via sequential quantization and encoding steps. In the reinforcement learning setting [1611.01142], the process entails:

1. **Parameter Selection**: Determining the observation horizon $\ell$ and cell length $c$; $M = \lceil \ell/c \rceil$ cells per lane.
2. **Lane Discretization**: Partition of roadway segment into $M$ contiguous cells for each of $n$ incoming lanes.
3. **Occupancy Encoding $B_t$**: Each vehicle detected in $0 \leq x_v \leq \ell$ on lane $j$ is mapped to $B_t[i,j]=1$ for $i = \lceil x_v/c \rceil$.
4. **Speed Encoding $V_t$**: For each occupied cell, retrieve speed $v_{i,j}$ and set $V_t[i,j] = v_{i,j}/v_{max}$; else set to zero.
5. **Phase Encoding $p_t$**: Generate a one-hot vector specifying the current phase among the set of available actions.

In the kinetic theory approach [1210.7988], the road is divided into cells $I_i$ of length $\ell$, and velocities are quantized into $N_v$ classes. The discrete state is thus $(x_i, v_k)$, with the population measured by $f_{i,k}(t)$.

Both frameworks rely on consistent and lossless mappings from raw simulation or measurement data to the respective discrete state representations, ensuring preservation of spatial and velocity granularity.

## 3. Mathematical Dynamics and Numerical Implementation

**Reinforcement Learning DTSE**: The DTSE $(B_t, V_t, p_t)$ forms the input to a deep CNN agent. The network processes $B_t$ and $V_t$ as separate channels, using convolutional towers with the following architecture [1611.01142]:

- Conv1: 16 filters, size $4 \times 4$, stride 2, ReLU activation.
- Conv2: 32 filters, size $2 \times 2$, stride 1, ReLU activation.
- Towers are flattened, concatenated with $p_t$, then processed by two dense layers (sizes 128 and 64, ReLU).
- Final linear layer produces $|A|$ Q-values for action selection.

**Kinetic Theory DTSE**: Dynamic evolution is governed by a set of ODEs for $f_{i,k}(t)$:
\[
\frac{d f_{i,k}}{d t} +
v_k \left[ \Phi_{i,i+1} f_{i,k} - \Phi_{i-1,i} f_{i-1,k} \right] = Q_{i,k}[f]
\]
where $\Phi$ is a flux-limiter (enforcing local density constraints), and $Q_{i,k}[f]$ encodes acceleration, braking, and tandem interactions via probabilistic kernels $A_{h\ell}^k$. Numerical integration employs explicit Euler schemes under CFL-type conditions to ensure stability.

## 4. Parameter Choices and Representation Rationale

**Reinforcement Learning Example Parameters** [1611.01142]:
- $\ell = 75\,\mathrm{m}$ (approximate span of 15 car lengths), $c = 5\,\mathrm{m}$ (average vehicle length), $M = 15$,
- $n = 4$ lanes,
- $|A| = 4$ phases,
- Input tensor: $15 \times 4 \times 2 = 120$-dimensional for $(B,V)$, plus 4-dimensional $p$.

This configuration captures individual vehicle granularity while maintaining computational tractability. Observation and quantization parameters reflect a balance between spatial/velocity resolution and input dimensionality.

**Kinetic Theory Example Parameters** [1210.7988]:
- $N_x = 10$ cells, $N_v = 6$ velocity classes, $\ell = 0.1$, $v_k = \frac{k-1}{5}$, $\eta_0=1$, $\beta=0.5$, time step $\Delta t = 0.01$.
- Key parameters $\alpha \in [0,1]$ (road condition), $\beta$ (anticipation).

These parameters allow macroscopic traffic patterns—such as free/congested flow phase transition at empirically realistic critical densities—to emerge from a finite, interpretable set of microstates.

## 5. Expressiveness and Analytical Properties

DTSE provides a rich, yet tractable, embedding of the traffic state. In the deep RL context, the DTSE outperforms low-dimensional state representations, yielding:

- 82% reduction in cumulative delay,
- 66% reduction in average queue length,
- 20% reduction in average travel time,
- Comparable throughput to baselines

[1611.01142]. The scheme’s expressive power arises from explicit modeling of individual vehicle positions and velocities per cell, as opposed to summary statistics (e.g., queue length per lane).

In kinetic traffic models, discrete state encoding enables exact reproduction of granular effects—such as queue formation behind bottlenecks and stop-and-go traffic at signals—through local and pairwise state updates [1210.7988]. The emergence of fundamental diagrams (flow vs. density, speed vs. density) and phase transitions is obtained directly from the dynamics over the discrete state space.

## 6. Applications in Traffic Control and Modeling

DTSE has proven especially impactful in:

- **Deep RL for Adaptive Traffic Signal Control**: Feeding DTSE into convolutional agents controlling intersection signals in micro-simulated environments (SUMO) enables context-sensitive, adaptive policies with robust performance improvements over traditional methods [1611.01142].
- **Analytically Tractable Kinetic Models**: DTSE forms the phase space for ODE systems in kinetic traffic theories, supporting rigorous study of large-scale, realistic networks while encapsulating microscopic features [1210.7988].
- **Reproducibility and Benchmarking**: Because the encoding is formally defined and constructed directly from raw data or simulation, DTSE-based studies are reproducible and facilitate robust cross-comparison.

These applications collectively demonstrate that information-dense, finite-dimensional representations such as DTSE are foundational to modern approaches for modeling, controlling, and analyzing vehicular traffic systems.

Source: https://www.emergentmind.com/topics/discrete-traffic-state-encoding-dtse