Adaptive Traffic Control Systems (ATCS)

• Adaptive Traffic Control System (ATCS) is a dynamic traffic signal framework that adjusts timings based on real-time data to optimize flow, reduce delays, and improve safety.
• It integrates methodologies such as reinforcement learning, optimization, and digital twin simulations, formalizing problems via MDP/POMDP and constrained models.
• ATCS designs incorporate hybrid controllers and safety enhancements that balance efficiency, congestion, and emissions, with evaluations showing up to 52% delay reduction.

Adaptive Traffic Control System (ATCS) refers to the class of signal control methodologies that dynamically adjust signal timings in response to observed real-time traffic states, with the objective of optimizing network performance metrics such as delay, queue length, throughput, emissions, or safety. Unlike fixed-time or cycle-based approaches, ATCSs leverage data-driven, model-based, or optimization techniques to respond to spatiotemporal fluctuations in traffic demand, vehicle arrivals, and congestion patterns. This article surveys foundational principles, algorithmic structures, representative solutions, and evaluation methods for state-of-the-art ATCS, with emphasis on learning-based and hybrid controllers.

1. Mathematical Foundations and Control Formulation

Most contemporary ATCS frameworks formalize the traffic signal control problem as a Markov Decision Process (MDP), Partially Observable MDP (POMDP), or Constrained MDP (CMDP). In this model-driven context, the system state $s_t$ aggregates features such as per-approach queue lengths, accumulated waiting times, current signal phase indices, and—optionally—neighboring intersections’ states or demands. The action space $A$ may comprise discrete phase selections, green duration assignments, or hybrid (phase, duration) tuples. The environment’s stochastic transition function $P(s_{t+1}|s_t,a_t)$ captures traffic flow evolution given agent actions.

Reward definitions are typically negative-weighted sums of delay, queue length, vehicle stops, or safety violations (e.g., $$R(s_t,a_t)=-[\alpha \cdot \text{delay}(s_t) + \beta \cdot \|q_t\|_1]$$). In multi-objective ATCS, rewards are constructed as weighted sums across efficiency, safety, and emissions (e.g., $$R_t = w_{\mathrm{saf}} r^{\mathrm{saf}}_t + w_{\mathrm{eff}} r^{\mathrm{eff}}_t + w_{\mathrm{dec}} r^{\mathrm{dec}}_t$$) (Mirbakhsh et al., 2024).

Extensions to the MDP formalism include constraints on minimum/maximum green times, phase or green skips, and fairness or safety rules, leading to CMDP or Lagrangian-relaxed optimization targets (Satheesh et al., 30 Mar 2025). Transition models can be explicit (microsimulation, queue transmission) or learned via world models (latent RSSM in DreamerV3 (Li et al., 1 Nov 2025)). In POMDP contexts, agents operate on partial local observations and must leverage historical or neighbor-informed encodings—this motivates attention-based (Transformer) architectures for context extraction under partial observability (Wang et al., 2024).

2. Algorithmic Paradigms and Representative Models

ATCSs are stratified into broad classes: rule-based, optimization-based, and learning-based (model-free/model-based RL, multi-agent RL). Recent advances emphasize distributed deep RL, game-theoretic schemes, and digital-twin–integrated architectures.

• Rule-based and Optimization Methods: Classical algorithms include split/offset/cycle adjustment via queue balancing (SCATS, SCOOT), dynamic programming (OPAC, SURTRAC), or MILP-based horizon scheduling (Shams et al., 2023). Max-pressure controllers maximize instantaneous pressure differentials to improve throughput (Shams et al., 2023).
• Reinforcement Learning (RL): RL-enabled ATCSs span single-agent and multi-agent settings, employing DQN, Dueling Double DQN, PPO, or actor-critic variants (Wang et al., 2022, Mirbakhsh et al., 2024). State encodings range from scalar vectors (queue counts, phases), convolutionally encoded tensors, to attention-weighted pressure representations (Duan et al., 2024). Multi-agent RL formulations model intersections as agents in a stochastic game, exchanging neighbor policies or states to stabilize learning and coordinate phase allocation (Fazzini et al., 2021, Fazzini et al., 2021, Önür et al., 8 Dec 2025). MARL approaches often exploit centralized-training–decentralized-execution (CTDE) for sample efficiency and scalability (Satheesh et al., 30 Mar 2025).
• Hybrid and Constrained Learning: Economic analogies (eATSC) embed continuous compounding penalty functions for vehicle delays, guiding signal decisions via a two-agent Double Dueling DQN that separately optimizes interest rates and green durations (Jiang et al., 2022). Constrained MARL introduces Lagrange multipliers and cost critics to simultaneously optimize reward (throughput, waiting time) and conform to real-world constraints such as GreenTime, PhaseSkip, and GreenSkip (Satheesh et al., 30 Mar 2025). Multi-level RL structures tune low-level state feedback controllers at lower frequency, improving sample efficiency and resilience (Önür et al., 8 Dec 2025).
• Digital Twin and Parallel Simulation: ATCS in digital twin frameworks maintain real-time twins of the network, synchronizing measurement states (positions, delays) and conducting parallel evaluations of candidate policies using simulated demand forecasts (Dasgupta et al., 2021, Dasgupta et al., 2023). Algorithms such as DT-ATSC and DT1/DT2 use real-time or cumulative delay metrics to select phase assignments, optionally incorporating upstream link information for redistribution of delay and enhanced user equity.
• Advanced Architectures and Safety Enhancements: Attention-based pressure encodings, Bayesian critique-tune frameworks, and parameterized hybrid-actor algorithms increase RL policy reliability, interpretability, and robustness under uncertainty (Duan et al., 2024, Wang et al., 18 Mar 2025). Multi-objective Dueling DQNs allow explicit tradeoffs across efficiency, safety (e.g., time-to-collision incidents), and emissions (Mirbakhsh et al., 2024).

3. State, Observation, and Action Modeling

State and action representations critically define ATCS performance and scalability.

• State Encoding: Per-approach queue lengths, running and waiting vehicle counts, phase indices, and approach/direction one-hots are standard (Wang et al., 2022). Advanced encodings utilize pressure metrics (difference between upstream and downstream queues), attention-weighted multi-head features, and TDTSE/DTSE multi-lane grid encodings (Duan et al., 2024, Shams et al., 2023). Digital twins maintain per-vehicle stopped delay and upstream history for granularity and fairness (Dasgupta et al., 2021, Dasgupta et al., 2023). In single-agent regional ATCS, an adjacency-structured matrix of normalized splits and queue lengths encodes the regional state (Li et al., 1 Nov 2025).
• Actions: Control actions span from binary (keep/switch) to discrete enumerations over legal phase patterns (turning, straight, left/right), green-extension durations, and hybrid pairs (phase, green time) (Wang et al., 18 Mar 2025, Jiang et al., 2022). Some frameworks (PH-DDPG) output in parallel for all phases, enabling independent fine-tuning of green times for each phase (Wang et al., 18 Mar 2025). CMDP-based approaches restrict action sets to preclude illegal phase or green skips (Satheesh et al., 30 Mar 2025).
• Reward Structure: Per-decision penalties/rewards encapsulate queuing, travel delay, throughput, stops, emissions (SUMO pollutant models), and safety (e.g., time-to-collision, conflicts) (Mirbakhsh et al., 2024). Distributed MARL variants adjust spatial-discounted neighborhood rewards to encourage local-global equilibrium (Fazzini et al., 2021).

4. Coordination, Scalability, and Robustness

ATCS must address coordination both at the intersection level and network scale, as well as ensure resilience against failures, attacks, and non-stationary demand.

• Coordination Mechanisms: Multi-agent systems employ neighbor communication (policy “fingerprints”, value-message passing), hierarchical region aggregation, and system-wide synchronization (global reward, uniform actor) (Fazzini et al., 2021, Shams et al., 2023, Satheesh et al., 30 Mar 2025). Digital twins coordinate upstream and downstream intersections to mitigate spillback and starvation (Dasgupta et al., 2021, Dasgupta et al., 2023). Emergent coordination arises in phase selection under platoon arrivals or via weighted priorities for up/downstream flows (Shams et al., 2023).
• Scalability: Full-network single-agent RL models entail quadratic or linear scaling in state/action space but can leverage existing probe vehicle data and central optimization resources (Li et al., 1 Nov 2025). Hierarchical partitioning and region-based training address deployment in large urban environments (Satheesh et al., 30 Mar 2025). Hybrid architectures—RL for slow parameter adaptation atop fast state feedback controllers—decrease sample complexity and facilitate partial-failure resilience (Önür et al., 8 Dec 2025).
• Resilience and Security: ATCS are vulnerable to input falsification (Sybil and collusion attacks), necessitating application-layer mitigation—e.g., game-theoretic minimax per-lane weighting (Mallah et al., 2020), vehicle authentication, and anomaly detection (Qu et al., 2021). Bayesian critique-tune layers refine policy outputs, bypassing outlier recommendations (Duan et al., 2024).

5. Digital Twin Integration and Real-World Deployment

Digital-twin–enabled ATCS instantiate a bi-directional linkage between the physical system and a high-resolution cyber-replica. Real-time data streams (vehicles, signals, trajectory) synchronize the physical and digital twins, supporting per-vehicle delay tracking, candidate policy simulation under forecast demands, and user-level equity optimization (Dasgupta et al., 2021, Dasgupta et al., 2023).

These systems have demonstrated delay and fairness improvements (e.g., DT1/DT2 reductions in control delay up to 52% vs. density-based baseline), particularly by flattening the tail of the waiting-time distribution to improve perceived user satisfaction under variable demand regimes. Infrastructure requirements include per-vehicle identification, robust communication gateways, and sufficient computational resources for simulation-in-the-loop deployment.

6. Evaluation, Metrics, and Limitations

Empirical performance is assessed via microscopic simulation (SUMO, VISSIM, CityFlow, Aimsun), network-level and per-approach measures, and statistical comparison to fixed-time, actuated, or legacy adaptive baselines (Jiang et al., 2022, Dasgupta et al., 2021, Mirbakhsh et al., 2024). Principal metrics include:

• Average delay per vehicle or cycle
• Queue length statistics (mean, variance, maximal)
• Throughput and destination arrival rates (DAR)
• Cumulative emissions (e.g., CO₂, NOₓ, PMₓ)
• Traffic conflicts (safety, e.g., $\mathrm{TTC}<3$ s incidents)
• Level of Service (LOS) per Highway Capacity Manual thresholds
• Constraint violation frequencies (for CMDP methods)

Recognized limitations include sample inefficiency and non-generalizability of deep RL under new or extreme demand regimes (Wang et al., 2022), partial observability and reward sparsity, scalability bottlenecks for centralized critics, and requirement for 100% CV penetration in some digital-twin evaluations. Transfer learning, robust regularization, and online/continual adaptation remain major directions for next-stage systems.

7. Best Practices and Future Research Directions

• State design should leverage rich sensor data (queue, running/waiting times, probe vehicle travel times) and structural representations suitable for the solution’s coordination paradigm.
• Incorporate hybrid controllers (RL tuning state feedback) for rapid adaptation and real-time resilience, especially in large heterogeneous networks (Önür et al., 8 Dec 2025).
• Enforce real-world constraints via Lagrangian or penalty-based formulations, guaranteeing deployability and compliance with legal and operational norms (Satheesh et al., 30 Mar 2025).
• Utilize digital-twin architectures for simulation-driven validation, user satisfaction analysis, and equitable control policy design (Dasgupta et al., 2021, Dasgupta et al., 2023).
• Advance adversarial and anomaly-resilient system models, integrating authentication and adaptive policy retraining where feasible (Mallah et al., 2020, Qu et al., 2021).
• Embrace multi-objective optimization to balance network efficiency, safety, and environmental objectives in the face of demand surges and heterogeneous fleets (Mirbakhsh et al., 2024).
• Explore hybrid RL/MPC and GNN-based transfer frameworks to facilitate generalization across cities, network scales, and traffic regimes (Wang et al., 2022).

This convergence of data-driven, optimization-based, and adversarially robust ATCS reveals a complex, multidimensional design space, with next-generation deployments expected to combine digital twin infrastructure, deep coordination architectures, explicit constraint handling, and robust, explainable control policy adaptation.