Traffic Sequence Charts

• Traffic Sequence Charts are formal visual languages that encode temporal and spatial traffic requirements, supporting both automated driving scenario design and real-world traffic analysis.
• They integrate structured time–space diagrams with spatial predicates and temporal logic, enabling consistency checks through methods like SMT-based bounded model checking.
• Their empirical application leverages video data and photogrammetric techniques to generate detailed trajectory plots, aiding infrastructure design and safety validation.

Traffic Sequence Charts (TSCs) are a formally defined visual language for representing temporal and spatial requirements in traffic analysis and automated driving. As formal constructs, TSCs support both scenario-based systems engineering for highly automated vehicles and the empirical analysis of real-world traffic by providing structured diagrams—time-space charts—that encode vehicle trajectories, spatial relations, and logical properties. Recent research delineates two principal axes: (1) the automated construction of time-space diagrams from video sequence data, enabling detailed traffic flow analysis, and (2) the formal modeling and automated consistency analysis of scenario-based requirements for automated driving using TSCs and satisfiability modulo theories (SMT)–based verification (Rastogi et al., 2023, Becker, 21 Aug 2024).

1. Formal Structure and Semantics of Traffic Sequence Charts

A TSC is underpinned by a world model ℳ, which specifies object types (such as Car, Lane) with attributes (positions, velocities, etc.), along with invariants and, where necessary, differential equations governing continuous dynamics. Each symbol present in a TSC is linked through a symbol dictionary to an object type τ, with specified anchor points facilitating the expression of spatial constraints.

Trajectories in TSCs are formalized as mappings $$\pi : \mathbb{R}_{\ge0} \to \mathcal{U}^{\mathcal{A}(O)}$$, where $\mathcal{U}$ is the universe of attribute values, and $\mathcal{A}(O)$ is the set of all attributes of the participating objects $O$. The spatial view ("sv") is a diagram of symbols encapsulating spatial configurations and predicates. Its semantics at time $t$ is a first-order formula $\Phi_{sv}$ in linear real arithmetic and uninterpreted functions evaluated over $\pi(t)(o.a)$.

The syntactic structure of TSCs is compositional. The grammar for basic charts is:

• invariant node $\langle sv \rangle$
• empty invariant $\top$
• sequence ($A;B$)
• choice ($A \oplus B$)
• concurrency ($A \parallel B$)
• time pins and hourglass duration constraints ($p_1, ..., p_n$, $\langle d \mid \Psi(d)\rangle$)

The semantics are defined inductively over intervals $[b, e]$, with time pins $t_p$:

• Invariant node: $$(\pi, [b, e], \{t_p\}) \models \langle sv \rangle \iff b < e$$ and $\forall t \in [b, e): \Phi_{sv}$ holds.
• Sequence: $A;B$ holds on $[b, e]$ if $\exists m \in [b, e]$ such that $A$ holds on $[b, m]$ and $B$ on $[m, e]$.
• Choice: $A \oplus B$ holds if either holds on $[b, e]$.
• Concurrency: $A \parallel B$ holds if both hold on $[b, e]$.
• Duration constraints are encoded via predicates over $e-b$.

A requirement-level TSC is a triple $(H, F, C)$, where the "history" $H$ and "future" $F$ (pre-chart) define the context, and $C$ (consequence, post-chart) stipulates the required property. Satisfaction of $(H, F, C)$ by $\pi$ requires that for all $b \le m \le e$, if $H$ holds on $[b, m]$ and $F$ on $[m, e]$, there exists a pin-assignment such that $C$ holds on $[m, e]$ (Becker, 21 Aug 2024).

2. Automated Construction of Empirical Time–Space Diagrams

One primary application of TSCs in traffic engineering is the construction of empirical time–space diagrams from roadside or street-view video, supporting the analysis of traffic flow and macroscopic variables. The pipeline documented by Rastogi & Björkman (Rastogi et al., 2023) demonstrates a complete method:

• Data acquisition: A monocular RGB camera is mounted on a probe vehicle, synchronized with GNSS/INS for exact timestamp and spatial position.
• Calibration: Camera intrinsic parameters (e.g., $F_{px}$, $I_{px}$, $S_{px}$) and extrinsic transformation relative to the vehicle are taken from established benchmarks (KITTI); full intrinsic matrix $K$ and distortion coefficients required for precise reproduction.
• Detection and tracking: Frames are processed using YOLOv5m, retrained on KITTI 2D data with reported “car” precision 0.926, recall 0.933, mAP[.5:.95] = 0.789. Detected objects are tracked across frames via StrongSORT, using OSNet appearance descriptors and two-step Hungarian matching, achieving HOTA = 74.64 (DetA=76.90, AssA=72.72, LocA=89.22).
• Photogrammetric distance estimation: Distance $D^i_m$ to each detected object $i$ is estimated via similar-triangle geometry:

$$D^i_m = \frac{F_{px} \cdot H^{i,Real}_m \cdot I_{px}}{H^i_{px} \cdot S_{px}}$$

where $H^{i,Real}_m$ is the assumed real-world object height (e.g., 1.5 m for cars), $H^i_{px}$ the detected bounding-box pixel height.

• World-coordinate association: Along-link position at time $t$: $d^t_i = d^t_{\rm probe} + D^i_m(t)$, with $d^t_{\rm probe}$ from geodesic calculation.
• Diagram creation: Per-track sequences $\{(t, d^t_i)\}_{t=t_0}^{t_{\rm end}}$ are plotted as families of $s_i(t) = d^t_i$ curves, yielding the time–space chart.

Error analysis shows trajectory RMSE vs. ground truth mean 2.97 m (σ=1.91 m), with dominating sources being bounding-box jitter, depth bias, and detection errors.

3. Modeling, Expressiveness, and Requirement Patterns

TSCs enable formalization of a wide range of scenario-based requirements for highly automated vehicles (HAVs). Standard patterns include:

• Existential TSC: Encodes existence of a property by a single basic chart $A$.
• Response TSC: Follows the “whenever $H$ then $C$” idiom, formulated as $(H, F, C)$, where occurrence of $H$ followed by $F$ imposes $C$.
• Complex temporal-spatial specification: Achieved by nesting sequence, choice, concurrency, and time constructs.

For example, the “No passing on the right side” rule is formalized with a pre-chart requiring a car $I$ in the left lane and another car $J$ overtaking to the right; the consequence prohibits such spatial view in parallel to the future.

The modeling leverages spatial predicates, temporal logic, and explicit quantification over trajectories:

$$\pi: \mathbb{R}_{\ge0} \rightarrow \mathcal{U}^{\mathcal{A}(O)}$$

$$Sat_\mathcal{M}((H,F,C)) \iff \exists \pi,\, \exists b < e,\, (\pi,[b,e],\text{pins}) \models (H,F,C)$$

This expressiveness enables encoding of constraints on relative position, velocity, and more intricate traffic interactions (Becker, 21 Aug 2024).

4. Automated Consistency Analysis via SMT and Bounded Model Checking

Consistency analysis addresses the question: “Does there exist at least one possible execution (trajectory $\pi$) such that all specified TSC requirements can be satisfied simultaneously?” The approach employs semi-decision methods:

• SMT-based bounded model checking (BMC): For each chart, generate a BMC instance represented by a triple $(I, T, F)$ over state vectors, corresponding to chart start, allowed transitions, and acceptance.
• Two-sided approximation: The “necessary” check (CheckSat_N) uses a discrete-time abstraction, only verifying invariants at step boundaries with continuity constraints (e.g., $|x' - x| \leq M\Delta$). The “sufficient” check (CheckSat_S) introduces Bézier spline parameterizations of vehicle trajectories, enforces dynamics (single-track model, bounded curvature and lateral acceleration), and approximates spatial predicates via constraints at spline control points.
• Conflict identification: Minimal conflicting sets are identified if there exist $S$ and $i \in S$ such that:
  • $$CheckSat_S(\Gamma_1^{i,S\setminus\{i\}}) = \mathrm{sat}$$ (premises are feasible)
  • $$CheckSat_N(\Gamma_2^{i,S\setminus\{i\}}) = \mathrm{unsat}$$ (premises plus consequences are infeasible)

The empirical evaluation on nine highway-rule TSCs (adapted from LTL formalizations of German autobahn regulations) showed that all conflicts detected were of minimal subset size two (e.g., "Keep in right-most lane" vs. "No passing on the right side" are mutually exclusive). The checking process achieved tractability: 467 SMT calls sufficed (out of a worst-case 2304), total runtime 80 s on commodity hardware, with SMT problems limited to dozens of variables and linear constraints (Becker, 21 Aug 2024).

5. Applications, Implications, and Limitations

TSCs serve two complementary domains:

• Empirical traffic flow analysis: Automatic construction of time–space diagrams offers detailed traffic pattern insights for infrastructure design and management, with advantages in spatial and temporal coverage over traditional fixed-sensor methods (Rastogi et al., 2023). The method demonstrates practicality for generation of accurate trajectory plots, though improvements are possible via better detector calibration, learned depth estimation, and outlier rejection.
• Scenario-based HAV development: Formal TSCs provide a bridge from human-friendly graphical requirements to machine-verifiable semantic contracts. Automated consistency analysis provides early detection of requirement conflicts prior to costly simulation or deployment.

The integration of Bézier-based witness generation enables the instantiation of virtual test scenarios (e.g., OpenSCENARIO, OpenDRIVE), runtime compliance monitoring, and criticality analysis.

Limitations exist in current methods:

• The necessary and sufficient approximations are incomplete; unsolved cases may remain undecided.
• Some aspects (e.g., negation, quantification, complex dynamics) require further extension for full expressivity and completeness.
• High-fidelity world calibration and robust data association remain engineering bottlenecks for empirical chart generation.

6. Future Research Directions

Research trajectories for TSCs include:

• Extension of TSC languages to support richer logical constructs (negation, quantifiers).
• Incorporation of hybrid-automaton reachability algorithms for stronger completeness in consistency analysis.
• Domain-specific modeling libraries and specification guidelines to reduce user modeling errors.
• Deeper integration with safety argumentation processes (e.g., ISO 26262, SOTIF).
• Scalability enhancements to address combinatorics as rule set sizes grow, and adaptation to multi-lane and curved road geometries via coordinate transformations (e.g., Lanelet maps).
• Enhanced empirical pipelines with multi-sensor fusion, 3D mapping, bundle adjustment, and trajectory smoothing for improved accuracy in time–space charting (Rastogi et al., 2023).

A plausible implication is that the formalization and verification capabilities of TSCs will be increasingly critical as automated vehicle requirements shift toward highly scenario-oriented, regulation-compliant, and safety-critical regimes. The synergy of data-driven empirical charting and formal specification/analysis positions TSCs as central artifacts in the next generation of traffic systems engineering (Rastogi et al., 2023, Becker, 21 Aug 2024).