---
title: Timed-LDGBA for Timed Control Synthesis
url: https://www.emergentmind.com/topics/timed-limit-deterministic-generalized-buchi-automata-timed-ldgba
type: topic
---

# Timed-LDGBA for Timed Control Synthesis

A Timed Limit-Deterministic Generalized Büchi Automaton (Timed-LDGBA) is a formal automaton model uniquely suited to represent time-bounded temporal logic specifications for control synthesis over stochastic environments. This construction combines the structural restrictions of limit-deterministic Büchi automata (LDBA) with real-valued clocks, enabling expressivity for specifying and monitoring sequences of events constrained by explicit time intervals. Timed-LDGBA are instrumental in synchronizing temporal logic specifications with Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs), facilitating reinforcement learning in environments with strict time-bounded requirements [2601.00087].

## 1. Formal Definition and Structure

A Timed-LDGBA is a tuple $A = (Q, \Sigma, C, E, \text{Inv}, q_0, \mathcal{F})$ comprising:

- $Q$: finite set of locations partitioned as $Q = Q_N \cup Q_D$, where
  - $Q_N$ (nondeterministic part): admits only $\epsilon$-transitions and contains no accepting locations.
  - $Q_D$ (deterministic part): contains all accepting locations and deterministic transitions only.
- $\Sigma$: finite alphabet ($2^\Pi$ for atomic propositions $\Pi$).
- $C$: finite set of real-valued clocks.
- $E \subseteq Q \times (\Sigma \cup \{\epsilon\}) \times \mathbb{B}(C) \times 2^C \times Q$: set of edges. Each edge is $(q, a, g, r, q')$ where:
  - $a \in \Sigma \cup \{\epsilon\}$: transition label.
  - $g \in \mathbb{B}(C)$: clock guard as conjunctions $x \preceq c$ or $x \succeq c$.
  - $r \subseteq C$: clocks to reset upon transition.
- $\text{Inv}: Q \to \mathbb{B}(C)$: invariant conditions at each location (conjunctions of constraints).
- $q_0 \in Q_N$: initial location.
- $\mathcal{F} = \{F_1, \ldots, F_k\}$: accepting sets, each $F_i \subseteq Q_D$.

Limit-determinism is enforced such that all nondeterministic branching ($\epsilon$-moves) occurs in $Q_N$, which is acyclic and never revisited once entered $Q_D$. All acceptance monitoring in $Q_D$ is strictly deterministic.

## 2. Generalized Büchi Acceptance Condition

Timed-LDGBA utilize a generalized Büchi acceptance mechanism to formalize satisfaction of temporal goals. For an infinite run $\sigma = (q_0, v_0) \xrightarrow{t_0, a_0} (q_1, v_1) \xrightarrow{t_1, a_1} (q_2, v_2) \ldots$ reading a timed word $(a_i, t_i)$, acceptance requires:

$$
\text{Run is accepted} \iff \forall i \in \{1,\dots,k\},\ \{ n \mid q_n \in F_i \} \text{ is infinite}
$$

This ensures every generalized Büchi set $F_i$ in $\mathcal{F}$ is visited infinitely often by the corresponding sequence of automaton states, encoding persistent timed obligations tied to the original temporal logic specification.

## 3. Clocks, Guards, Invariants, and Resets

Clocks provide the quantitative dimension necessary for time-bounded semantics:

- **Clock set $C$**: $C = \{x_1, x_2, \ldots, x_m\}$, each $x_j$ real-valued.
- **Valuation $v: C \to \mathbb{R}_0^+$**: tracks elapsed time since last reset for each clock.
- **Guards $g$**: conjunctions of atomic constraints, $x \preceq c$ or $x \succeq c$ ($c \in \mathbb{N}$).
- **Invariants $\text{Inv}(q)$**: conjunctions $x \preceq c$ constraining the allowable time in location $q$ as time elapses.
- **Resets $r \subseteq C$**: upon taking an edge, clocks in $r$ are set to zero; the valuation updates as $v' = v[r := 0]$.
- **Time elapse**: at any location $q$, duration $d \geq 0$ is permissible as long as $\text{Inv}(q)$ is true at each intermediate valuation $v + \delta$, $\delta \in [0, d]$.

## 4. Translation from MITL to Timed-LDGBA

Metric Interval Temporal Logic (MITL) formulas $\,\varphi$ are systematically compiled into Timed-LDGBAs:

1. **Negation Normal Form & Interval Normalization**: MITL formulas are normalized for transition monitoring.
2. **Monitor Construction**: For each subformula $\psi$, a "timed monitor" automaton $A_\psi$ is built, typically with a single clock:
   - *Example for $F_{[a,b]} \pi$*: The automaton contains:
     - Initial state $q_0$ with invariant $x \leq b$.
     - On letter $\pi$ and $a \leq x \leq b$, reset $x$ and transition to $q_{accept}$.
     - Sink state $q_{sink}$ if $x > b$ before $\pi$.
     - Accepting set $F = \{q_{accept}\}$.
3. **Synchronous Product**: Monitors $A_\psi$ are composed in product, tracking all clocks simultaneously.
4. **Limit-Determinization**: All nondeterminism is grouped into initial states $Q_N$, then collapsed into deterministic $Q_D$ with acceptance sets corresponding to fulfilled obligations.
5. **Pruning Unreachable States**: Ensures model compactness.

*Construction Example*:

For $\varphi = F_{[1,3]} a$:
- $C = \{x\}$.
- $Q = \{q_0, q_{accept}, q_{sink}\}$.
- $\text{Inv}(q_0) = x \leq 3$; $\text{Inv}(q_{accept}) = \text{true}$; $\text{Inv}(q_{sink}) = \text{true}$.
- Edges include:
  - $q_0 \xrightarrow{a, 1 \leq x \leq 3, \{x\}} q_{accept}$
  - $q_0 \xrightarrow{any, x > 3, \varnothing} q_{sink}$
- $q_0 \in Q_N$, $q_{accept}, q_{sink} \in Q_D$.
- $\mathcal{F} = \{\{q_{accept}\}\}$.

## 5. Synchronization with MDPs and POMDPs

Timed-LDGBA are synchronized with stochastic environment models to facilitate policy synthesis:

- **MDP**: $M = (S, A, T, s_0, R, \Pi, L)$.
- **POMDP**: $M = (S, A, T, s_0, R, O, \Omega, \Pi, L)$.

A product timed model $M \times A_\varphi$ is constructed:

- **States**: $S^\times = S \times Q \times V$, where $V$ is the (discretized) space of clock valuations.
- **Actions**: $A^\times = A \cup \{\epsilon\}$.
- **Transitions**:
  - $(s, q, v) \xrightarrow{a} (s', q', v')$ where $s \xrightarrow{a} s'$ via $T$, and edge $(q, L(s'), g, r, q')$ is enabled by $v+1$.
  - If no edge is enabled, transition to a global sink state.
  - For $\epsilon$ in $Q_N$: $s' = s$, $a = \epsilon$, and $T^\times = 1$ for the chosen $\epsilon$-move.
- **Reward**: $R^\times(s, q, v) = R(s)$ only upon entering an accepting $q' \in F_i$; otherwise zero.
- **Observations (POMDP)**: $\Omega^\times((s, q, v), a, o) = \Omega(s', a, o)$ if the automaton edge passes; else $0$.
- **Acceptance**: A path is accepting iff its $Q$-component visits each $F_i$ infinitely often.

Crucially, the automaton state $q$ and clock valuation $v$ are perfectly tracked and can augment the input to Q-learning or belief trackers; in POMDPs, these quantities remain fully observable, while the base state $s$ is inferred by belief $b_t$.

## 6. Application in Reinforcement Learning under Timed Constraints

MITL specifications are offline-compiled into Timed-LDGBA and synchronized online with MDP/POMDP models:

- The reward structure enforces temporal correctness via positive reward on accepting set entry, optionally combined with performance objectives.
- Standard RL algorithms (Q-learning, DQN) operate on the product model, learning policies to satisfy all time-bounded constraints or maximizing acceptance probability under stochasticity.
- Evaluations in grid-world and robotics scenarios demonstrate scalability, robustness to partial observability, and faithful satisfaction of MITL constraints in learned policies [2601.00087].

This framework enables reliable policy synthesis in dynamic, uncertain environments where temporal obligations are explicit and time-critical.

Source: https://www.emergentmind.com/topics/timed-limit-deterministic-generalized-buchi-automata-timed-ldgba