TC-Bench: Timed Verification Benchmark

• TC-Bench is a benchmark library for parametric timed model checking featuring 34 benchmarks, 80 models, and 122 properties for broad evaluation.
• It categorizes examples into academic, industrial, and unsolvable sets to support fair comparisons and scalability studies.
• The library utilizes parametric timed automata and L/U-PTAs, targeting tools like IMITATOR to stimulate further research through open challenges.

TC-Bench is a benchmark library for parametric timed model checking, created to provide a public, stable, and varied collection of benchmark models and properties for fair evaluation of tools and verification techniques in real-time systems (Étienne, 2018). It was introduced in response to a methodological gap: although many parametric timed verification methods had been proposed, the field lacked a common benchmark base, and tools were often tested on ad hoc examples or on case studies informally shared between papers. In its first version, the library contains 34 benchmarks, 80 different models, and 122 properties, spanning academic benchmarks, industrial case studies, and examples unsolvable using existing techniques (Étienne, 2018).

1. Scope and rationale

TC-Bench is designed for parametric timed model checking, a verification setting in which timing constants are not fully fixed, or are affected by uncertainty or imprecision. The motivating use cases are real-time systems with hard timing constraints and concurrency, including industrial contexts in which periods or delays are known only approximately. The paper links this motivation to the growing importance of parametric verification in real-time systems and to the availability of multiple tools and techniques such as IMITATOR, HyTech, Romeo, PSyHCoS, Symrob, PHAVer, and SpaceEx (Étienne, 2018).

The primary role of the library is methodological. It is intended to support fair comparison of tools by replacing isolated, one-off examples with a common reference collection. The benchmark base is therefore presented not merely as an archive of models, but as an infrastructure for comparative evaluation. The paper explicitly argues that performance comparisons had been unreliable because models and properties varied from paper to paper, and TC-Bench is meant to standardize that experimental substrate.

The library is also designed to stimulate further research by including examples that current state-of-the-art methods cannot solve. This is a notable aspect of its philosophy: solved instances are not the only target. Hard and unsolved instances are included as open challenges, including cases that may be solvable by hand but not by existing automated methods.

2. Formal basis in parametric timed automata

TC-Bench is built around parametric timed automata (PTAs), with IMITATOR as the main target format (Étienne, 2018). PTAs extend timed automata by allowing parameters in guards and invariants, so that models can represent unknown timing constants or timing uncertainty. This makes them suitable for verification tasks in which exact constants are unavailable during design or need to be synthesized from correctness requirements.

The paper also situates the library within the broader PTA family by discussing the subclass of L/U-PTAs. In this subclass, parameters are divided into lower-bound and upper-bound parameters. Lower-bound parameters appear only in constraints such as $p \leq x$ or $p < x$, while upper-bound parameters are used oppositely. This distinction matters because the library includes both general PTAs and benchmarks that either satisfy or violate the L/U restriction, allowing tool behavior to be studied across structurally different subclasses.

The examples used in the paper illustrate the symbolic timing behavior that the library aims to capture. One toy PTA has a reachability set exactly for parameters of the form $p = 1/n$, and another toy PTA has a reachability condition over all positive parameters. These examples are significant because they exemplify cases in which the parameter space has nontrivial symbolic structure, including patterns that are difficult for existing automated procedures.

3. Benchmark categories and application domains

The library is organized into three main categories: academic benchmarks, industrial case studies, and examples known to be unsolvable by current state-of-the-art techniques (Étienne, 2018). Academic benchmarks include standard examples studied in the literature, such as Fischer’s mutual exclusion protocol. Industrial case studies arise from concrete industrial problems or collaborations. The unsolvable category includes models for which no existing automated tool can compute the result, even though the result may be known mathematically.

A fourth category is used in the online library but omitted from the paper’s main discussion: education benchmarks. These are simple teaching examples, such as coffee machines, and are described as not especially relevant for performance evaluation. Their inclusion in the online resource indicates that the library also functions as a general repository, but the paper distinguishes them from the core evaluation-oriented material.

The benchmark set spans a broad range of application domains. These include hardware asynchronous circuits, communication protocols, mutual exclusion protocols, real-time systems and schedulability problems, parametric timed pattern matching (PTPM), train-gate controller models, automation systems, and production-consumption systems. The breadth of this domain coverage is central to TC-Bench’s role as a comparative benchmark: it is intended to prevent conclusions from being driven by a narrow family of examples.

Representative benchmark names reported in the paper include And-Or, CSMA/CD, Fischer-AHV93, Fischer-HRSV02, Flip-flop, idle-time-sched, Jobshop, NP-FPS, SSLAF, ProdCons, train-AHV93, WFAS, and industrial cases such as accel, gear, FMTV, RCP, SIMOP, and SPSMALL, as well as toy cases toy:n and toy:1/n (Étienne, 2018). The inclusion of both classical academic models and industrial instances underscores the library’s dual use as a research benchmark and as a repository of practically motivated verification problems.

4. Structural metadata and property taxonomy

TC-Bench classifies benchmarks not only by domain, but also by structural and semantic features (Étienne, 2018). The recorded structural characteristics include the number of automata, clocks, parameters, and discrete variables; whether the benchmark is scalable; whether it contains shared rational-valued variables; whether it uses stopwatches; whether it contains location invariants; and whether it satisfies the L/U restriction. This metadata is critical for tool comparison because it allows experiments to be stratified by model structure rather than reported as undifferentiated aggregates.

The library also classifies benchmarks by the property being verified. Four major property types are identified. Reachability or safety asks for parameter valuations that make a given state reachable or avoidable. Optimal reachability seeks valuations that optimize time or a parameter. Unavoidability asks whether every run must eventually reach a target state. Robustness concerns preservation of discrete behavior under parameter perturbations. In addition, the library includes PTPM cases, where a temporal pattern is checked on a log for some parameter values and some log segment, and a small number of benchmarks with miscellaneous ad hoc properties such as liveness or observer-based specifications.

This dual organization by structure and property type is one of the library’s most important design choices. It turns the collection into more than a list of files: the benchmark instances can be selected according to model features, verification objective, or both. A plausible implication is that TC-Bench supports finer-grained empirical claims, such as whether a technique performs well specifically on scalable L/U-PTAs with invariants, or on robustness problems with discrete variables, rather than only on a heterogeneous overall average.

5. Evaluation role, scalability, and hard instances

A central purpose of TC-Bench is to support fair comparison of tools, and the paper emphasizes that this requires more than merely storing models (Étienne, 2018). For that reason, the library records multiple characteristics of each benchmark and, where applicable, provides multiple models and multiple properties for the same case study. Benchmark families may include scaled instances. The paper cites Flip-flop with models of 2, 5, and 12 parameters, and Fischer with variants corresponding to different numbers of processes. This allows scalability studies to be carried out on coherent families of related models.

The paper also reports execution times for selected benchmarks on an Intel i7-7500U CPU @ 2.70GHz with 8 GiB RAM running Linux Mint 18. Some instances solve quickly, such as train-AHV93 at 0.01s and Fischer-AHV93 at 0.04s, while others are significantly harder, such as Flip-flop:12 at 23.07s, NP-FPS-3tasks:100-2 at 65.23s, FMTV:1A3 at 87.39s, and BRP at 248.35s. Entries marked in red correspond to time-outs after 300 seconds.

The treatment of difficulty in TC-Bench is more nuanced than a simple solved/unsolved dichotomy. The paper distinguishes between time-outs that are merely difficult and cases that are unsolvable by existing tools because the required algorithm has not been implemented. It also identifies benchmarks that are time-out but still human-solvable, with the Fischer family and the toy PTAs given as examples. This distinction is methodologically important because it exposes different failure modes: computational difficulty, missing algorithmic support, and gaps between symbolic mathematics and automation.

Some benchmarks yield only partial results. The paper notes that IMITATOR’s reachability-synthesis procedure can sometimes return partial information if interrupted early, and that in some industrial cases the original publication provides punctual valuations even when symbolic synthesis is unavailable. ProdCons is given as an example: IMITATOR cannot synthesize a symbolic constraint, but the original work gives some concrete valuations. This makes TC-Bench relevant not only to complete synthesis success, but also to intermediate forms of verification output.

6. Distribution, extensibility, and nomenclatural ambiguity

TC-Bench is a web-accessible collection available at https://www.imitator.fr/library.html, and all benchmarks are distributed in the IMITATOR input format under the GNU General Public License (Étienne, 2018). If a benchmark originally came from another model checker such as HyTech or Uppaal, the library may also include that tool’s native syntax. The paper states a future goal of adding translations, especially toward Uppaal timed automata, while also noting that some information would be lost because Uppaal lacks parameters and only supports stopwatches in a limited way.

The library is explicitly described as open, collaborative, and extensible. It is not treated as a fixed one-off list; the authors state that it will be enriched with future benchmarks and that a versioning system is planned so additions and modifications can be tracked. At the same time, the paper acknowledges practical limitations: the library is centered on the IMITATOR ecosystem, not every benchmark comes with fully automated solutions, and translation to other formalisms is constrained by expressiveness mismatches.

A common source of confusion is the non-uniqueness of the name. Later arXiv papers use the closely similar labels “TC-Bench” or “TCBench” for unrelated benchmarks in other domains, including temporal compositionality in text-to-video and image-to-video generation (Feng et al., 2024), tropical cyclone track and intensity forecasting (Gomez et al., 30 Jan 2026), and scientific alignment analysis for vision foundation models in tropical cyclone research (Yao et al., 23 May 2026). This suggests that, within arXiv discourse, “TC-Bench” is not a unique identifier. In the context of formal verification, however, the term specifically denotes the benchmark library for parametric timed model checking introduced in 2018 (Étienne, 2018).