---
title: 'If-T: Benchmark for Type Narrowing'
url: https://www.emergentmind.com/topics/if-t
type: topic
---

# If-T: Benchmark for Type Narrowing

If-T is a language-agnostic design benchmark for type narrowing in gradually-typed languages. It is intended to characterize how a type system validates correct code and rejects incorrect code when programs rely on runtime tests to refine the usage of incoming data, rather than on datatype-driven design. Unlike a traditional performance-focused benchmark, If-T measures narrowing behavior; unlike a test suite, it does not require full conformance, and deviations are acceptable when justified by design considerations such as compile-time performance. The benchmark is guided by the literature on type narrowing, the documentation of gradual languages such as TypeScript, and experiments with typechecker implementations [2508.03830].

## 1. Motivation and problem setting

Type narrowing is central to gradual typing because dynamic languages such as JavaScript, Python, and Racket commonly use runtime tests to distinguish among possible value shapes. In these settings, a static checker must be flow-sensitive: after a test such as `x is T`, the checker should refine the type environment differently on the true and false branches. Without such a mechanism, the system is either too permissive, missing real errors, or too conservative, forcing manual casts throughout existing codebases [2508.03830].

If-T was introduced to address the absence of a shared, language-agnostic specification of desirable narrowing behavior. Existing systems, including TypeScript, Flow, mypy, Pyright, and Typed Racket, embody different mixtures of ad-hoc rules and formal mechanisms. Prior formalizations such as set-theoretic types and full occurrence typing are powerful but significantly more complex than a standard type system, and the practical payoff of that extra complexity had remained unclear in the absence of a common benchmark.

A recurring misconception that If-T explicitly rejects is that narrowing quality can be assessed purely through implementation complexity or throughput. Its purpose is instead comparative and diagnostic: it isolates concrete narrowing behaviors and lets designers decide whether supporting each behavior is worthwhile. A second misconception is that a benchmark of this kind must define a strict standard. If-T is deliberately framed as a design benchmark, not a normative conformance suite.

## 2. Formal basis: flow-sensitive typing, set-theoretic refinement, and occurrence typing

If-T takes as its conceptual baseline a standard typing environment $\Gamma$ that maps variables to possibly union types. A test `x is T` splits control flow and refines the environment asymmetrically:

- on the then-branch, $\Gamma(x)$ is refined to $\Gamma(x) \cap T$;
- on the else-branch, $\Gamma(x)$ is refined to $\Gamma(x) \setminus T$.

This is the core notion of flow-sensitive typing, or type narrowing. The benchmark is not itself a calculus, but it is organized around two canonical formal approaches that make these refinements explicit [2508.03830].

The first is the set-theoretic view of types, in which types are interpreted as sets of values and admit Boolean-algebra operators:
$$
\tau ::= \text{Base} \mid \tau \cup \tau \mid \tau \cap \tau \mid \neg \tau \mid \tau_1 \setminus \tau_2
$$
with subtyping defined by semantic inclusion,
$$
\tau_1 \le \tau_2 \quad\text{iff}\quad \llbracket \tau_1 \rrbracket \subseteq \llbracket \tau_2 \rrbracket.
$$
Under this interpretation, narrowing is set intersection on the positive branch and set difference on the negative branch.

The second is occurrence typing, exemplified by Typed Racket. Here a typing judgment for an expression $e$ has the form
$$
\Gamma\;\vdash\;e:\tau\;;\;\Phi^+\;|\;\Phi^-\;;\;o
$$
where $\tau$ is the ordinary type of $e$, $\Phi^+$ and $\Phi^-$ are sets of propositions describing refinements in the true and false branches, and $o$ is an objective or path identifying the part of a value that the expression inspects. The associated environment updates are expressed through `erase`, which replaces $\Gamma(o)$ with either $\Gamma(o)\cap T$ or $\Gamma(o)\setminus T$ depending on the proposition.

The benchmark reproduces a canonical conditional typing rule of this style:
$$
\displaystyle
\frac{
   \Gamma\vdash e_1:\tau_1;\Phi^+;\Phi^-;o
   \quad
   \mathrm{erase}(\Gamma,\Phi^+)\vdash e_2:\tau_2
   \quad
   \mathrm{erase}(\Gamma,\Phi^-)\vdash e_3:\tau_3
}{
   \Gamma\vdash\bigl(\texttt{if }e_1\texttt{ then }e_2\texttt{ else }e_3\bigr):
   \tau_2\cup\tau_3
}
$$
and notes that logical connectives compose these propositions; for example, negation swaps $\Phi^+$ and $\Phi^-$. The significance of this formal backdrop is methodological: If-T uses minimal programs to ask which of these refinement capabilities an implementation actually realizes.

## 3. Benchmark structure and coverage

The core If-T suite contains 13 items. Each item has a “Success” example that should typecheck and a corresponding “Failure” example that should be rejected. The suite is divided into four categories that isolate distinct dimensions of narrowing behavior [2508.03830].

| Category | Items |
|---|---|
| Basic Narrowing | `positive`, `negative`, `connectives`, `nesting_body` |
| Compound Structures | `struct_fields`, `tuple_elements`, `tuple_length` |
| Advanced Control Flow | `alias`, `nesting_condition`, `merge_with_union` |
| Custom Predicates | `predicate_2way`, `predicate_1way`, `predicate_checked` |

The **basic** category tests whether systems refine on the true branch, on the false branch, through logical connectives such as `and`, `or`, and `not`, and through nested conditionals in the body. These programs represent the minimal expectations for a practical narrowing system. For example, `positive` asks whether a successful test such as `x is String` permits subsequent string-specific operations, while `negative` asks whether the complement information in the else-branch is propagated precisely enough to exclude non-members of the tested type.

The **compound-structure** category asks whether narrowing tracks paths into data structures. `struct_fields` tests whether field tests refine exactly that property. `tuple_elements` checks whether indexing and testing refine the selected element. `tuple_length` asks whether a test such as `Tuple.length(x) is 2` narrows between alternative tuple types.

The **advanced-control-flow** category examines whether implementations preserve and compose refinement information beyond direct syntax. `alias` asks whether let-bound tests can carry through as guards when the variable is immutable. `nesting_condition` asks whether an inner test embedded in an if/else expression composes with the outer branch condition. `merge_with_union` requires that when two branches assign different refined types to the same variable, the join be the precise union rather than `Top`.

The **custom-predicate** category addresses user-defined narrowing functions. `predicate_2way` concerns symmetric predicates that refine positively and negatively. `predicate_1way` concerns asymmetric predicates of the form `implies x is T`, which refine only when true. `predicate_checked` asks whether the body of a user-defined predicate is itself checked against its declared narrowing annotation.

## 4. Implementations and comparative findings

The benchmark was implemented for five typecheckers: TypeScript, Flow, Typed Racket, mypy, and Pyright. The authors translated the 26 mini-programs into these systems and summarized the outcomes in a pass/fail table in which a dot indicates that the Success example typechecks and the Failure example is rejected soundly, while a cross denotes either unsound acceptance or over-conservative rejection [2508.03830].

The reported results show a common baseline. All five engines pass the four basic items. All also pass `struct_fields` and `tuple_elements`. The paper further reports that only the three JavaScript-based systems pass `tuple_length`, while Typed Racket requires a manual `cdr`/`car` style check instead of `Tuple.length`.

The advanced-control-flow items expose sharper divergence. Flow and mypy do not support `alias`, because they ignore tests bound to variables. Only Typed Racket supports `nesting_condition`; the JavaScript and Python checkers treat `if (if e …):` as a parsing statement rather than as an expression, and the benchmark notes that supporting this behavior would require moving to an expression-style typing judgment such as occurrence typing. On `merge_with_union`, mypy is described as overly conservative, collapsing the join to `Top` in one test.

The custom-predicate items distinguish systems even more clearly. All accept two-way predicates, but only Flow and Typed Racket check them thoroughly. Only Typed Racket and Pyright accept the one-way form `implies x is T`; TypeScript mandates that predicates refine both ways. TypeScript, mypy, and Pyright do not verify that a predicate’s body actually meets its annotation, so unsound definitions slip through, whereas Typed Racket rejects them.

These findings matter because If-T is not measuring language popularity or implementation maturity. It is measuring which refinement behaviors each checker chooses to support, and therefore where each design sits in the precision–complexity design space.

## 5. Design trade-offs exposed by If-T

If-T organizes narrowing implementations into three broad design levels. The first is **ad-hoc narrowing**, exemplified by TypeScript, Flow, Pyright, and mypy. Its advantages are minimal complexity and scalability in large existing codebases. Its limitations, according to the benchmark discussion, include missed support for alias-based tests, nested-condition expressions, one-way narrowing, and predicate-body soundness checks. In such systems, developers often compensate with boilerplate or duplicated logic [2508.03830].

The second is **occurrence typing**, represented by Typed Racket. Its principal advantage is uniform support across the benchmark’s dimensions, including custom asymmetric predicates, nested tests, and precise branch merges. Its cost is implementation complexity: the typing judgment must be extended with propositions $\Phi^+$ and $\Phi^-$ and objectives, together with the associated metatheory. If-T presents this as a direct precision-for-complexity trade-off rather than as an unconditional improvement.

The third is **set-theoretic types**, which If-T treats as an open frontier rather than a mainstream deployed solution in JavaScript or Python checkers. Their attraction lies in semantic completeness and in the ability to express arbitrarily refined predicates. Their drawback is computational: the benchmark notes that such systems are known to blow up compile times even on small examples, citing minutes on a `flatten` function.

A central contribution of If-T is therefore comparative calibration. It does not claim that every implementation should realize every narrowing feature. Instead, it provides concrete examples through which language designers can decide whether the additional annotation burden, implementation effort, and compile-time cost of a more expressive narrowing mechanism are justified.

## 6. Significance, misconceptions, and future directions

If-T provides a baseline for future research on narrowing systems. For researchers, it offers criteria for categorizing designs through a shared collection of positive and negative examples. For language designers, it makes the payoff of typechecker complexity visible in concrete programs. Both the benchmark and its implementations are described as freely available online, and the project includes a datasheet template for documenting how a system approaches narrowing [2508.03830].

Its significance is partly terminological and methodological. The benchmark clarifies that type narrowing is not merely a convenience feature for dynamic-language ergonomics; it is a foundational mechanism for validating gradually-typed programs whose structure is driven by runtime tests rather than by closed algebraic datatypes. It also clarifies that “better” narrowing is not a one-dimensional notion. The relevant axes include precision, annotation burden, and performance, and different systems occupy different points in that design space.

The paper outlines several future extensions. These include mutation and concurrency, where side effects or multithreading may invalidate assumptions established by earlier tests; subtyping tests, such as the distinction between `isinstance` and `type(x) is T`; deeper path-sensitive struct refinements, such as nested field tests; explicit performance benchmarks for particular features such as `predicate_1way` and `nesting_condition`; and broader language coverage, including Hack, Luau, Pyre, Sorbet, Static Python, and Typed Clojure. These directions suggest that If-T is intended not as a finished taxonomy but as an extensible benchmarking framework for the ongoing design of gradual type systems.

Source: https://www.emergentmind.com/topics/if-t