ImpRAT: Imperative Relational Algebra with Types

• The paper presents ImpRAT as the formal algebraic core for TQL, using imperative control flow and dynamic type tests to model discovery queries.
• It defines an explicit syntax with transformation and constraint clauses, providing a small-step operational semantics for relational operations.
• ImpRAT bridges classical relational algebra with typed, implementation-oriented features, offering practical insights for designing domain-specific query languages.

Searching arXiv for the ImpRAT/TQL paper and the related papers cited in the supplied data. (Kang et al., 11 Aug 2025, Fatemi et al., 2021) Searching arXiv by identifier and title to ground the article in the cited papers. Imperative Relational Algebra with Types (ImpRAT) is the formal algebraic core for TQL, a domain-specific language for general-purpose data discovery. It is introduced to provide an implementation-oriented operational semantics for a setting in which discovery queries combine transformations of datasets with constraints on the resulting data. In the formulation given for TQL, ImpRAT turns that idea into a language with explicit syntax, dynamic type tests, imperative control flow, and small-step evaluation. The paper presents TQL as a surface syntax representation of a discovery problem encoded in ImpRAT, with meaning determined by a semantic translation function $\llbracket - \rrbracket$ from TQL programs into ImpRAT programs (Kang et al., 11 Aug 2025).

1. Definition and role in data discovery

ImpRAT arises in a paper that frames data discovery as a search-heavy problem in machine learning and data science. The stated motivation is that efficient and effective data discovery is critical for many modern applications, but a major bottleneck to a general-purpose discovery tool is the absence of an expressive formal language, and corresponding implementation, for characterizing and solving generic discovery queries. The paper addresses that bottleneck through TQL and characterizes the core language through ImpRAT (Kang et al., 11 Aug 2025).

A central organizing distinction is between transformation clauses and constraint clauses. Transformation clauses specify how raw data is modified, whereas constraint clauses specify what properties the resulting data must satisfy. The paper connects this split to programming-language ideas, especially the Curry–Howard perspective that “transformations” behave like terms or operations and “constraints” behave like types. In that sense, ImpRAT is not merely a variant of relational algebra; it is a formalism in which data-discovery queries are represented as programs that both compute candidate datasets and check whether those datasets satisfy designated constraints (Kang et al., 11 Aug 2025).

The paper also states that the earlier set-theoretic model for TQL, Type-Collected Relational Algebra (TCRA), was useful for intuition and computability, but was not yet a convenient basis for an implementation-oriented operational semantics. ImpRAT is therefore presented as the step that gives the language explicit syntax, dynamic type tests, imperative control flow, and small-step evaluation. The paper further claims that ImpRAT is essentially equivalent to the earlier set-theoretic model, while being better suited to implementation and operational semantics (Kang et al., 11 Aug 2025).

2. Formal language and semantic domains

The formal syntax of ImpRAT is given by the following BNF:

$$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$

In this presentation, relational expressions $R$ include dataset literals $D$, variables $x$, union, difference, product, the bottom value $\bot$ for failure, projection, selection, renaming, and a test operator $R\{T_0,\ldots,T_n\}$. Types $T$ are lightweight predicates: an attribute name $a$, an existential test $\exists \varphi$, or a universal test $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$0. Statements $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$1 include $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$2, $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$3, assignment, free-variable declaration, sequencing, and return (Kang et al., 11 Aug 2025).

A discovery problem is modeled as a tuple $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$4, where $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$5 is an ImpRAT program and $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$6 is the repository of all datasets. A program is viewed as a function

$$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$7

where $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$8 is the closure of $$\begin{aligned} R &::= D \mid x \mid R_0 \cup R_1 \mid R_0 - R_1 \mid R_0 \times R_1 \mid \bot \ &\quad \mid \Pi_{a_1,\ldots,a_n}(R) \mid \sigma_\varphi(R) \mid \rho_{a/b}(R) \mid R\{T_0,\ldots,T_n\} \ T &::= a \mid \exists \varphi \mid \forall \varphi \ S &::= \top \mid \bot \mid x := R \mid !x \mid S_0;S_1 \mid \Delta R \end{aligned}$$9 under relational-algebra operations and $R$0 means failure or invalid evaluation. The valid inputs are defined by

$R$1

and the resultant datasets by

$R$2

The environment mapping free variables to datasets is written $R$3, and variables evaluate according to

$R$4

These definitions make the repository, the program, and the free-variable environment explicit components of the semantic model (Kang et al., 11 Aug 2025).

3. Small-step operational semantics

The semantics of ImpRAT are given as small-step operational semantics, chosen over big-step semantics for clarity. The paper uses two kinds of judgments. Relational expressions are evaluated by judgments of the form

$R$5

where $R$6 is either a dataset or $R$7. Statements are evaluated by judgments of the form

$R$8

or to terminal forms such as $R$9, $D$0, or $D$1 (Kang et al., 11 Aug 2025).

The paper emphasizes a global consistency condition: a valid ImpRAT expression is expected to evaluate only to either $D$2 or $D$3, and a correct ImpRAT expression cannot evaluate to $D$4. Within execution, however, $D$5 functions as a neutral continuation token in the statement semantics. For sequencing, the paper gives:

$D$6

$D$7

$D$8

These rules make explicit that sequencing continues after $D$9, propagates failure after $x$0, and stops after successful return $x$1 (Kang et al., 11 Aug 2025).

Assignment is also given operationally. The paper provides:

$x$2

$x$3

$x$4

Thus, assignment commits only when the right-hand relational expression successfully evaluates to a dataset. The return construct is handled analogously:

$x$5

with terminal success when $x$6 evaluates to a dataset (Kang et al., 11 Aug 2025).

4. Relational operators, type tests, and imperative features

ImpRAT includes the standard relational algebra operators: union $x$7, difference $x$8, Cartesian product $x$9, projection $\bot$0, selection $\bot$1, and rename $\bot$2. The paper’s distinctive claim is that it adds two features beyond standard relational algebra: first-class type tests, and an imperative structure for solver-style execution (Kang et al., 11 Aug 2025).

The operator

$\bot$3

is a dynamic test that checks whether relation $\bot$4 satisfies a collection of type or predicate constraints. The types are intentionally simple: attribute existence tests, existential predicate tests, and universal predicate tests. This is the paper’s “types-as-constraints” mechanism. The paper gives representative rules. For universal tests,

$\bot$5

if all tuples in $\bot$6 satisfy $\bot$7, and $\bot$8 otherwise. For existential tests,

$\bot$9

if some tuple satisfies $R\{T_0,\ldots,T_n\}$0, and $R\{T_0,\ldots,T_n\}$1 otherwise. For attribute-presence tests,

$R\{T_0,\ldots,T_n\}$2

if $R\{T_0,\ldots,T_n\}$3 has those attributes, and $R\{T_0,\ldots,T_n\}$4 otherwise (Kang et al., 11 Aug 2025).

Projection and selection are guarded by those tests. The projection rule is presented as

$R\{T_0,\ldots,T_n\}$5

where $R\{T_0,\ldots,T_n\}$6 is the restriction to those attributes. Selection is given by

$R\{T_0,\ldots,T_n\}$7

where $R\{T_0,\ldots,T_n\}$8 is the subset of tuples satisfying $R\{T_0,\ldots,T_n\}$9. Renaming is defined similarly: if the source and target attribute tests behave appropriately, then

$T$0

where $T$1 is the renamed dataset; otherwise the expression reduces to $T$2. The paper also gives a type-chain rule:

$T$3

if the first test succeeds, and $T$4 otherwise (Kang et al., 11 Aug 2025).

The imperative layer consists of free-variable declaration !x, assignment x := R, sequencing S_0;S_1, and return \Delta R. The paper describes this as giving the language a solver-like execution discipline: compute intermediate datasets, store them in variables, check constraints, continue or fail. It further states that the design borrows inspiration from Kleene Algebra with Tests and from imperative programming semantics, but uses types or predicate tests rather than purely boolean terms (Kang et al., 11 Aug 2025).

5. Translation from TQL and implementation model

TQL is the user-facing language, whereas ImpRAT provides the semantics. The paper states that TQL is both a refined semantic subset of ImpRAT, avoiding syntactically valid but nonsensical programs, and a user-oriented extension of ImpRAT, adding convenient syntax such as +, -, *, bracketed filters, projections, and type annotations. The relationship is therefore explicitly presented as:

$T$5

The meaning of a TQL query is the meaning of its translated ImpRAT program (Kang et al., 11 Aug 2025).

Several translation clauses are given explicitly. A typed variable-binding form

$T$6

translates as

$T$7

An untyped variable declaration translates by

$T$8

and return translates by

$T$9

The paper also maps TQL relational operators into algebraic notation: expr [bop](https://www.emergentmind.com/topics/binary-optimizer-bop) expr maps to $a$0 where bop is +, -, or *; expr[ attr0 -> attr1 ] maps to $a$1; expr[ pred ] maps to $a$2; and expr[ attr1; ...; attrN ] maps to $a$3. Type notation is translated by \/(pred) to $a$4, /\(pred) to $a$5, and [attrN] to the attribute symbol $a$6 (Kang et al., 11 Aug 2025).

The proof-of-concept implementation is described as a modular query engine for TQL, implemented in Python. Its architecture has two main parts. The first is a parser-interpreter built with PLY (Python Lex-Yacc), which parses TQL according to the BNF, constructs an AST, and supports interpretation and static analysis through recursive node methods. The second is a query engine solver that treats discovery as finding an input tuple that makes the ImpRAT or TQL program succeed. The main solver algorithm is given procedurally: initialize feedback $a$7; choose a candidate input $a$8; run the program on $a$9; if output $\exists \varphi$0, return $\exists \varphi$1; otherwise add $\exists \varphi$2 to feedback and continue. For the prototype, the choice function is a classical backtracking search over repository datasets, recursively enumerating multisets of size $\exists \varphi$3 from $\exists \varphi$4. The paper notes that the problem is NP-hard, so the implementation is intentionally modular and extensible rather than optimized (Kang et al., 11 Aug 2025).

6. Completeness, semantic guarantees, and related algebraic context

The paper distinguishes between the theorem corpus attached to TCRA and the guarantees stated for ImpRAT. For TCRA, it gives a Natural Inclusion lemma, with $\exists \varphi$5 from relational algebra into TCRA, and a Relational Completeness theorem stating that TCRA is relationally complete, that is, a valid extension of relational algebra. For ImpRAT, the paper does not present a comparable large theorem corpus such as preservation or progress, but it does assert several semantic guarantees: valid expressions are designed to evaluate to either $\exists \varphi$6 or $\exists \varphi$7; correct ImpRAT expressions cannot evaluate to $\exists \varphi$8; failures from illegal relational operations are represented as $\exists \varphi$9; and type tests are used to catch many illegal operations before they occur, improving semantic consistency (Kang et al., 11 Aug 2025).

Within a broader research context, ImpRAT occupies a position between classical relational algebra and typed or semantically enriched relational systems. A closely related but methodologically different line is the embedding-based model ReAlE, which treats knowledge hypergraph completion in terms of relational algebra operations and proves representability results for renaming, projection, selection, set union, and set difference. ReAlE is therefore relevant to the algebraic side of ImpRAT, but it is an embedding-based scoring model rather than an imperative typed algebra or rule-execution language (Fatemi et al., 2021).

Another relevant comparison is to the algebraic CSP literature on typed local structure. In Bulatov’s work on graphs of relational structures, edges of a local graph are classified by tractable operation types—semilattice, majority, and affine—and restrictions on allowed edge types are shown to be preserved under standard algebraic constructions, with direct consequences for CSP tractability. This suggests a conceptual analogy rather than an identity of frameworks: both settings use typed restrictions on relational or algebraic objects to control valid constructions and computational behavior, but the typing in Bulatov’s framework is semantic and algebraic, defined by term identities and quotient behavior, whereas ImpRAT’s typing mechanism is implemented as dynamic tests over relations within an operational language (Bulatov, 2020).

Taken together, these comparisons place ImpRAT within a wider pattern of research that treats relational structure as carrying more than tuple membership alone. In ImpRAT, that enrichment takes the form of a typed, imperative, small-step algebra for expressing discovery queries as programs over datasets. Its specific contribution is to formalize the conjunction of relational algebra, dynamic type and predicate tests, and imperative sequencing as the semantic core of TQL (Kang et al., 11 Aug 2025).