Table-Level Data Integrity (TDI)

• Table-Level Data Integrity (TDI) is a multifaceted concept that ensures a table's overall data consistency and authenticity, extending beyond single-cell validations.
• It leverages lifecycle-driven constraints, dynamic rules, and update-specific checks to enforce data quality across evolving business states and interrelated tables.
• TDI incorporates cryptographic methods and ledger-backed attestation to detect tampering and ensure historical traceability in data-intensive systems.

Table-Level Data Integrity (TDI) denotes, across recent database, data-quality, and integrity-attestation research, the protection and verification of integrity properties whose natural scope is a table, a set of related tables, or the evolution of table contents over time. In this usage, integrity is not limited to attribute-domain validation. It includes lifecycle-constrained state transitions of business objects, generalized entity-integrity conditions over incomplete relations, update-specific checking of denial-style constraints, semantic-domain validation of columns, and tamper-evident attestation of tuples, batches, and tables through ledgers or blockchain-backed metadata (Thi et al., 2017, Hannula et al., 2021, Martinenghi, 2024, Srivastava et al., 2019, Chen et al., 14 Apr 2025, Yu et al., 18 Jul 2025).

1. Conceptual scope

One explicit table-scoped definition appears in the chain-table literature:

“The introduced chain table guarantees the Table-Level Data Integrity (TDI) throughout the life cycle of a data table. It means the data integrity of the data table is protected by its chain table from its creation time. One chain table only focuses on the data integrity of its associated data table but nothing else.”

That formulation is narrower than the broader uses found elsewhere. The same paper first frames Relational Data Integrity (RDI) as persisting critical relational data and preventing anyone, “even high-level administrators,” from “tempering or altering the recorded data,” including both “the data content and its relational schema,” and then presents TDI as the table-scoped realization of that goal (Yu et al., 18 Jul 2025).

Other papers operationalize table integrity differently. In the dynamic-rules literature, the relevant object is the lawful evolution of records across lifecycle states and linked tables rather than a cryptographic table digest. In the key-set literature, the central concern is entity integrity in incomplete relations, where no single primary key may be complete and unique for all rows. In the semantic-domain literature, the integrity object is often a semantically coherent column inside a table. In blockchain-assisted systems such as Verity and AUDITEM, table integrity is treated as authenticity or attestation of tuples, table subsets, or batches, with table-wide assurance arising from verification of all relevant metadata rather than from a single native table-state proof object (Thi et al., 2017, Hannula et al., 2021, Chen et al., 14 Apr 2025, Srivastava et al., 2019, Shi et al., 2022).

Taken together, this suggests that TDI functions less as one canonical formalism than as an umbrella label for integrity guarantees that exceed single-cell or single-attribute validation. Depending on the paper, the protected object may be a current table snapshot, a table’s write history, a batch subset in a warehouse, a set of cross-table lifecycle relations, or the separability of tuples in the presence of nulls.

2. Lifecycle-driven and rule-based integrity

A process-oriented account of TDI is given by the work on dynamic integrity rules derived from object life cycles. There, integrity rules are defined as formalized constraints on data derived from business rules. The paper explicitly distinguishes static rules, which can be checked “at any time in the data life-cycle,” from dynamic rules, which concern “data value changes” caused by business-process transactions and are validated “at the moment of data changes.” The formalism for object life cycles is the Node-Star Net,

$$OLC = \langle S, P, fi, fo, loop, back\text{-}inactive \rangle,$$

where $S$ is a set of states, $P$ a set of processes, $fi : (S \times P) \to \{0,1\}$, $fo : (T \times S) \to \{0,1\}$, $loop : (S \times S) \to \{0,1\}$, and $back\text{-}inactive : (S \times S) \to \{0,1\}$. The paper notes $fo : (T \times S)\to\{0,1\}$ while the surrounding definition ties it to processes $P$. The $back\text{-}inactive$ function is especially important because it distinguishes exclusive from cumulative state changes (Thi et al., 2017).

The derivation pipeline has three stages: modelling object life cycles, mapping corresponding elements of database schema and OLC, and deriving rules. The first two are manual and the last can be automated. The paper identifies four schema mappings for lifecycle states: a state may be a table; may be explicitly described by one or more attributes; may be implicitly described by links between objects; or may be defined by a combination of attribute values and links. From those mappings it derives rule templates that include: value-domain constraints on status attributes; transition-order constraints on old and new status values; consistency constraints between status values and linked records; ordering constraints on link establishments across tables; and exclusivity constraints for incompatible states. Representative formulas include

$S$0

and

$S$1

The examples also include cross-table existence constraints such as: if order.OrderStatus = "Paid" then there exists receipt such that receipt.OrderNo = order.OrderNo, and precedence constraints such as: if a delivery exists for an order, then a receipt for that order must already exist (Thi et al., 2017).

This formulation pushes TDI beyond attribute-level validation and simple single-row checks. It supports constraints spanning multiple rows, multiple tables, and successive states of the same business object. The paper’s own summary of supported rule types—attribute-domain constraints, dynamic before/after transition constraints, cross-table state consistency checks, inter-record existence constraints, ordering constraints, and mutual exclusivity constraints—locates TDI close to temporal constraints and business rules rather than to keys, foreign keys, or functional dependencies alone.

3. Update-specific checking and inconsistency tolerance

A second major strand of TDI concerns integrity enforcement under updates. The work on simplified integrity checking for extended denials studies the problem: given a database $S$2, a constraint theory $S$3 such that $S$4, and an update $S$5, determine whether $S$6. Rather than rechecking $S$7 directly, it derives a smaller theory $S$8. A weakest precondition satisfies

$S$9

and a conditional weakest precondition (CWP) satisfies the same equivalence for databases $P$0 such that $P$1. The paper targets extended denials, built from negated existential expressions of the form

$P$2

and defines an extended denial as a universally quantified formula of the form

$P$3

This strictly enlarges the standard denial style because tuple-generating dependencies can be expressed as

$P$4

while equality-generating dependencies appear as ordinary denials such as

$P$5

The derivation pipeline is After followed by Optimize, with unfolding, reduction, subsumption, and resolution-based entailment approximation; the final result $P$6 is proved to be a CWP for schemas in the hierarchical class $P$7 (Martinenghi, 2024).

The practical consequence is that many table-level checks can be compiled into update-specific probes. For the denial

$P$8

an insertion of $P$9 simplifies to

$fi : (S \times P) \to \{0,1\}$0

Similarly, existential constraints such as

$fi : (S \times P) \to \{0,1\}$1

become local witness checks under insertions or deletions. This is the OLTP-oriented face of TDI: enforcing cross-row and cross-table conditions without rescanning the full database after every mutation (Martinenghi, 2024).

A closely related but conceptually distinct result is the theory of inconsistency tolerance. That work challenges the longstanding assumption that simplified integrity checking is correct only when the old state already satisfies all constraints. It defines pre-tests and post-tests under the classical consistent-old-state premise, then introduces the notion of a case of a constraint and says that a simplification is inconsistency-tolerant if, for every case $fi : (S \times P) \to \{0,1\}$2 that held before the update, acceptance by the simplified check guarantees

$fi : (S \times P) \to \{0,1\}$3

The exact definition states, for a pre-test $fi : (S \times P) \to \{0,1\}$4 or post-test $fi : (S \times P) \to \{0,1\}$5, that previously satisfied cases remain satisfied after the update whenever the simplified check succeeds. This is a non-worsening guarantee: it does not imply global repair or global consistency, but it does imply preservation of “what was consistent across updates” even when the old database is dirty (Decker et al., 2024).

For TDI, the combination is important. Simplified checking explains how to reduce update-time work; inconsistency tolerance explains why update-relative integrity control can remain meaningful over legacy, federated, or partially repaired tables. A plausible implication is that these two lines together provide a theoretical basis for “do not make the table worse” enforcement in systems where full cleanliness is unattainable.

4. Generalized entity integrity and learned semantic domains

Entity integrity at table level is generalized by the theory of key sets. A key set is a finite, non-empty collection of subsets of a relation schema $fi : (S \times P) \to \{0,1\}$6. A relation $fi : (S \times P) \to \{0,1\}$7 satisfies a key set $fi : (S \times P) \to \{0,1\}$8 iff for all distinct tuples $fi : (S \times P) \to \{0,1\}$9, there is some $fo : (T \times S) \to \{0,1\}$0 such that $fo : (T \times S) \to \{0,1\}$1 and $fo : (T \times S) \to \{0,1\}$2 are $fo : (T \times S) \to \{0,1\}$3-total and $fo : (T \times S) \to \{0,1\}$4:

$fo : (T \times S) \to \{0,1\}$5

A primary or candidate key is thus a singleton key set; a general key set allows different tuple pairs to be distinguished by different keys. The paper gives a linear-time validation algorithm with complexity

$fo : (T \times S) \to \{0,1\}$6

proves that the general implication problem is coNP-complete, and shows that the unary fragment is much easier, with a quadratic-time decision procedure and computable Armstrong relations. It also notes a hard limit: if two distinct tuples agree on all columns, then no key and no key set can distinguish them (Hannula et al., 2021).

A different generalization of table integrity is given by Semantic-Domain Constraints (SDCs) for unsupervised error detection in tables. An SDC is defined as

$fo : (T \times S) \to \{0,1\}$7

where $fo : (T \times S) \to \{0,1\}$8 is a pre-condition deciding whether the constraint applies to a column, $fo : (T \times S) \to \{0,1\}$9 is a post-condition identifying erroneous values, and $loop : (S \times S) \to \{0,1\}$0 is a calibrated confidence score. The precondition and postcondition are

$loop : (S \times S) \to \{0,1\}$1

and

$loop : (S \times S) \to \{0,1\}$2

The system unifies CTA functions, embeddings, regex patterns, and validation functions as domain-evaluation functions; generates over 100,000 candidate SDCs; filters them with Cohen’s $loop : (S \times S) \to \{0,1\}$3 and a chi-squared significance test; calibrates confidence with a Wilson lower bound; and then distills the candidate set through the CSS and FSS optimization problems. The CSS problem is NP-hard and cannot be approximated with a factor of $loop : (S \times S) \to \{0,1\}$4 unless $loop : (S \times S) \to \{0,1\}$5, while the proposed randomized-rounding scheme gives

$loop : (S \times S) \to \{0,1\}$6

under expected size and false-positive-rate bounds (Chen et al., 14 Apr 2025).

Empirically, the paper reports that on ST-Bench (real), Fine-Select achieves F1@P=0.8 = 0.34 and PR-AUC = 0.45, and on RT-Bench (real) it achieves 0.21 and 0.34. Across 9 existing data-cleaning benchmarks, the paper reports 17 columns receiving new SDC coverage, 94% column-level precision for these newly applied constraints, 183 detected erroneous cells, and 95% strict cell-level precision, rising to 97% after manually augmenting incomplete ground truth. This is still explicitly “single-columns only,” but it expands TDI toward automatically learned, semantically grounded domain constraints that can be applied when hand-authored rules are absent or incomplete (Chen et al., 14 Apr 2025).

5. Ledger-backed authenticity and tamper evidence

A cryptographic branch of TDI focuses on authenticity of stored rows, batches, or tables against insider tampering. Verity is a “dataless framework” that stores fixed-length metadata about SQL tuples on a blockchain rather than migrating the database itself. For a tuple $loop : (S \times S) \to \{0,1\}$7 in table $loop : (S \times S) \to \{0,1\}$8, it defines

$loop : (S \times S) \to \{0,1\}$9

and

$back\text{-}inactive : (S \times S) \to \{0,1\}$0

The system parses and rewrites SQL, verifies base tuples contributing to query results, and stores tuple fingerprints plus table row counts on Hyperledger Fabric. Its guarantee is tuple-granular: unauthorized modification of an accessed tuple is detected exactly through fingerprint mismatch. Table-level assurance is indirect. The paper explicitly states that Verity “does not define table integrity as a first-class cryptographic object”; instead, one obtains practical whole-table assurance by routing SELECT * FROM T through Verity, verifying every returned tuple, and comparing the row count with the blockchain record. The paper also identifies a “special case of illegitimate delete”: if a row is deleted directly in the DBMS, the missing tuple may never appear in later result sets, so stronger detection requires periodic count checks or full-table scans (Srivastava et al., 2019).

AUDITEM is a batch-oriented integrity-attestation architecture for enterprise repositories. Its protected object is a batch subset $back\text{-}inactive : (S \times S) \to \{0,1\}$1 of a data warehouse. The core symbols are $back\text{-}inactive : (S \times S) \to \{0,1\}$2 for identification attributes, $back\text{-}inactive : (S \times S) \to \{0,1\}$3 for verification attributes, $back\text{-}inactive : (S \times S) \to \{0,1\}$4 for the SHA-256 hash of $back\text{-}inactive : (S \times S) \to \{0,1\}$5, $back\text{-}inactive : (S \times S) \to \{0,1\}$6 for the verification record, $back\text{-}inactive : (S \times S) \to \{0,1\}$7 for the private encryption key, $back\text{-}inactive : (S \times S) \to \{0,1\}$8 for ciphertext, and $back\text{-}inactive : (S \times S) \to \{0,1\}$9 for the IPFS location hash. The workflow is

$fo : (T \times S)\to\{0,1\}$0

followed by encryption, IPFS storage, and blockchain registration. Verification first compares $fo : (T \times S)\to\{0,1\}$1 with the registered $fo : (T \times S)\to\{0,1\}$2; on mismatch it decrypts the historical record and performs detailed comparison. Three verification options are defined. Option 1 includes subset hash, column names, row count, GDPR fields, a GDPR-free hash, and vertically calculated SHA-256 for each column. Option 2 adds horizontally calculated SHA-256 for each row. Option 3 stores all data inside the subset. The paper reports that a batch containing 1,000,000 records can be uploaded and verified in about 100 seconds, and that a database with 10,000,000 geographic records, about 4 GB, can be validated within 3 hours (Shi et al., 2022).

Chain Table moves the ledger idea inside the database. For a protected table such as

$fo : (T \times S)\to\{0,1\}$3

it creates a companion ledger

$fo : (T \times S)\to\{0,1\}$4

with

$fo : (T \times S)\to\{0,1\}$5

The protection claim depends not only on the hash chain but on two operational principles: only append operations are allowed on the chain table, and the chain table “only accepts one record written at one time.” Updates and deletions to the protected table are also represented append-only: a deletion is treated as “a special case of update where the original data content is changed to a null value and a new timestamp is added.” Under those assumptions, tampering with an intermediate or last record would require prohibited modifications to existing ledger rows and is therefore “detected and aborted.” The paper presents this as an in-database, blockchain-inspired route to TDI without consensus or full blockchain deployment (Yu et al., 18 Jul 2025).

These systems share a common architectural move—externalizing or separately preserving integrity evidence—but differ sharply in integrity granularity. Verity is fundamentally tuple-based with operational whole-table auditing; AUDITEM is subset- and batch-based with configurable column and row traceability; Chain Table is explicitly table-lifecycle oriented and ledger-centric. This suggests that cryptographic TDI is not one design but a spectrum from per-tuple authentication to append-only historical attestation of full-table updates.

6. Limitations, misconceptions, and unresolved boundaries

Several recurrent misconceptions are corrected explicitly in the literature. In the dynamic-rule paper, “discovery” is not empirical mining from data but deterministic derivation from metadata, object life cycles, and schema-state mappings; the paper provides no search algorithm, support or confidence measure, exception threshold, or quantitative evaluation (Thi et al., 2017). In the Verity paper, table integrity is not a native authenticated table state with a single proof object; it is an emergent property obtained by verifying all tuples and the row count, and omission attacks remain only partially addressed unless periodic whole-table or count audits are performed (Srivastava et al., 2019). In the inconsistency-tolerance paper, preservation of already-satisfied cases is not equivalent to global consistency restoration or repair; old violations may remain, and the guarantee is explicitly non-worsening rather than cleansing (Decker et al., 2024).

The learned-constraint and generalized-key lines have equally clear boundaries. SDCs “operate automatically, but on single-columns only,” assume that training corpora are mostly clean, and do not provide cross-column dependencies, transaction-time enforcement, repair semantics, provenance-aware reasoning, or formal consistency guarantees under updates (Chen et al., 14 Apr 2025). Key sets are a superior generalization of primary keys for incomplete relations, but general implication is coNP-complete, perfect models do not always exist, and exact duplicate tuples remain indistinguishable by any key set (Hannula et al., 2021).

The update-simplification literature is powerful but not universal. Extended denials are aggregate-free, function-free, and non-recursive, and the paper explicitly leaves null handling and richer built-ins outside its core treatment (Martinenghi, 2024). Chain Table, although framed as a protection mechanism even against powerful adversaries such as DBAs, relies on append-only semantics, trusted enforcement of ledger-writing principles, and ideally “independent trusted storage” for the chain table; it provides no decentralized trust, no consensus mechanism, no formal concurrency model, and no explicit treatment of external anchoring or rollback resistance (Yu et al., 18 Jul 2025). AUDITEM likewise is best understood as a table/batch snapshot attestation system rather than an in-engine constraint mechanism; it does not prevent tampering, only detects it later, and its deterministic hashing requires careful canonicalization that the paper does not fully formalize (Shi et al., 2022).

The field therefore has no single complete TDI framework. Instead, it contains several complementary paradigms: lifecycle-driven business constraints, update-relative logical enforcement, generalized entity integrity for incomplete data, automatically learned semantic-domain validators, and ledger-backed authenticity mechanisms. A plausible implication is that future TDI systems will need to combine these strands rather than choose among them: logical constraints for process semantics, incremental checking for OLTP practicality, learned validators for weak-schema environments, and tamper-evident attestation for insider-threat settings.