Compile-Time Schema Absorption

Updated 5 July 2026

Compile-Time Schema Absorption is a mechanism that incorporates external structural specifications at compile time to enforce type conformance and generate optimized code.
It shifts challenges from runtime to build time by embedding schema contracts into the compilation process, thereby reducing late error discovery.
The technique spans diverse applications—from typed data pipelines and API bindings to GPU kernel fusion—improving consistency and performance.

Compile-time schema absorption is a mode of language and compiler design in which schema-like structure is incorporated into compile-time artifacts and then used to constrain typing, code generation, optimization, or boundary validation before ordinary execution. In recent work, the term appears explicitly in Turn, where use schema::<protocol> synthesizes typed bindings from external specifications at compile time, but closely related mechanisms also appear in Scala/Spark structural contracts, schema-first telemetry, F# type providers from data, dimensional and coeffect metadata carried through native compilation, and compile-time expression fusion for GPU kernels (Kizito, 7 Mar 2026, Mirji, 18 Apr 2026, Shkuro et al., 2022, Petricek et al., 2016, Haynes, 17 Mar 2026, Curtin et al., 24 Apr 2026).

1. Conceptual scope

Across these systems, the absorbed “schema” is not confined to database schemas. It may denote a sink contract for a data pipeline, an OpenAPI document, a Thrift telemetry IDL, a sample-derived structural shape, a type hierarchy inferred from operations, a regular tree grammar for XML, or even the type-level abstract syntax tree of a linear algebra expression. The common pattern is that structure which would otherwise remain external, implicit, or runtime-only becomes a first-class compile-time input to later phases of checking or generation (Mirji, 18 Apr 2026, Shkuro et al., 2022, Petricek et al., 2016, Haynes, 17 Mar 2026, Curtin et al., 24 Apr 2026, Birillo et al., 2022, Amavi et al., 2014, Kuijper et al., 2011).

Domain	Absorbed schema	Compile-time effect
Typed JVM/Spark pipelines	Scala 3 contract types plus compatibility policy	Structural conformance proof and sink construction
Agentic API use	External protocol specification	Synthesized structs and typed closures
Telemetry	Thrift schema and annotations	Generated strongly typed emission APIs and CI checks
Structured data access	Sample-inferred shape	Provided nominal host-language types
Native compilation	Dimensions, ranges, lifetimes, coeffects	Representation and allocation decisions
GPU linear algebra	Type-level expression AST	Fused kernel generation
Program reflection	Classes, objects, functions, annotations	IR rewriting to static symbol collections
XML evolution	RTG/LTG schema mappings	Conservative extension and guided translation

This suggests a broader interpretation of the topic: compile-time schema absorption is less a single technique than a family of techniques for moving structural knowledge “left,” from ad hoc runtime interpretation toward compilation, static analysis, and build-time generation.

2. Representations and compilation pipelines

A central design choice is the representation used to hold the absorbed schema. In the Scala/Spark framework, the contract is a Scala 3 case-class type R, the producer is another Scala type Out, and compatibility is expressed by the type class SchemaConforms[Out, Contract, P]. An inline given plus macro inspects both types through quotes.reflect, normalizes them into a TypeShape model, compares them under a policy, and either emits a witness or aborts compilation with a structural drift report. The same contract type is also used to derive a Spark StructType, so there is no separate hand-maintained sink schema (Mirji, 18 Apr 2026).

Turn embeds the mechanism more directly into the surface language. The construct use schema::<protocol>("url-or-path") is parsed into an Expr::UseSchema AST node, consumed by a dedicated SchemaCompiler phase that runs before analysis, and replaced by an IIFE containing generated struct definitions and request closures. The OpenAPI adapter is fully implemented, while graphql, fhir, and mcp are in active development. The key point is not merely code generation but placement: the generated symbols are visible to the ordinary type checker because expansion happens before semantic analysis (Kizito, 7 Mar 2026).

In schema-first telemetry, the representation is an authoritative Thrift schema. The build system generates language bindings, application code emits telemetry through those generated types, and a schema actualization service in CI validates backward compatibility before committing updated schemas to a metadata store. The schema is therefore absorbed into both code and deployment workflow rather than being reconstructed from free-form labels after the fact (Shkuro et al., 2022).

F# Data takes a different route because many inputs have no declared schema. It infers a structural “shape” from sample JSON, XML, or CSV and then maps that shape into nominal F# types through type providers. The formal translation is written as $\llbracket \sigma \rrbracket = (\tau, e, L)$ , where $\tau$ is the provided host-language type, $e$ is a conversion function from raw data, and $L$ is the set of generated declarations used by the type (Petricek et al., 2016).

Other systems absorb structure that is not usually called a schema. In the dimensional compilation framework, the Program Semantic Graph carries HM types, dimensions in $\mathbb{Z}^n$ , and coeffects such as allocation strategy and lifetime across MLIR lowering stages (Haynes, 17 Mar 2026). In Bandicoot, the C++ type of an expression is treated as the expression AST, and template metaprogramming turns that type-level structure into fused GPU kernels at compile time (Curtin et al., 24 Apr 2026). In Reflekt, compiler-plugin queries over classes, objects, and functions are resolved at compile time and rewritten into static IR or generated implementations, eliminating runtime discovery (Birillo et al., 2022).

3. Semantic models and compatibility relations

Compile-time schema absorption is not merely storage of metadata; it depends on an explicit compatibility semantics. The Spark framework is exemplary because it makes schema evolution policy-parametric. Its policy family includes Exact, ExactUnorderedCI, ExactOrdered, ExactOrderedCI, ExactByPosition, Backward, Forward, and Full. Backward uses contract-by-name subset semantics: producer extras are allowed, while missing contract fields are permitted only if the contract field is optional or has a default. Forward reverses the subset direction. The macro compares normalized TypeShapes rather than raw Scala types and tracks field-level optionality separately from nested collection optionality, enabling checks such as SequenceShape(OptionalShape(IntShape)) versus SequenceShape(IntShape) (Mirji, 18 Apr 2026).

The dimensional compilation work formalizes an even richer absorbed schema. Dimensions live in $\mathcal{D}=\mathbb{Z}^n$ , with inference expressed as integer-linear constraint solving over finitely generated abelian groups. The same Program Semantic Graph also carries coeffects for allocation, lifetime, reachability, and capabilities. Escape analysis is expressed by a lifetime-promotion rule and escape kinds $\{\text{StackScoped},\ \text{ClosureCapture}(t),\ \text{ReturnEscape},\ \text{ByRefEscape}\}$ , so representation choice, word width, memory footprint, and allocation strategy become downstream consequences of the same semantic graph rather than independent passes (Haynes, 17 Mar 2026).

For JSON Schema, static absorption requires a decision procedure over schema logic itself. Witness generation provides that substrate. Satisfiability, inclusion, and equivalence reduce to witness generation: S is satisfiable iff a witness exists; S \subseteq S' iff S \wedge \neg S' is unsatisfiable; and equivalence reduces to checking unsatisfiability in both directions. The paper gives a direct witness-generation algorithm for essentially the full language except uniqueItems, after translation to a positive algebra, stratification, guarded DNF, and bottom-up generation over objects, arrays, and scalar groups (Attouche et al., 2022).

A related but more language-centric formalization appears in automatic type hierarchy inference. There, compile-time absorption takes the form of a largest simulation-based static subsumption relation over a finite set of relevant types. Typeflow analysis then propagates sets of possible types through a syntax graph using antichains rather than explicit bitvectors, yielding bidirectional inference over arbitrary syntax graphs. This gives a static lattice that can express, for example, container subsumption induced automatically from element-type subsumption (Kuijper et al., 2011).

4. Compile-time knowledge and runtime enforcement

A recurrent misconception is that compile-time schema absorption eliminates the need for runtime checks. The literature does not support that claim. Instead, several systems pair compile-time absorption with a runtime pin at the actual boundary where data or external behavior appears.

In the Scala/Spark framework, SchemaConforms[Out, R, P] is required to build a sink, but at sink execution the framework still derives a StructType from R, retrieves df.schema, and invokes a PolicyRuntime[P] comparator. This comparator mirrors compile-time policy semantics and adds a nested collection optionality check that Spark’s built-in comparators omit by examining ArrayType.containsNull and MapType.valueContainsNull explicitly (Mirji, 18 Apr 2026).

Turn follows the same pattern in a different setting. Schema absorption generates native Turn types and typed closures, but Cognitive Type Safety still uses generated JSON Schema plus VM-side validation before binding model output. The same struct shape can therefore constrain both an external API binding and the output of infer, while capability-based identity ensures that generated API closures use opaque VM-managed credentials rather than raw secrets in agent memory (Kizito, 7 Mar 2026).

Schema-first telemetry likewise splits responsibilities. Generated Thrift types and CI actualization enforce schema compatibility before deployment, but telemetry backends still parse binary payloads using schemas stored in a metadata service, and those schemas continue to drive introspection, correlation, and privacy enforcement at runtime (Shkuro et al., 2022).

F# Data makes the same separation explicit in formal terms. The provided type is static, but the generated conversion code still performs dynamic checks such as convField, convNull, and convElements. The result is a relative type soundness theorem: if runtime data has a shape that is a subshape of the inferred sample shape, then well-typed user programs using the provided types do not go wrong (Petricek et al., 2016).

This division of labor is one of the defining features of the field. Compile-time absorption shifts validation earlier and makes more structure available to the compiler, but it does not erase uncertainty arising from external data, model output, or evolving systems.

5. Empirical consequences

The reported consequences are heterogeneous because the absorbed schema plays different roles in different systems. In the Scala/Spark contract framework, evaluation covers compile-time proofs, runtime policy tests, builder-path end-to-end tests, and benchmarks. Reported compile-time overhead for 10/25/50 schema pairs is on the order of 0.27–0.51 s on local macOS and 0.85–1.88 s on GitHub Ubuntu, with overhead fractions of about 11–17%. The runtime comparator costs about 100–200 ns for by-position exact comparison and about 4.7–8.1 $\mu$ s for unordered exact comparison, versus Spark baselines of about 0.3–0.4 $\mu$ s; the custom comparator is therefore about 17–25 $\times$ slower than Spark’s simplest comparator, but still runs only once per sink write rather than per row (Mirji, 18 Apr 2026).

In Reflekt, moving reflective discovery to compile time produces a small compilation-time increase and a larger startup-time reduction in serverless settings. Reflection-related compilation cost is reported at about 1.1–1.2% of total compilation time. Replacing runtime reflection in Kotless yields total startup reductions from 320 ms to 265 ms for the Kotless website and from 290 ms to 250 ms for the URL shortener, corresponding to about 13.8% and 17% faster startup, respectively (Birillo et al., 2022).

TSCG shows that compile-time schema compilation can be consequential even when the “schema” is a tool description for LLM invocation rather than a traditional data contract. The compiler combines eight composable operators, proves a compression bound of at least 51% on well-formed schemas, and reports 52–57% token savings in scaling experiments. On TSCG-Agentic-Bench, Phi-4 14B is restored from 0% to 84.4% accuracy at 20 tools and reaches 90.3% at 50 tools. A format-versus-compression decomposition gives $\tau$ 0 when improvement is regressed against JSON-baseline accuracy and $\tau$ 1 when the baseline is already text, indicating that representation change rather than compression is the dominant mechanism (Sakizli, 4 May 2026).

Bandicoot shows the performance side of absorption in compiled numerical code. Because the expression AST is available in C++ types at compile time, template metaprogramming can fuse a compound linear algebra expression into a single GPU kernel. On an RTX 4090 with peak theoretical bandwidth 1008 GB/s and measured maximum achievable bandwidth 868 GB/s, the reported result is that only Bandicoot generalizes to arbitrary operations while maintaining peak memory bandwidth in the add- $\tau$ 2 benchmark, whereas ArrayFire eventually splits the computation into multiple GPU operations (Curtin et al., 24 Apr 2026).

These results do not imply a single empirical law. Rather, they show that compile-time schema absorption can target different operational outcomes: earlier drift detection, reduced cold-start overhead, stronger tool-use prompts, or more aggressive low-level fusion.

6. Limits, distinctions, and open directions

The literature also shows that compile-time schema absorption is neither universal nor cost-free. The Spark contract framework explicitly describes itself as “a narrow, honest mechanism artifact” and leaves the broader claim that compile-time structural contracts deliver measurable productivity or reliability in practice for future work (Mirji, 18 Apr 2026). Turn currently has a fully implemented openapi adapter, while graphql, fhir, and mcp remain in active development (Kizito, 7 Mar 2026). DTS and DMM deliberately restrict themselves to decidable algebraic fragments rather than full dependent typing, and the paper states that no full formal proof of decidability is yet given for the combined system (Haynes, 17 Mar 2026).

Sample-based absorption is especially conditional. In F# Data, the guarantee is explicitly relative to representative samples; if future data is not a subshape of the inferred sample shape, conversion may fail at runtime (Petricek et al., 2016). JSON Schema witness generation is complete only for the supported fragment and excludes uniqueItems; handling of full JSON Schema regular expressions is also incomplete in practice because unrestricted ECMA regex features are not directly tractable in the same way (Attouche et al., 2022). XML schema absorption through conservative extension is formulated for regular tree grammars and local tree grammars, so its guarantees are tied to that formal setting rather than full XML Schema with richer constraints (Amavi et al., 2014).

Another important distinction is between compile-time absorption and neighboring techniques. It is not identical to wholesale typed APIs, because the Spark work targets only the producer-to-sink boundary rather than requiring a full typed Dataset rewrite (Mirji, 18 Apr 2026). It is not identical to out-of-band code generation, because Turn and schema-first telemetry place schema expansion inside the language or build pipeline itself (Kizito, 7 Mar 2026, Shkuro et al., 2022). It is not identical to runtime reflection replacement, although Reflekt is a clear adjacent case, because some absorbed schemas describe external data or protocols rather than program declarations (Birillo et al., 2022).

A plausible synthesis is that compile-time schema absorption becomes most valuable when three conditions hold simultaneously: the relevant structure is stable enough to compile against, late discovery is operationally expensive, and the language or compiler can preserve the absorbed structure long enough for downstream phases to use it. Where those conditions fail, the literature generally reintroduces runtime validation, bounded fragments, or explicit scope limits rather than claiming total static resolution.