- The paper introduces ONTO, a schema-once columnar notation that reduces token consumption by 46–51% compared to JSON.
- The paper demonstrates that ONTO’s efficient encoding leads to a 5–10% reduction in LLM inference latency and lowers API costs.
- The paper validates that ONTO’s human-readable, hierarchical format supports complex nested data while ensuring deterministic parsing.
Problem Statement and Motivation
The increasing integration of LLMs into operational intelligence and analytical pipelines necessitates prompt serialization methods that are context-window-aware rather than document-interchange-focused. Traditional serialization formats, particularly JSON and YAML, introduce substantial token overhead, primarily attributable to per-record key repetition and redundant structural punctuation. This redundancy severely limits the effective context window utilization, increases inference latency, and inflates API costs in large-scale deployments. Existing solutions such as CSV are efficient but lack support for nested structures, while binary formats like Parquet, although highly compressive, are not promptable and hinder human readability. Simple prompt compression schemes are lossy and agnostic to the regularity of structured data. The paper precisely identifies the LLM input serialization task as an optimization regime distinct from document interchange, proposing ONTO as a dedicated solution.
Onto introduces a schema-once, data-many columnar serialization format tailored for LLM input. Its syntax and structure are engineered explicitly to optimize token economy, retain human readability, and support arbitrary levels of hierarchy.
Core features:
- Schema declaration occurs once per data entity, followed by pipe-delimited (∣) value rows, effectively eliminating per-record key redundancy.
- Indentation (fixed, four spaces per level) encodes hierarchy and replaces brace/bracket nesting, minimizing structural token cost compared to brace-heavy formats.
- Pipe delimiter is used exclusively for value separation, leveraging its minimal tokenization cost and rarity in operational data.
- Arrays within records are represented using the caret ($^$), avoiding nested bracket overhead.
- Type inference is adopted, with explicit string demarcation via backticks to avoid ambiguity. Nulls and empty strings are distinctly encoded.
- Record counts are declared up-front as Entity[N]: for deterministic value alignment and validation.
The design choices collectively ensure deterministic parsing, strict structure, and operational robustness while maintaining human readability—a property lacking in most highly compressed or binary formats.
Empirical Evaluation
Token Efficiency and Scaling
Evaluation on three synthetic datasets—IoT Telemetry (nested), System Metrics (flat), Log Entries (flat)—demonstrates ONTO's robust token reduction properties. When scaling from 100 to 1,000 records, ONTO consistently reduces token consumption by 46–51% relative to JSON. The majority of savings derive from the elimination of repeated keys (>100% of gross reduction in some regimes), with secondary contributions from punctuation elimination. Indentation overhead for hierarchy introduces a 4–5 percentage point gap between flat and nested data efficiency. The tokenization cost decomposition further validates that structural changes, not content optimization, are responsible for the observed efficiency.
Latency and Inference Cost
Reductions in token count yield sublinear but significant improvements in LLM inference latency (5–10% reduction in TTFT and total time at the 1,000-record scale). The latency savings are consistent, model-agnostic, and scale with prompt size, offering direct inference cost and capacity benefits in production workloads.
Task Comprehension and Robustness
When benchmarked on lookup, counting, extraction, and aggregation tasks using both cold (no context) and warm (with format explanation) prompts, ONTO demonstrates no task-specific comprehension degradation. It matches or surpasses JSON on all semantic tasks except counting, where LLM model limitations dominate performance regardless of format. This validates that the syntactic innovations of ONTO do not impede LLM task efficacy provided minimal contextualization.
Comparative Analysis
CSV achieves similar efficiency but cannot encode nested structures; YAML provides negligible token savings relative to JSON due to persisting key repetition. Recent approaches such as schema-aware JSON compaction (e.g., TOON) may approach ONTO's efficiency on flat data but underperform on hierarchical data due to persistent structural redundancy. ONTO, by contrast, explicitly occupies the previously unfilled intersection of columnar efficiency, hierarchical representation, human readability, and promptability.
Applicability, Integration, and Limitations
Onto's ideal deployment regime is for automated preprocessing of large, homogeneous, structured datasets (100+ records, repeated schema) destined for prompt-based LLM analysis. Integration is minimal—single-call serialization prepends downstream inference, preserving existing LLM pipelines. Immediate practical implications include notable API input cost reduction, higher throughput under strict context window limits, and latency improvements.
Limitations include lack of support for highly heterogeneous schemas, unbenchmarked performance on very deep nesting, absence of robust minification baselines, synthetic data bias in experiments, and non-streaming parser implementation. Furthermore, comparative assessments are restricted to a specific tokenizer (cl100k_base); cross-tokenizer validation remains a future direction.
Related Work Contextualization
ONTO is positioned distinct from legacy data interchange formats (JSON, YAML, Protocol Buffers), which prioritize interoperability or storage, and from lossy prompt compressors (LLMLingua series), which are not schema-aware. Retrival-augmented methods and batch prompting optimize payload content, not representation format. Schema-aware approaches such as TOON provide incremental efficiency but do not fundamentally alter structural token redundancy in nested data. Onto thus defines a unique methodological niche in prompt-preprocessing serialization.
Implications and Future Directions
ONTO's introduction foregrounds a new optimization domain for LLM-centric pipeline design, shifting focus from engineering model-side efficiency to pre-inference data representation. The demonstrated correspondence between structural token reduction and practical system efficiency establishes a strong case for schema-aware prompt data engineering.
Immediate future work includes development of a streaming Onto encoder/decoder for large data volumes, tokenization-sensitivity benchmarks across major LLM architectures, and formal schema definition language extensions. Systematic ablation of nesting depth and edge-case analysis on production corpora are required to fully characterize the limitations and optimal operational envelope of the format. Tool definition serialization, compact hierarchy notation, and integration with retrieval and augmentation protocols represent other promising directions.
Conclusion
ONTO establishes a formal, practical columnar format for LLM input serialization, achieving an empirically validated 46–51% token reduction over conventional formats on large, structured data while retaining full human readability and nested structure support. Its schema-once principle minimizes the principal source of serialization inefficiency, yielding measurable improvements in latency and cost. Onto occupies a distinct and previously unaddressed niche in the prompt engineering pipeline, and its open-source specification and infrastructure enable immediate adoption and further empirical research.