Koza-Hub: Modular Biomedical Ingest
- Koza-Hub is a modular framework that transforms heterogeneous biomedical data into KGX format, reducing redundant data-wrangling efforts.
- It uses declarative YAML configurations and Python decorators to enforce Biolink Model compliance, ensuring consistent node and edge definitions.
- The system supports scalable, reusable pipelines with automated releases, streamlining the construction of biomedical knowledge graphs.
Searching arXiv for the specified paper to ground the article in the cited source. Koza-Hub is an associated set of conversion processes for gold-standard biomedical data sources built around Koza, a Python software package for transforming raw biomedical information into KGX format under Biolink Model constraints. It is presented as a response to the repeated reinvention of downloading, parsing, filtering, transforming, and serializing heterogeneous datasets during biomedical knowledge graph construction, with the stated aim of making datasets “born-interoperable” by centering ingestion on the KGX standard and enforcing compliance with a chosen data schema (Korn et al., 11 Sep 2025).
1. Origins, goals, and problem setting
The motivating problem is the high amount of redundant labor in biomedical knowledge graph construction. The reported source of this redundancy is the lack of data standards and “knowledge-graph ready” data from upstream sources, such that each new build often re-implements data-wrangling pipelines over dozens of heterogeneous datasets (Korn et al., 11 Sep 2025). In this framing, Koza-Hub is intended to reduce repeated engineering effort while making filtering and mapping decisions more visible.
Its mission is described as providing a growing library of pre-built, tested, “modular ingests” that transform gold-standard biomedical sources into KGX (Knowledge Graph eXchange) format. The emphasis on KGX and the Biolink Model is central: ingested nodes and edges are required to conform to shared vocabulary and relational patterns, with the stated consequence that downstream graph integration, federation, and explainability are accelerated (Korn et al., 11 Sep 2025).
The paper positions Koza-Hub within biomedical knowledge graph practice rather than as a general-purpose ETL framework. This suggests that its principal technical contribution is not a novel graph model, but a standardized ingest layer that collapses recurring preprocessing work into reusable primitives and declarative configuration.
2. Core architecture and execution model
At the center of the system is Koza, a Python package whose pipeline is composed of three primitive modules: Reader, Transform, and Writer (Korn et al., 11 Sep 2025). The architecture is deliberately minimal, with each stage handling a distinct part of the ingest lifecycle.
The Reader parses raw input artifacts, including CSV, TSV, JSON, XML, and SQL exports. Readers are declared in YAML with parameters such as delimiters, header presence, JSON paths, and record-level filters, including conditions of the form status == 'approved' (Korn et al., 11 Sep 2025). This makes source-specific parsing a configuration concern rather than hard-coded pipeline logic.
The Transform layer is a decorator-based Python module that consumes records from the reader and emits Biolink-compliant node or edge objects. Two primary decorators are specified: @transform_record, which applies a function to each record automatically, and @transform, which hands control to the module for custom iteration. Typical hooks include @prepare_data, @on_data_begin, and @on_data_end. Each hook receives a KozaTransform object exposing methods such as write() for emitting nodes and edges, state for cross-record storage, and log for diagnostics (Korn et al., 11 Sep 2025).
The Writer serializes output into KGX property-graph files, either as separate TSVs or JSONL for nodes and edges. These outputs conform to Biolink Model slots including id, category, name, relations, qualifiers, and evidence (Korn et al., 11 Sep 2025). The writer is configured in the same YAML specification as the reader and transform.
Koza-Hub itself is described as the GitHub-hosted collection of per-source repositories, “currently ~25 artifacts across 14 repositories.” Each repository contains a YAML pipeline descriptor, one or more Python transform scripts, and GitHub Actions workflows for monthly automated releases of the KGX outputs (Korn et al., 11 Sep 2025). This repository layout makes the ingest unit a source-specific, versioned artifact.
3. Declarative orchestration and primitive operations
Koza reduces an ingest to a small set of primitives: read, filter, map, and write (Korn et al., 11 Sep 2025). These operations are orchestrated declaratively in YAML. A representative pipeline includes a pipeline name, a reader type and path, delimiter and header configuration, field-level filters, a transform module, and a KGX writer definition with node and edge output files (Korn et al., 11 Sep 2025).
In the ClinVar example, the CSV reader is configured to read only pathogenic or likely pathogenic variants through a filter on clinical_significance, and the transform module then determines how each record produces Biolink:SequenceVariant nodes, associated Gene and Disease nodes, and edges such as biolink:causes or biolink:related_to (Korn et al., 11 Sep 2025). The significance of this design lies in the explicit declaration of parsing, filtering, and serialization decisions “up front,” avoiding bespoke boilerplate.
A second example shows JSON ingestion for an Alliance genotype pipeline, using jsonpath: v = (id=r[id\_field],\ category=Biolink:Class,\ name=r[label\_field],\ properties=\{\dots\})e = (subject=r1[id],\ object=r2[id],\ predicate=chosen\_relation,\ qualifiers=\{\dots\})edge(r) = (s, p, o)\ \text{where}\ s = URI(Gene, r.gene\_id),\ p = biolink:causes,\ o = URI(Disease, r.disease\_id)e_r = \bigl(\mathrm{Gene}\!:\!g,\ \mathrm{biolink}\!:\!causes,\ \mathrm{Disease}\!:\!d\bigr)normalize\_id(raw\_curie) = replace\_prefix(raw\_curie, mapping\_dict)$
with all such operations codified in the Python transform module through KozaTransform.write() calls (Korn et al., 11 Sep 2025). This formalization indicates that Koza’s abstraction boundary is compatible with conventional ETL reasoning: records are transformed into typed vertices and predicate-labeled edges under schema constraints, with normalization treated as a first-class step.
5. Biomedical use cases and operational scale
The paper does not publish detailed throughput benchmarks in terms of CPU or time per ingest, but states that Koza-Hub is exercised in production on monthly builds of the Monarch KG encompassing 40+ datasets (Korn et al., 11 Sep 2025). This places the framework in a recurring operational setting rather than a one-off demonstration.
Two use cases are highlighted. In the ClinGen ingest, selecting only pathogenic or likely pathogenic variants yields a pipeline that completes in under 30 minutes on a single modern server, generating approximately 1 million variant–disease edges and 200 K variant nodes (Korn et al., 11 Sep 2025). Because the selection criterion is encoded in the reader configuration, this example also demonstrates the interaction between declarative filtering and tractable downstream graph generation.
In the Model Organism KG (MOKG) case, nine Koza-Hub sources—HGNC, HPOA, uPheno, PHENIO, PANTHER, PomBase, Dictybase, Alliance, and NCBI Gene—were merged by reusing existing modular ingests plus one custom script to produce a Neo4j-backed knowledge graph viewable at mokg.monarchinitiative.org. The entire pipeline, from raw downloads through Neo4j import, is reported to run in approximately 2–3 hours (Korn et al., 11 Sep 2025).
These examples are significant because they show reuse across both human clinical variation and model organism integration settings. This suggests that Koza-Hub’s modularity is intended to support composition across heterogeneous but schema-alignable biomedical sources.
6. Advantages, limitations, and future directions
The stated advantages are modularity, declarative configuration, standard compliance, Python-based hooks, and automated releases (Korn et al., 11 Sep 2025). Modularity means ingest pipelines per source are developed, tested, and versioned independently and then “simply concatenated.” Declarative YAML configuration specifies formatting, filtering, and output while eliminating repetitive boilerplate. Standard compliance is framed as strict adherence to Biolink/KGX for cross-source interoperability. Python hooks support preparation, per-record processing, and teardown. Automated monthly releases through GitHub Actions provide pre-computed KGX artifacts and reduce compute load for end users (Korn et al., 11 Sep 2025).
The stated limitations are equally specific. GitHub Actions impose an artifact size cap of less than 2 GB, which excludes very large datasets such as gnomAD in raw form. Source updates are only reflected when a modular ingest is rerun and released, introducing update lag. Although KGX files are line-iterable, Koza assumes file-based readers, so true streaming for TB-scale data is not yet fully supported (Korn et al., 11 Sep 2025).
Future extensions include support for artifacts larger than 2 GB through alternative hosting such as cloud buckets or Zenodo, or through partial-artifact sharding; native KGX provision by data providers; enhanced provenance for filtering and transformation decisions at the record level; and publication of standard performance benchmarks such as records per second and memory usage for core readers and transforms (Korn et al., 11 Sep 2025).
A common misconception would be to treat Koza-Hub as a complete solution to large-scale streaming ingestion or as a source of native upstream standardization. The paper does not support either interpretation. Instead, it presents Koza-Hub as a modular ingest framework that improves interoperability and reuse within the current constraints of file-based processing and release-driven updates (Korn et al., 11 Sep 2025).