BDIViz: Biomedical Data Integration Visualization

Updated 7 July 2026

BDIViz is a visual analytics system for biomedical data harmonization that integrates ensemble schema matching with LLM-powered validation.
The system employs multi-view interactive heatmaps and benchmarking tools to reduce curation workload and improve match accuracy.
It supports scalable, provenance-aware curation and developer-driven integration for diverse applications including terabyte-scale image visualization and embedding-based analysis.

Searching arXiv for the cited BDIViz-related papers and closely related records to ground the article. BDIViz most directly denotes Biomedical Data Integration Visualization, an open-source, AI-powered visual analytics system for biomedical schema matching that combines a method-agnostic ensemble of schema matching algorithms, an LLM-powered validation and explanation agent, and a scalable interface centered on matrix-based heatmaps, coordinated value-level views, and provenance-aware curation (Wu et al., 22 Jul 2025). In the literature, the label is also discussed in broader technical senses: as a BigDataViewer-style architecture for client-side exploration of terabyte-scale image volumes (Pietzsch et al., 2014), as a user-relevant multi-view latent variable factorization for interactive embedding (Virtanen et al., 2015), and as a quantitative visualization framework for bipartite interaction data based on localization methods (Einav et al., 2022). Taken together, these usages associate BDIViz with interactive visualization systems in which display, data access, inference, and human validation are tightly coupled.

1. Scope and naming

The most specific and current use of BDIViz refers to the schema-matching system introduced for biomedical data harmonization and later extended for human-in-the-loop benchmarking and matcher development (Wu et al., 22 Jul 2025). That system addresses the alignment of attributes across heterogeneous datasets and target schemas, especially in settings where automated matchers remain unreliable and semantic ambiguity is high.

The broader uses are best understood as technical analogues rather than as a single unified software lineage. They describe, respectively, very large image visualization on ordinary workstations, multi-view embedding that emphasizes statistically shared structure between primary and user data, and low-dimensional mapping of bipartite measurements such as antibody–virus interactions. This suggests a family of related design ideas—interactive exploration, latent or geometric structure, and progressive refinement—rather than a single monolithic definition.

Sense	Core object	Paper
Biomedical Data Integration Visualization	Schema matching and harmonization with LLM-powered validation	(Wu et al., 22 Jul 2025)
Benchmarking extension of BDIViz	Human-in-the-loop benchmarking and iterative matcher development	(Wu et al., 12 Apr 2026)
BigDataViewer-style BDIViz	Terabyte-scale 3D/4D image visualization and arbitrary re-slicing	(Pietzsch et al., 2014)
User-relevant BDIViz	Multi-view latent variable factorization for 2D embedding	(Virtanen et al., 2015)
Bipartite-localization BDIViz	Quantitative maps of bipartite interaction datasets	(Einav et al., 2022)

2. Biomedical schema matching as the primary meaning

In its primary sense, BDIViz was developed for biomedical data harmonization, where source datasets must be mapped to semantically corresponding attributes in a target schema or common data model such as the Genomic Data Commons (GDC) or Proteomics Data Commons (PDC) (Wu et al., 22 Jul 2025). The system is motivated by a conjunction of difficulties that are unusually severe in biomedical domains: target schemas with 400–700+ attributes, source tables with 100–200 attributes, candidate spaces in the tens or hundreds of thousands, and semantically subtle distinctions such as age_at_diagnosis versus age_at_index, or tumor_stage versus FIGO_stage.

The underlying problem is not merely lexical mismatch. Biomedical harmonization also involves heterogeneous terminologies, differing coding schemes, unit mismatches, incomplete metadata, noisy or sparsely populated value sets, and attributes that are split or merged differently across studies. Existing schema matching GUIs are described as relying heavily on node-link or line-based representations and score lists, which become hard to interpret under dense candidate spaces and provide limited support for comparing value distributions or seeing how multiple matchers and LLM judgments interact (Wu et al., 22 Jul 2025).

BDIViz is explicitly designed to address those weaknesses. Its stated goals are to make large-scale biomedical schema matching faster, more accurate, and less cognitively demanding; to combine multiple automated matchers with LLM-based validation to surface high-confidence and ambiguous cases; and to provide an interface aligned with expert workflows, emphasizing value-level evidence, semantic context, and provenance rather than isolated scores (Wu et al., 22 Jul 2025). A common misconception is that BDIViz is simply an LLM matcher. The system description rejects that interpretation: the LLM is used not as an end-to-end matcher over entire schemas, but as a match validator and explainer operating on candidate pairs (Wu et al., 22 Jul 2025).

3. Architecture and end-to-end workflow

The base architecture comprises a backend matching engine, an LLM agent module, a visualization and interaction layer, and a provenance manager (Wu et al., 22 Jul 2025). Users provide a source dataset and either a target schema or a target dataset. The backend performs optional easy-match detection, runs multiple matchers over remaining attribute pairs, normalizes scores, combines them into an ensemble score, and records matcher support for each candidate. The LLM module receives structured context for selected pairs—names, descriptions, value summaries, and history—and returns typed explanations, per-evidence confidence scores, and an overall match or non-match judgment. The frontend renders the heatmap, treemap axes, value-distribution views, value mapping table, UpSet matcher view, explanation panel, and control panel; every user action is logged for undo/redo, timeline navigation, and reproducibility (Wu et al., 22 Jul 2025).

The 2026 extension makes the workflow more explicitly asynchronous and benchmarking-oriented (Wu et al., 12 Apr 2026). Users create a new task by uploading a source dataset and a target schema/dataset. The backend converts these raw inputs into persistent artifacts consumed by the frontend: inferred ontologies for source and target, ranked match candidates with per-matcher scores, value comparisons and distribution bins, provenance logs of user operations, and evaluation summaries for benchmarking. Long-running computations are executed via two Celery worker pools, one for ontology inference and another for candidate generation and value profiling, keeping the UI responsive.

The extended system also formalizes component roles. An ontology worker infers “attribute properties and semantic groupings” when explicit schemas are absent; a matcher-agnostic ensemble runs heuristic, embedding-based, and SLM/LLM-based matchers; value profiling computes frequency counts for categorical attributes, binned histograms for numeric attributes, and a one-to-one value-mapping proposal using fuzzy matching on unique values; and a provenance/export layer records live ground truth and produces the harmonized dataset plus attribute- and value-level mapping specs in CSV/JSON formats (Wu et al., 12 Apr 2026).

4. Matching model, LLM validation, and interaction design

BDIViz’s matching layer is intentionally matcher-agnostic. The prototype integrates a contrastive-learning matcher from bdi-kit, Magneto in zero-shot and fine-tuned variants, and a Jaccard distance matcher from Valentine (Wu et al., 22 Jul 2025). Before invoking heavier matchers, the system applies an easy-match heuristic based on both name similarity and value similarity. If both exceed thresholds, the pair is treated as an “easy match,” assigned score 1, and excluded from further search. On the benchmark of 10 real biomedical datasets mapped to GDC, that heuristic correctly matched 91% of its “easy matches” and accounted for 35% of all ground-truth pairs, thereby reducing curation load (Wu et al., 22 Jul 2025).

The ensemble score is computed as

$s_{\text{ensemble}}(a,b)=\frac{\sum_m w_m\cdot s_m(a,b)}{\sum_m w_m},$

where $s_m(a,b)\in[0,1]$ is the normalized score from matcher $m$ and $w_m$ is its current weight (Wu et al., 22 Jul 2025). BDIViz updates these weights from user feedback. If a candidate $i$ is accepted, then for all supporting matchers $m\in\mathcal{M}_i$ ,

$w_m \leftarrow w_m + \alpha \cdot s_i \cdot \frac{1}{r_i},$

and if rejected,

$w_m \leftarrow w_m - \beta \cdot s_i \cdot \frac{1}{r_i},$

where $s_i$ is the ensemble score and $r_i$ the candidate rank (Wu et al., 22 Jul 2025). This gives the ensemble an online, task-specific adaptation mechanism.

The visualization layer is organized around an interactive heatmap. Rows encode source attributes, columns encode target attributes, and color intensity encodes aggregated matcher confidence (Wu et al., 22 Jul 2025). In the 2025 system, the target axis is a three-level hierarchy rendered as a space-filling treemap aligned with the heatmap, while the y-axis may be clustered using sentence-transformer embeddings plus KNN clustering (Wu et al., 22 Jul 2025). In the 2026 extension, the heatmap supports hierarchical navigation, zoom, and filtering, and cell expansion reveals “value-distribution comparisons as immediate supporting evidence” (Wu et al., 12 Apr 2026). Coordinated views show attribute names, descriptions, example values, frequency counts, binned histograms, one-to-one value alignments, and a Value Wrangler view for checking “row-level edge cases” before finalizing mappings (Wu et al., 12 Apr 2026).

The LLM-powered validation component is designed as advisory evidence rather than authoritative truth. It evaluates candidate pairs using names, descriptions, values, distributions, historical mappings, and domain knowledge, and returns up to four explanations with flags, rationale categories, detailed reasoning, and confidence scores (Wu et al., 22 Jul 2025). The system explicitly notes that LLMs are not fully reliable: depending on model and test set, they can misclassify both true matches and hard non-matches, so the interface exposes explanations with icons and confidence bars rather than binary assertions (Wu et al., 22 Jul 2025). This addresses a recurring concern in LLM-assisted curation: BDIViz uses the model as a transparent collaborator whose output remains visible, grounded, and contestable.

The 2026 extension repositions BDIViz as both a curation environment and a benchmarking platform (Wu et al., 12 Apr 2026). As users accept, reject, or edit matches, the system logs operations into a provenance timeline, records accepted correspondences as live ground truth, and updates evaluation metrics and benchmark visualizations in real time. The benchmarking views include a Matcher Analytics radar plot, a Ranked Breakdown view, and a Consensus/UpSet-style view that reveals agreement and disagreement across matchers (Wu et al., 12 Apr 2026).

The explicitly mentioned metrics are Precision, F1 score, and Mean Reciprocal Rank (MRR). MRR is defined in the standard form

$s_m(a,b)\in[0,1]$ 0

and is emphasized because interactive review depends strongly on whether correct matches are near the top of each source attribute’s candidate list (Wu et al., 12 Apr 2026). The system also stresses top- $s_m(a,b)\in[0,1]$ 1 behavior: new matchers must return the top- $s_m(a,b)\in[0,1]$ 2 target candidates per source attribute.

For developers, BDIViz exposes a standardized Python interface centered on a required top_matches method (Wu et al., 12 Apr 2026). In Developer Mode, a “Create New Matcher” option opens an in-browser code editor; developers can paste code—for example, a HuggingFace-based BERT encoder that embeds column names, computes cosine similarity, and returns top- $s_m(a,b)\in[0,1]$ 3 candidates—and the matcher is registered and executed without modifying the system (Wu et al., 12 Apr 2026). Its scores are materialized into the candidate artifacts and immediately appear in analytics panels.

The demonstration scenario on the manually curated WikiData benchmark shows why this matters (Wu et al., 12 Apr 2026). As curation progresses, the analytics reveal that a BERT matcher can have reasonable Precision and F1 but MRR $s_m(a,b)\in[0,1]$ 4, indicating that correct matches are often ranked low. The UpSet-style consensus view further shows limited overlap between BERT and best-performing lexical matchers on accepted correspondences, while final metrics indicate that a simple syntactic matcher such as Jaccard achieves near-perfect scores on that task (Wu et al., 12 Apr 2026). One practical caveat is also made explicit: early in curation, when live ground truth is dominated by auto-accepted trivial matches, metrics can obscure real performance differences.

6. Broader technical senses of BDIViz

In a BigDataViewer-style sense, BDIViz denotes an architecture for interactive exploration of huge 3D/4D image datasets—often terabyte-scale light-sheet microscopy time-lapses or EM volumes—directly on a client machine (Pietzsch et al., 2014). Its design separates meta-data / data model, data access and caching, and rendering / visualization; organizes data by timepoints, setups, and views; stores each view as chunked, multi-resolution HDF5 volumes; and uses ImgLib2 abstractions, on-demand loading, MRU caching, and volatile pixel types to support non-blocking rendering (Pietzsch et al., 2014). The renderer guarantees smooth interaction at >20 frames per second during navigation by lowering displayed resolution during motion and refining after motion stops. This sense of BDIViz emphasizes client-side arbitrary virtual re-slicing, cache-aware rendering, and tight Fiji integration.

In a user-relevant embedding sense, BDIViz is naturally grounded in the multi-view latent variable factorization proposed for “Visualizations Relevant to The User By Multi-View Latent Variable Factorization” (Virtanen et al., 2015). There the visualization is the shared latent space $s_m(a,b)\in[0,1]$ 5, while view-specific factors explain structure present only in the primary or user view. The latent pairwise similarities are modeled as

$s_m(a,b)\in[0,1]$ 6

$s_m(a,b)\in[0,1]$ 7

so that the displayed 2D structure is constrained to capture only what is statistically shared between the data and user signals (Virtanen et al., 2015). This formulation is useful for understanding BDIViz as a user-adaptive visualization engine in which irrelevant or noisy structure is “explained away” rather than forced into the visible embedding.

In a quantitative bipartite-visualization sense, BDIViz refers to localization-based embedding of bipartite data, where only cross-set distances are observed and both classes must be embedded in a shared low-dimensional Euclidean space (Einav et al., 2022). The paper develops metric MDS, bipartite MDS, and SDP formulations for recovering coordinates from distances $s_m(a,b)\in[0,1]$ 8. In the antibody–virus application, 2D metric MDS achieved cross-validation RMSE $s_m(a,b)\in[0,1]$ 9 in log distance, corresponding to about a 3-fold error in predicted IC $m$ 0, and the resulting map was interpreted as a basis set of antibody behaviors (Einav et al., 2022). This use of BDIViz emphasizes quantitative geometry: proximity directly encodes transformed measurement values, and geometric constructions express trade-offs such as potency versus breadth.

7. Empirical results, limitations, and significance

The strongest empirical evidence for BDIViz concerns the biomedical schema-matching system (Wu et al., 22 Jul 2025). On the 10-dataset→GDC benchmark, the BDIViz Ensemble achieved P@10 = 0.938, P@20 = 0.954, and P@40 = 0.965, substantially exceeding the individual matchers listed in the same evaluation (Wu et al., 22 Jul 2025). In a within-subject expert study with $m$ 1, overall workload scores were $m$ 2, $m$ 3 for BDIViz versus $m$ 4, $m$ 5 for the baseline, with $m$ 6, $m$ 7, $m$ 8, $m$ 9; matching accuracy was $w_m$ 0, $w_m$ 1 for BDIViz versus $w_m$ 2, $w_m$ 3 for the baseline; completion-time analysis gave $w_m$ 4, $w_m$ 5; the reported time reduction was >14%; and the SUS score was 71.87 (Wu et al., 22 Jul 2025).

The case studies reinforce those results. In CPTAC→GDC, using Dou et al.’s endometrial carcinoma dataset with 179 attributes, a biomedical researcher in 15 minutes confirmed all 19 previously identified matches and identified 10 additional correct matches (Wu et al., 22 Jul 2025). In the LUAD dataset→dataset case, an expert identified 13 matches in 10 minutes, compared with 8 previously found manually (Wu et al., 22 Jul 2025). The 2026 demonstration uses a related harmonization scenario with 179 source attributes and 736 target attributes, plus a developer scenario centered on WikiData and runtime integration of a BERT matcher (Wu et al., 12 Apr 2026).

The limitations are stated with equal clarity. BDIViz depends on the quality of underlying matchers and LLM outputs; if all automated methods miss a match, the system cannot propose it (Wu et al., 22 Jul 2025). The matrix-plus-treemap design scales better than node-link diagrams but can still overload visual space for extremely large schemas (Wu et al., 22 Jul 2025). The present interface focuses on dataset→schema and dataset→dataset harmonization rather than explicit many-source synchronization (Wu et al., 22 Jul 2025). In the benchmarking extension, ground truth is live and provenance-rich, but explicit versioning beyond the timeline is not detailed (Wu et al., 12 Apr 2026). These constraints do not negate the system’s contribution; rather, they indicate that BDIViz is best understood as a human-in-the-loop platform that amplifies expert judgment instead of replacing it.

A plausible implication is that BDIViz occupies two roles simultaneously. First, it is a practical visual analytics system for high-stakes schema matching and data harmonization. Second, it is a broader design pattern in which visualization is inseparable from a formal backend—whether chunked multi-resolution rendering, shared-versus-specific latent structure, or geometric reconstruction from partial bipartite measurements. In both senses, BDIViz treats interactivity not as presentation alone but as a coupling between computational inference, progressive evidence, and expert validation.