Spezi Data Pipeline Toolkit

• Spezi Data Pipeline is an open-source Python toolkit that streamlines digital health data analysis using modular FHIR-based representations and rigorous data validation.
• It decomposes workflows into five key modules—Data Access, Flattening, Processing, Exploration, and Export—to enhance scalability, auditability, and interoperability.
• The toolkit supports real-world applications like the PAWS study by efficiently processing sensor, ECG, and clinical questionnaire data while preserving traceability.

Spezi Data Pipeline is an open-source Python toolkit designed to streamline the analysis of digital health data, from secure access and retrieval to processing, visualization, and export. It is integrated into the larger Stanford Spezi open-source ecosystem for developing research and translational digital health software systems, and it leverages HL7 FHIR-based data representations to support standardized handling of diverse data types, including sensor-derived observations, ECG recordings, and clinical questionnaires across research and clinical environments (Bikia et al., 17 Sep 2025).

1. Architectural organization

The system is organized into five loosely coupled modules: Data Access, Data Flattening, Data Processing, Data Exploration, and Data Export. Each module can be invoked independently or composed into a complete workflow. The high-level interaction is linear: Data Access produces “FHIR JSON → Objects,” Data Flattening produces “Flat tables,” Data Processing produces “Filtered/aggregated,” and Data Exploration produces “Plots, tables” for Data Export (Bikia et al., 17 Sep 2025).

Module	Responsibilities	Key classes / methods
Data Access	Secure authentication; query and download raw FHIR JSON resources; materialize JSON into strongly typed Python objects	DataAccessClient, FirebaseDataAccessClient, connect(), fetch_resources(...)
Data Flattening	Convert hierarchical FHIR objects into flat, analysis-ready tables; retain provenance metadata	FHIRDataFrame, ResourceFlattener, ObservationFlattener, QuestionnaireResponseFlattener, ECGFlattener
Data Processing	Clean/filter observations; subset by user/date/LOINC; aggregate; compute derived scores	filter_outliers(...), select_users(...), aggregate_daily(...), compute_activity_index(...), score_phq9(...)
Data Exploration	Plot time series, distributions, multi-user overlays; specialized ECG visualization	plot_time_series(...), plot_ecg(...), plot_risk_trends(...)
Data Export	Write tables and save figures while retaining mapping back to original FHIR resource IDs	to_csv(...), to_excel(...), save_figure(...)

This decomposition emphasizes modularity rather than a monolithic analysis stack. A plausible implication is that the toolkit is intended to reduce bespoke development by isolating authentication, representation conversion, analysis, visualization, and export into separable layers. The paper explicitly associates this modularity with enhanced workflow efficiency and improved scalability and interoperability in digital health research (Bikia et al., 17 Sep 2025).

2. FHIR-based representation and validation

The pipeline builds directly on HL7 FHIR through the Python package fhir.resources, described as a Pydantic-based code generation of the FHIR specification. Raw FHIR JSON resources such as Observation, QuestionnaireResponse, and Patient are parsed through calls of the form Observation.parse_obj(json_dict). This enforces field-level type-checking, required elements, allowed valueSets, extensions, and versioning (Bikia et al., 17 Sep 2025).

The flattening layer formalizes the conversion from hierarchical FHIR resources into tabular structures. For an observation resource, the source gives the mapping

$$f_{\mathrm{Obs}} : \mathcal{R}_{\mathrm{Obs}} \longrightarrow (u,\,r,\,c,\,v,\,u_i,\,t,\dots)$$

where $u$ is the subject reference (user ID), $r$ is resource.id, $c$ is coding.code such as a LOINC code, $v$ is valueQuantity.value, $u_i$ is valueQuantity.unit, and $t$ is effectiveDateTime. For ECG resources, the mapping is described as

$$f_{\mathrm{ECG}}(\mathcal{R}_{\mathrm{Obs}}) = (u,\,r,\,\mathrm{waveform},\,\mathrm{samplingFreq},\,\mathrm{hr},\,\mathrm{class},\dots).$$

The resulting tables retain provenance metadata, including resource identifiers, coding, and extension fields. An example observation table includes columns such as user_id, resource_id, loinc, display, value, unit, timestamp, and device_code. This structure preserves auditability while making the data analysis-ready (Bikia et al., 17 Sep 2025).

Validation behavior is central to the pipeline’s FHIR integration. The fhir.resources layer enforces FHIR release versions such as DSTU2 and R4 at build time; unknown extensions are preserved in an .extension field; and loading an invalid resource raises a pydantic.ValidationError, which the Data Access module surfaces to the user. This indicates that the toolkit is not merely a file converter but a standards-constrained interface between storage backends and analytical workflows (Bikia et al., 17 Sep 2025).

3. Data access, flattening, and analytical APIs

The Data Access module handles secure authentication, including examples such as OAuth2 and Firebase SDK credentials, and it queries cloud stores such as Google Firebase Firestore for raw FHIR JSON resources. The abstract interface is DataAccessClient, with FirebaseDataAccessClient as a concrete implementation. Its exposed methods are connect() -> None and fetch_resources(resource_type: str, filters: dict) -> List[pydantic.BaseModel] (Bikia et al., 17 Sep 2025).

Flattening is implemented through FHIRDataFrame, a subclass of pandas.DataFrame, and a family of resource-specific flatteners. ObservationFlattener, QuestionnaireResponseFlattener, and ECGFlattener are concrete implementations of the abstract ResourceFlattener. Their role is to convert hierarchical FHIR objects into flat tables without discarding the metadata needed to trace derived rows back to source resources (Bikia et al., 17 Sep 2025).

Processing operations are explicitly oriented toward digital health analysis. The documented functions include filter_outliers(df: FHIRDataFrame) -> FHIRDataFrame, select_users(df, user_ids: List[str]) -> FHIRDataFrame, aggregate_daily(df, method: 'sum'|'mean') -> FHIRDataFrame, compute_activity_index(df, window_days: int=7) -> FHIRDataFrame, and score_phq9(df) -> pd.Series[int]. The associated responsibilities are outlier removal, cohort restriction by user or date range, daily sums or means, 7-day moving averages, and domain-specific derived scores such as PHQ-9 total score (Bikia et al., 17 Sep 2025).

The exploration layer provides high-level plotting routines for time series, distributions, and multi-user overlays, together with specialized ECG visualization including multi-lead line plots and annotation overlays. The export layer writes tables to CSV or XLSX and saves figures to PNG, TIFF, or JPEG, while guaranteeing that exports retain mapping back to original FHIR resource IDs for auditability. A common misconception is that FHIR-native data are already analysis-ready; the explicit presence of flattening, processing, and export modules suggests otherwise, because the toolkit treats hierarchical standards compliance and analytical usability as distinct stages (Bikia et al., 17 Sep 2025).

4. End-to-end workflow and computational properties

The documented end-to-end workflow consists of five stages. First, a FirebaseDataAccessClient is instantiated with credentials_path and a fhir_endpoint, and connect() is called. Second, fetch_resources("Observation", filters={"code": ["heart-rate", "ecg-apple-watch"]}) retrieves raw observation resources. Third, ObservationFlattener().flatten(raw_observations) converts them into a table. Fourth, a DataProcessor is used in a chained pipeline with .pipe(...) to apply filter_outliers, select_users, and aggregate_daily. Fifth, DataExplorer().plot_time_series(...) generates a figure, and DataExporter(df_clean) persists the table and figure through to_csv(...) and save_figure(...) (Bikia et al., 17 Sep 2025).

The performance characterization in the source is asymptotic rather than benchmark-driven, except for the PAWS case described separately. Fetching $N$ resources scales roughly as $O(N)$ in network I/O. Flattening and parsing are $u$0 per resource, hence $u$1 overall. Pandas operations such as filtering and groupby-aggregation are stated to be on the order of $u$2 to $u$3 depending on aggregation strategy, with memory $u$4. Visualization rendering cost is $u$5 in drawing elements (Bikia et al., 17 Sep 2025).

These complexity statements position the toolkit as a pragmatic analysis layer rather than a high-throughput distributed system. This suggests that the intended operating regime is research and translational workflows in which standards compliance, auditability, and iteration speed are primary design constraints. The paper’s own phrasing—streamlining workflows from secure access through export—supports that reading (Bikia et al., 17 Sep 2025).

5. PAWS at Stanford: Apple Watch ECG workflow

A concrete application is provided through the Pediatric Apple Watch Study (PAWS) at Stanford University. PAWS enrolled at least 100 pediatric patients, ages 6–18, who were undergoing clinical arrhythmia monitoring. Over 4,000 Apple Watch ECG recordings were captured and stored as FHIR Observation resources in Firebase (Bikia et al., 17 Sep 2025).

Within this study, the pipeline was used for secure retrieval of ECG and vital-sign data, followed by flattening into a table with columns including (patient_id, resource_id, timestamp, hr, waveform[], aw_classification, …). The resulting data were uploaded into a clinician dashboard implemented as a Python notebook. That dashboard displayed each ECG trace side by side with traditional monitor output and supported point-and-click annotation, including rhythm diagnosis such as Normal, SVT, and AF, quality scores from 1 to 5, and free-text notes. The annotations were then persisted back to Firestore as FHIR Observation resources with extension fields (Bikia et al., 17 Sep 2025).

The study description also reports comparative and descriptive statistics. The paper notes the distribution of the number of ECG recordings per subject, “time in study” in weeks per subject, and event-based comparisons involving sensitivity and specificity of Apple Watch versus the gold standard, with the latter characterized as ongoing. The quantitative summary reports more than 100 subjects, more than 4,000 ECGs collected, a median of approximately 40 ECGs per subject, and an end-to-end refresh, flatten, and plot time of less than 2 minutes for the full dataset on a standard laptop (Bikia et al., 17 Sep 2025).

In this setting, the pipeline’s role was not limited to batch extraction. It mediated an iterative loop among secure retrieval, transformation, clinician review, annotation, and comparison with traditional monitors. A plausible implication is that the system is intended to support both retrospective analysis and annotation-centric translational workflows in which clinicians and data scientists operate on the same FHIR-grounded substrate (Bikia et al., 17 Sep 2025).

6. Extensibility and relation to the Stanford Spezi ecosystem

The Spezi Data Pipeline is described as one component of the broader Stanford Spezi ecosystem and is identified as MIT-licensed. The 2025 paper presents the Python toolkit as plug-and-play extensible: new Data Access backends can be added by implementing the DataAccessClient interface for AWS HealthLake, Azure Health Data Services, or on-premise FHIR servers; new FHIR resources can be supported by subclassing ResourceFlattener for resources such as ImagingStudy and DeviceMetric; and new risk calculators can be registered through entry_points so that processor.score('myquestionnaire') dispatches to custom code. The source also notes a GitHub template repository for scaffolding a new spezi_data_pipeline plugin for domain-specific needs (Bikia et al., 17 Sep 2025).

The broader Spezi ecosystem had earlier been characterized as modular and standards-based, organized around a small “core” framework plus optional modules, with a simple Module interface including configure(_:), start(), handle(event:), and stop(). That earlier source explicitly states that its pipeline realization is a “logical extrapolation” built from Spezi’s modular, standards-first design rather than a direct description of the later Python toolkit (Schmiedmayer et al., 2023). Accordingly, the strongest direct claims about Spezi Data Pipeline itself come from the 2025 paper, while the 2023 paper provides ecosystem-level context about modularity, standards, and software reuse (Schmiedmayer et al., 2023).

This relationship clarifies a recurrent ambiguity around the term “Spezi.” In the available literature, Spezi refers both to a broader digital health ecosystem and, in the later work, to a specific data pipeline toolkit within that ecosystem. The 2025 paper locates the pipeline inside Stanford Spezi, while the 2023 paper frames Spezi more generally as a modular and standards-based digital health ecosystem aimed at heterogeneous data acquisition, data standardization, software reuse, security, and privacy considerations (Bikia et al., 17 Sep 2025).

7. Significance, scope, and interpretive boundaries

The stated purpose of the toolkit is to reduce the need for bespoke development while enhancing workflow efficiency in interoperable digital health research. Its standards-first design is anchored in HL7 FHIR representations and in the use of typed validation through fhir.resources; its workflow design spans retrieval, transformation, analysis, visualization, and export; and its demonstrated use case centers on Apple Watch ECG data and clinician-driven review alongside traditional monitors (Bikia et al., 17 Sep 2025).

At the same time, the available description defines its scope with some precision. The pipeline is presented as a toolkit for handling FHIR-based digital health data, not as a replacement for the underlying cloud store, FHIR server, or clinical monitoring devices. It standardizes access and analysis around FHIR resources and preserves mappings back to original resource identifiers for auditability. This suggests that its principal contribution lies in operationalizing interoperable data workflows rather than introducing a new data standard or a new clinical metric (Bikia et al., 17 Sep 2025).

The literature also imposes a boundary on claims about generalization. The real-world evidence presented is the PAWS deployment, and the paper reports that event-based sensitivity and specificity comparisons of Apple Watch versus the gold standard are ongoing rather than complete. Consequently, the documented contribution is strongest at the level of architecture, workflow integration, and research infrastructure, with quantitative operational evidence drawn from a pediatric arrhythmia-monitoring cohort and full-dataset processing on a standard laptop (Bikia et al., 17 Sep 2025).