Unified Patient Records: A Data
Engineering Playbook for Healthcare Interoperability

A patient with a chronic condition might interact with a primary care physician, two specialists, a diagnostic lab, a pharmacy, a physical therapist, and a remote monitoring device — all in a single year. Each of these touchpoints generates clinical data. In a well-functioning system, that data would flow together into a coherent longitudinal record that any treating clinician could access. In practice, it sits in six different systems, in four different formats, with the patient identified by three different record numbers and two different name spellings.

Unifying this data is one of the hardest problems in healthcare data engineering — harder than most domains not because the volumes are extreme, but because the stakes of getting it wrong are clinical. A duplicate patient record that results in a missed allergy alert is not a data quality incident. It is a patient safety failure. The engineering discipline required to build reliable unified patient records reflects this: the tolerance for error is lower, the regulatory environment is stricter, and the data is structurally more complex than in almost any other industry.

The Heterogeneous Ingestion Problem

Healthcare data arrives in formats that span five decades of standards evolution. Legacy hospital systems emit HL7 version 2 messages — a pipe-delimited format that dates to 1987 and remains the most widely deployed clinical messaging standard in the world. Modern systems expose FHIR R4 APIs. Imaging systems use DICOM. Payer data arrives in X12 EDI transactions. Wearables and remote monitoring devices produce streams of time-series data in proprietary formats. A unified patient record platform must ingest all of these without losing clinical fidelity.

Ingestion reality:  HL7 v2 is deceptively difficult. The standard allows extensive local customisation through Z-segments, and two health systems sending the same message type will frequently produce structurally different messages. Parser configuration is a per-source engineering task, not a one-time implementation.

• HL7 v2 to FHIR transformation is the most common ingestion challenge — mapping ADT messages (admissions, discharges, transfers), ORU messages (observation results), and ORM messages (orders) to their FHIR equivalents requires both technical translation and clinical terminology normalisation
• DICOM metadata — patient demographics, study descriptions, series information — should be extracted and linked to the corresponding FHIR ImagingStudy resource, enabling the patient record to surface imaging history even when the image files themselves remain in the PACS
• Wearable and remote monitoring data requires a stream ingestion architecture — Kafka or Kinesis — capable of handling high-frequency writes, with FHIR Observation resources as the target model and anomaly detection at the ingestion boundary to flag physiologically implausible readings before they enter the record

Terminology normalisation is the ingestion step most commonly underestimated. A diagnosis coded as ICD-9 in a legacy record must be mapped to ICD-10. A medication recorded as a free-text string must be resolved to an RxNorm concept. A lab result described with a local LOINC variant must be mapped to the canonical LOINC code. Without this normalisation, queries across sources — find all patients with a diagnosis of Type 2 diabetes — return incomplete results, and the unified record is unified in name only.

Probabilistic Patient Matching: The Identity Resolution Problem

The most technically distinctive challenge in healthcare data engineering is patient matching — determining, across multiple source systems with no shared identifier, whether two records refer to the same person. Unlike B2C identity resolution, healthcare patient matching cannot rely on email addresses or device fingerprints. It must work from demographic attributes — name, date of birth, address, phone number, gender — that are frequently incomplete, inconsistently formatted, and subject to change.

Deterministic matching — linking records only when a unique identifier like an MRN or SSN matches exactly — achieves high precision but misses the substantial portion of records that lack a shared identifier across systems. Probabilistic matching uses a weighted scoring algorithm across multiple demographic attributes, assigning match confidence based on the statistical likelihood that two records with a given attribute overlap refer to the same person.

• The Fellegi-Sunter model and its variants are the academic foundation for probabilistic patient matching — machine learning implementations trained on labelled match/non-match pairs have improved on these models in practice, particularly for name matching across cultural variations
• Blocking strategies — limiting the candidate pairs that are scored to those sharing at least one attribute — are essential for computational tractability at scale; scoring every record against every other record is infeasible beyond tens of thousands of patients
• Match confidence thresholds require clinical input, not just statistical tuning: the threshold above which records are automatically linked, and below which they are routed for human review, has patient safety implications that data engineers alone should not determine

Operational requirement:  A patient matching system must include a human review workflow for records that fall below the auto-link threshold — and an audit trail of every match decision, whether automatic or human-reviewed, that is queryable when a matching error is suspected.

HIPAA-Grade Access Control at the Data Layer

Unified patient records concentrate sensitive PHI in a way that makes access control both more important and more complex than in systems where data remains siloed. The access model must enforce HIPAA’s minimum necessary standard — the principle that each user, application, and process should have access only to the specific PHI required for their defined purpose — at the data layer, not just at the application layer.

Attribute-based access control (ABAC) is the framework that makes this tractable at scale. Rather than assigning access through static role definitions, ABAC evaluates access decisions dynamically based on attributes of the user (role, department, assigned patients), the resource (patient consent status, data sensitivity classification, record type), and the context (treatment relationship, time of access, purpose code). This allows the access model to enforce nuanced clinical access patterns — a treating clinician has broad access to their patient’s record, but a clinician without a documented treatment relationship does not — without requiring a separate access configuration for every combination of user and resource.

• Patient consent management — recording and enforcing patient preferences about who can access their data and for what purposes — must be integrated into the access control layer, not managed as a separate process downstream
• Break-glass access — the mechanism by which a clinician can access a record outside their normal access scope in an emergency — must be logged comprehensively, reviewed systematically, and designed to create friction without creating barriers to emergency care
• Data access auditing must operate at the field level for the most sensitive PHI categories — substance abuse records, mental health records, and reproductive health records have stricter access controls under 42 CFR Part 2 and state laws that require finer-grained audit capability than standard HIPAA logging

The Engineering Investment That Justifies Itself

Building unified patient record infrastructure is expensive, slow, and unglamorous work. The ingestion pipelines that handle HL7 v2 edge cases, the matching algorithms that require ongoing tuning as new sources are added, the access control layer that must satisfy both regulatory auditors and clinical workflow requirements — none of this generates a demo-able feature. But it is the infrastructure that makes every downstream use case — clinical decision support, population health analytics, AI-assisted diagnosis — possible and trustworthy.

The organisations that invest in getting this foundation right — clean ingestion, reliable matching, rigorous access control — are the ones whose clinical data assets are worth building on. The ones that treat it as a commodity integration problem find themselves repeatedly rebuilding from incomplete and untrustworthy data, at progressively higher cost.

At Nineleaps, we help healthcare organisations build the data engineering infrastructure that makes unified patient records real — from heterogeneous ingestion pipelines to probabilistic matching and HIPAA-grade access control.