Every AI predictive model built on health data depends on knowing what happened to the individuals in its training data. One outcome is often missing: whether the individual died.
Electronic health records (EHR) and claims data record episodes of care, not deaths. When an individual dies at home, in hospice, or in another health system, the record usually just stops, with nothing to show that the stop was a death. A model can't tell an individual who died from one who stopped showing up. It treats both the same way, and that is where bias enters into training models and the outcomes it will predict for cohorts.
The problem with proxy signals
Data teams often fill the gap with proxy signals: no encounters for 12 months, a canceled enrollment, or a closed claims history. The problem is that each one is a guess. An individual who moves to a new health system looks the same as one who died. Someone who stops filling prescriptions may have switched insurance plans. And an individual who dies the week after their last encounter looks healthy in the data. When proxy signals become outcome labels, the model learns to predict the proxy.
All RWE datasets for claims and EHR can have mortality record gaps. Using gapped data to train models risks the inaccuracy propagating to expected outcomes.
Four ways missing deaths distort a model
Unrecorded deaths affect a model at every stage, from training to production:
- Training labels are wrong. A model that counts an unrecorded death as a survivor learns that a fatal risk profile is safer than it is. Models that predict other outcomes are affected too. A readmission model counts a death after discharge as "not readmitted," which looks like a success in the data
- Validation rewards the wrong answer. When the validation set has the same gaps as the training set, a model that misses deaths still scores well. Performance looks stable while the model is wrong in the cases that matter most
- The error concentrates in specific cohorts. Deaths go unrecorded more often for individuals with fragmented care: those who switch payers, use multiple health systems, or die outside a hospital. Training on these labels lowers performance for exactly those groups, and a fairness audit run on the same incomplete labels won't catch it
- Deployed models drift from reality. Without current mortality data, a deployed model keeps scoring individuals who have died. Outreach lists, risk scores, and capacity forecasts carry the error forward
Defining data freshness for your training use case
No single freshness standard exists for mortality data. The right standard depends on what the model does:
- Training a model on historical data: Prioritize completeness, so every death in the cohort is captured with a date.
- Validating or benchmarking a model: Prioritize completeness, plus a verified source for every death.
- Scoring a live population with a deployed model: Prioritize recency, so deaths are identified within weeks and updated on a regular schedule.
A training set built once can tolerate some lag. A model that re-scores or refines a population every month cannot.
Preventing the mortality gap
The fix is to link your data to a mortality source built for completeness and recency:
- Match on tokens. Privacy-preserving tokens let you link a deidentified cohort to mortality data without exchanging PHI.
- Use confirmed deaths. Use date of death as the outcome label, and add cause of death when the model needs to separate all-cause from condition-specific outcomes.
- Re-validate. Re-run performance and subgroup metrics on the corrected labels. The difference shows the size of the bias you were carrying.
- Monitor in production. For deployed models, refresh mortality data on a schedule that matches how often the model scores.
If your model is trained or validated on health data, the first question is how many deaths it is missing.
Talk to our team to find out.
