MIMIC-III/NHANES/ICES → Early-Death and Disease Oracle Harness

01Load the data

Select the CSV files from your archive.zip (at minimum PATIENTS.csv and ADMISSIONS.csv; structured_medical_records.csv is optional and used only to read the stated Age). Files are parsed locally in your browser — nothing is uploaded.

NHANES mode (free, no-application data). This harness also reads NHANES .XPT files directly (SAS-transport, parsed in-browser). Drop a demographics file (DEMO_*.xpt, with SEQN/RIDAGEYR/RIAGENDR), a prescriptions file (RXQ_RX_*.xpt, RXDDRUG), the medical-conditions questionnaire (MCQ_*.xpt/DIQ/KIQ for routing), and a linked-mortality file (SEQN+MORTSTAT+PERMTH_EXM); it auto-detects NHANES, routes by self-reported condition, and runs the same standard-of-care vs Pareto comparison on an ambulatory population (general-population survival baseline). You download the files from CDC and drop them here — the tool can't fetch wwwn.cdc.gov directly (cross-origin). NHANES III fixed-width files (e.g. adult.dat) also load: drop the data file together with its SAS layout (adult.sas) — the tool parses the INPUT column positions and LABELs, then routes by condition label. (Bring one fixed-width file + its .sas per load; prescriptions/mortality can come from .XPT or CSV. Continuous cycles (1999+) are all-.XPT and need no layout.)

Harmonized NHANES 1988–2018 (figshare/Kaggle, Nguyen et al.) — drop the raw modules directly. Download the cleaned demographics, questionnaire (or response), mortality and medications module CSVs and drop all of them here at once. The tool now streams each file and keeps only the handful of columns it needs (SEQN, age, sex, self-reported conditions, MORTSTAT/PERMTH, drug names), so the 1000+-column questionnaire file loads without overwhelming the browser; it then merges them on SEQN and runs the analysis. If the medications module stores drug codes, also drop dictionary_drug_codes.csv. No Python, no fetching — still fully local. Drop the response module too and the harness computes a clinical stage from the labs (kidney/liver/metabolic/pulmonary) for §04b; otherwise stage shows NA-stage. If a file's columns aren't recognised, the load status lists exactly what each file was read as, so a name mismatch is visible rather than silent.

ICES mode (Ontario administrative data — secure analytic environment). For population-scale Ontario data, build a single joined extract inside the ICES Data & Analytic Services (DAS) environment — one row per person keyed on the encrypted IKN (which replaces SEQN) — then drop that CSV here and run in Batched (3M+) mode (§01). Map the DAS holdings to the harness inputs: demographics + death from RPDB; cause of death from ORGD; condition routing from validated cohorts (ODD diabetes, CHF, COPD, ASTHMA, HYPER, DEMENTIA, ORAD, HIV, OCCC) plus OCR (cancer) and diagnoses in DAD/NACRS/OHIP; staging labs from OLIS; medications from ODB (65+/ODSP/OW/LTC) + NMS (all-payer controlled substances). Carry cohort flags and OLIS labs under the harness’s canonical column names (e.g. OSQ060, LBXSCR, LBXGH) so routing and §04b/§04c staging run unchanged. This ascertainment is whole-population, longitudinal and algorithm-validated — better than self-report. Cells representing <6 people display as “Too Low”, matching ICES release rules. Caveats: ODB is age-restricted (65+); OLIS labs begin ~Oct 2007 with no spirometry; measured BP/BMI/race are absent from administrative data (link OHS/CLSA). See the companion note ices_harness_port.html for the full mapping.

What the MIMIC 100-patient DEMO archive contains. The open PhysioNet demo has demographics (PATIENTS), admissions with a free-text diagnosis and death flags (ADMISSIONS), labs, and free-text reports, but omits PRESCRIPTIONS, DIAGNOSES_ICD and PROCEDURES_ICD. Consequences for that specific file set: routing uses the free-text admission diagnosis (not ICD codes), and the §06 doctor-prescribed-vs-Pareto comparison stays inactive until a medications table is loaded. It is not a limitation of the harness: §06 computes whenever medications are present — the full credentialed MIMIC PRESCRIPTIONS, NHANES RXQ_RX, or ICES ODB/NMS. In every mode the harness reports each person's maximum-achievable risk reduction and life-years (and the observed outcome where the source is outcome-linked).

02Early-death oracles included

Every atlas oracle whose primary endpoint is mortality / early death is included here. Population-count analyses (us-mortality, self-caused-harm, rare-disease) are excluded. Symptom-scale and biomarker endpoints (osteoarthritis, depression, anxiety, ADHD, schizophrenia, PTSD, OCD, bipolar, LDL cholesterol) have no patient-level early-death endpoint and are handled separately in §08 as non-mortality reductions, not life-years.

03Per-record output (per-person / per-admission rows withheld — enclave release rules)

04Aggregate roll-up (cells representing <6 people display as “Too Low”, mirroring enclave release rules)

04bRoll-up by family × disease × subtype × stage (stage computed from response-module labs where a validated algorithm exists; else NA-stage)

The same statistics as the per-family table above, resolved to a finer grain. Each family (oracle) row carries its family-total record count and statistics; beneath it, every disease within the family — read from the actual NHANES condition flags (e.g. cardiac splits into congestive heart failure, coronary heart disease, angina and heart attack; pulmonary into COPD, emphysema and chronic bronchitis; liver into its reported items) — is broken out, then any recorded subtype (diabetes sub-item or cancer site, where the cycle asks it) and stage. Where a person reports several conditions in one family they are attributed to the highest-priority flag, so per-disease counts sum exactly to the family total. NHANES does not report a clinical stage, so this build computes one from the response-module labs for the four oracles with a validated cross-sectional algorithm: kidney → KDIGO G-stage (CKD-EPI 2021 eGFR), liver → FIB-4 fibrosis band, metabolic → ADA glycaemic stage, pulmonary → GOLD (where spirometry exists). Oracles with no NHANES staging analog (cancer/heart/brain — no TNM, NYHA, or Child-Pugh) remain NA-stage. Computed stages are single-visit categorisations (chronicity unconfirmable), not chronicity-confirmed diagnoses; an explicit variant_stage column still overrides them. Cells with fewer than 6 records are suppressed, mirroring enclave release rules.

04cLab-derived disease staging (whole loaded cohort — independent of self-reported diagnosis)

Every loaded participant who has the required response-module labs is staged across all 18 validated algorithms — not gated by self-report or by routing to a mortality oracle, and not subject to the §04b small-cell suppression. This is the population-level staging the lab data actually supports: a person with eGFR 40 is CKD G3b whether or not they reported kidney disease. Requires demographics + response module CSVs. Stages are single-visit categorisations (chronicity unconfirmable). If the table is empty, the diagnostic line below states why.

05Actual vs usual-care baseline vs Pareto-optimum life-years ("years added")

The usual-care baseline is the standard of care: the empirical survival of patients with this disease who received ordinary treatment, including medications taken in the recent past (e.g. a statin they were already on). It is not an untreated counterfactual — there is no plausible untreated cohort to estimate it from. The Bayesian Pareto optimum is a different, specified set of interventions. So this table compares two regimens — standard of care vs the Pareto-optimum set — and the years added is Pareto-optimum LE − usual-care baseline LE. A high disease-specific acute first-year mortality is carried by both regimens; the Pareto set acts on the modifiable post-acute hazard. Observed actual life-years (from dod − admittime) are shown as the empirical anchor.

What the delta is — and the one caveat that remains. This compares the standard-of-care regimen to the Pareto-optimum set. The Pareto effect is applied multiplicatively to the usual-care hazard, so where the two regimens overlap — e.g. both include a statin — the model still credits that shared intervention, making the headline Δ an upper bound on the incremental gain. The clean, overlap-free version is in §06: the prescribed-vs-Pareto headroom measures the Pareto optimum relative to what the patient was actually given (from PRESCRIPTIONS), so it nets out the standard care already in the baseline. Two further notes: the Pareto set acts only on the post-acute hazard (a statin does not avert acute septic death), and observed actual LY runs below the usual-care baseline here because this demo cohort is selected decedents — the baseline reflects the disease group's realistic standard-of-care expectation, not this biased sample.

06Doctor-prescribed vs Pareto-optimum (requires the PRESCRIPTIONS table)

The demo archive you may have loaded omits PRESCRIPTIONS. Load it (it ships with the full credentialed MIMIC-III, ~4.16M rows, and with the open 100-patient demo on PhysioNet) and this section activates: each admission's ordered drugs are string-matched (DRUG/DRUG_NAME_GENERIC) to the routed oracle's interventions, giving the doctor-prescribed risk reduction, the gap to the Pareto optimum, and a split of life-years into already secured by the prescribed protocol vs remaining headroom.

Interpretation caveats. MIMIC PRESCRIPTIONS are inpatient CPOE orders during the stay — a mix of acute ICU drugs (pressors, sedatives, antibiotics, which map to no prevention oracle) and continued chronic medications (statins, antihypertensives, etc., which do). So the prescribed RR here is a lower bound on the true outpatient regimen and is not adherence-weighted (a single inpatient order ≠ chronic use). Lifestyle and procedural interventions (exercise, diet, weight loss, rehab) never appear in a drug table, so part of the "gap to Pareto" is structurally unmeasurable from prescriptions alone.

07Methods & caveats

08Alternative (non-mortality) endpoints (records with no mortality endpoint)