Benchmarks

Task NameDataset NameSOTA ResultTrendResults

Mortality Prediction

MIMIC-III target v1.4

16.95AUPRC

ICD coding

MIMIC-III Full v1.4 (test)

0.123Macro F1

In-hospital mortality prediction

MIMIC-III Hypertension

47.27AUPRC

Irregularly Sampled Time Series Forecasting

MIMIC-III 75%-3 task (test)

0.396MSE

Irregularly Sampled Time Series Forecasting

MIMIC-III 50%-50% task (test)

0.394Inference Time (s)

Decompensation Prediction

MIMIC-III (test)

35.5AUPRC

Text-to-UMLS linking

MIMIC-III unseen in WS n=174 (test)

94.1Precision

In-hospital mortality prediction

MIMIC-III continual protocol A (time-window shift by admission year)

0.86Full-data UB Score

Text-to-UMLS

MIMIC-III Radiology n=53+/198 (External val)

89.4Precision

Off-Policy Evaluation

MIMIC-III sepsis MNAR 40% (test)

9.55V_hat(pi)

Off-Policy Evaluation

MIMIC-III sepsis 40% MNAR 20% (test)

10,408.2V_hat(pi)

Multi-step-ahead prediction

MIMIC-III Cardiovascular African

4.71RMSE ($ au=1$)

Multi-step-ahead prediction

MIMIC-III African subgroup OOD infectious and inflammatory diseases (out of distribution)

4.92RMSE (τ=1)

Counterfactual Average Path Outcome (CAPO) Estimation

Semi-synthetic MIMIC-III Expanded Time-varying Confounding (test)

0.53Bias (CF 1)

ICD-9 Code Prediction

MIMIC-III v1.4 (test)

91.58Accuracy (140-239)

Mortality Prediction

MIMIC-III v1.4 (test)

78.1Accuracy

Hospital Course Summarization

MIMIC-III-ext-VeriFact-BHC 120 prompts (test)

10.7Local Verifier NS-rate

Text-to-UMLS linking

MIMIC-III n=400 (val)

90.1Precision

Mechanical Ventilation Prediction

MIMIC-III 1.4 (test)

87.1Onset AUROC

Multi-task prediction

MIMIC-III ICD-9 20 tasks (test)

71.23Average AUC

Phenotyping Prediction

MIMIC-III All patients

49.96Acute Renal Failure Score

Mortality prediction

MIMIC-III v1.4 (1000 patients)

13.5AUPRC

Rare Disease Identification (Text-to-ORDO)

MIMIC-III discharge summaries (val)

67.6Precision

Text-to-UMLS linking

MIMIC-III seen in WS n=499 (test)

83.3Precision

In-ICU Mortality Prediction

MIMIC-III MIMIC-Extract pipeline (test)

89.7AUROC