HealthBench
Benchmarks
TaskDatasetSOTA resultTrendResults
HealthBench Hard
40.74Accuracy
41
HealthBench (offline)
31.79Accuracy
38
HealthBench
59.5Score
38
HealthBench Medicine N=5,000 (overall)
26.1Rubric Score
36
HealthBench
70.41Accuracy
36
HealthBench Main
46.38Average Score
24
HealthBench ResearchQA DRB Macro Average
62.8Average Score
21
HealthBench Hard
39.02Accuracy
19
HealthBench 500
65.7Score
18
Healthbench Hard
28.06Performance
18
HealthBench
37.2Accuracy
17
HealthBench
44.92Accuracy
17
HealthBench Overall
60.1Overall Score
16
HealthBench held-out heart-related n=500 seed 42
57Accuracy
15
HealthBench English (test)
32.7Overall Score
15
HealthBench treatment-related conversations
48.94Overall Score
15
HealthBench-500
43.6Score
15
HealthBench In-domain, Seen
22.97Score
14
HealthBench
59.5Overall Score
14
HealthBench Hard 1000
86Accuracy
12
HealthBench
54Score
11
HealthBench Hard
46.2Total Score
11
HealthBench Professional (525 cases)
62.72Overall Score
11
HealthBench
92.82Pass@1
11
HealthBench (test)
62.6HealthBench Score (%)
11