Medical Response Refinement on HealthBench 254 medical queries
59Base ScoreSelf-Critique
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Self-CritiqueGuidance Source=Intrinsic self-correction (No external rubric), Role=Lower-bound control, Base Response Model=Llama-3.1-8B-Instant, Evaluator Model=Llama-3.3-70B, Editor Model=Llama-3.1-8B, Protocol=Critique-then-Refine2026.01 | 59 | 64.4 | 5.5 | 25 | — | |
| GPT-4o RubricsGuidance Source=GPT-4o generated rubrics, Base Response Model=Llama-3.1-8B-Instant, Evaluator Model=Llama-3.3-70B, Editor Model=Llama-3.1-8B, Protocol=Critique-then-Refine2026.01 | 59 | 65.7 | 6.7 | 33.2 | — | |
| Our RubricsGuidance Source=Proposed instance-specific rubrics, Base Response Model=Llama-3.1-8B-Instant, Evaluator Model=Llama-3.3-70B, Editor Model=Llama-3.1-8B, Protocol=Critique-then-Refine2026.01 | 59 | 68.2 | 9.2 | 35.8 | — | |
| Reference RubricsGuidance Source=Expert physician-authored rubrics, Role=Oracle Upper Bound, Base Response Model=Llama-3.1-8B-Instant, Evaluator Model=Llama-3.3-70B, Editor Model=Llama-3.1-8B, Protocol=Critique-then-Refine2026.01 | 58.9 | 74.1 | 15.2 | 54.8 | — |