Medical Question Answering on Medical Benchmarks (MedQA, MedMCQA, BULLET) (test)
0.5533MedQA AccuracyC3oT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| C3oTBackbone=Qwen2.5-7B2026.02 | 0.5533 | 0.615 | 0.3833 | 0.5172 | 430.1 | 297.32 | 429.26 | 385.56 | 1.42 | |
| CoT-ValveBackbone=Qwen2.5-7B2026.02 | 0.55 | 0.61 | 0.4033 | 0.5211 | 564.33 | 411.82 | 584.32 | 520.16 | 1.17 | |
| S3-CoTBackbone=Qwen2.5-7B2026.02 | 0.545 | 0.6017 | 0.3483 | 0.4983 | 302.19 | 192.57 | 309.47 | 268.07 | 1.45 | |
| S3-CoTscBackbone=Qwen2.5-7B2026.02 | 0.5267 | 0.5933 | 0.3383 | 0.4861 | 304.07 | 195.01 | 306.22 | 268.43 | 1.3 | |
| Efficient_pBackbone=Qwen2.5-7B2026.02 | 0.4917 | 0.6033 | 0.3167 | 0.4706 | 118.43 | 62.23 | 107.11 | 95.92 | 1.5 | |
| TokenSKIPBackbone=Qwen2.5-7B2026.02 | 0.4567 | 0.5133 | 0.2317 | 0.4006 | 461.66 | 335.16 | 464.99 | 420.6 | -0.21 | |
| Standard_pBackbone=Qwen2.5-7B2026.02 | 0.4083 | 0.56 | 0.2683 | 0.4122 | 498.3 | 345.58 | 514.86 | 452.91 | — | |
| S3-CoTscBackbone=DeepSeek-R1-7B2026.02 | 0.4 | 0.385 | 0.2817 | 0.3556 | 1,647.53 | 960.44 | 1,555.83 | 1,387.93 | 0.21 | |
| AutoTHINKBackbone=DeepSeek-R1-7B2026.02 | 0.395 | 0.3833 | 0.3133 | 0.3639 | 1,742.88 | 1,117.9 | 1,692.67 | 1,517.82 | 0.26 | |
| S3-CoTBackbone=DeepSeek-R1-7B2026.02 | 0.395 | 0.3667 | 0.3017 | 0.3545 | 1,648.16 | 1,024.41 | 1,608.56 | 1,427.04 | 0.16 | |
| Standard_pBackbone=DeepSeek-R1-7B2026.02 | 0.3867 | 0.3733 | 0.3083 | 0.3561 | 1,865.66 | 1,339.95 | 2,169.36 | 1,791.66 | — | |
| LASER_DEBackbone=DeepSeek-R1-7B2026.02 | 0.3833 | 0.3767 | 0.2933 | 0.3511 | 1,354.27 | 868.28 | 1,286.06 | 1,163.54 | 0.21 | |
| ShorterBetterBackbone=DeepSeek-R1-7B2026.02 | 0.3717 | 0.36 | 0.3047 | 0.3454 | 692.33 | 357.91 | 716.78 | 589 | 0.37 | |
| EffReaBackbone=DeepSeek-R1-7B2026.02 | 0.37 | 0.3633 | 0.3183 | 0.3506 | 1,736.26 | 948.04 | 1,651.76 | 1,445.35 | 0.04 | |
| Efficient_pBackbone=DeepSeek-R1-7B2026.02 | 0.3683 | 0.3433 | 0.275 | 0.3289 | 1,362.7 | 746.79 | 1,297.02 | 1,135.5 | -0.4 | |
| C3oTBackbone=DeepSeek-R1-7B2026.02 | 0.365 | 0.3683 | 0.3267 | 0.3533 | 1,832.41 | 1,265.82 | 1,802.02 | 1,633.42 | 0.01 | |
| LC-R1Backbone=DeepSeek-R1-7B2026.02 | 0.3583 | 0.375 | 0.2917 | 0.3417 | 1,310.58 | 672.8 | 1,248.25 | 1,077.21 | -0.01 | |
| CoT-ValveBackbone=DeepSeek-R1-7B2026.02 | 0.28 | 0.3083 | 0.2467 | 0.2783 | 1,153.82 | 1,676.99 | 918.82 | 1,249.88 | -1.88 |