Medical Knowledge Evaluation on Medical Knowledge Item Bank CAT
73Test Lengtho1
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| o1evaluation_protocol=CAT2026.02 | 73 | 2.42 | 90.41 | 75,008 | 790.05 | |
| gemini-2.5-proevaluation_protocol=CAT2026.02 | 70 | 2.34 | 87.14 | 108,122 | 1,116.99 | |
| gemini-proevaluation_protocol=CAT2026.02 | 69 | 2.36 | 89.86 | 107,350 | 1,043.98 | |
| deepseek-r1evaluation_protocol=CAT2026.02 | 59 | 2.25 | 89.83 | 107,159 | 3,241.27 | |
| gemini-2.5-flashevaluation_protocol=CAT2026.02 | 55 | 2.09 | 89.09 | 68,188 | 429.8 | |
| claude-4-sonnet-thinkingevaluation_protocol=CAT2026.02 | 54 | 2.17 | 85.19 | 39,635 | 1,068.06 | |
| gpt-4oevaluation_protocol=CAT2026.02 | 52 | 1.7 | 86.54 | 13,003 | 48.47 | |
| gpt-oss-120bevaluation_protocol=CAT2026.02 | 50 | 1.97 | 80 | 39,114 | 932.48 | |
| glm-4.5-airevaluation_protocol=CAT2026.02 | 49 | 1.57 | 83.67 | 94,582 | 1,756.69 | |
| minimax-m1evaluation_protocol=CAT2026.02 | 48 | 1.83 | 75 | 133,680 | 6,566.4 | |
| qwen3-30b-a3bevaluation_protocol=CAT2026.02 | 45 | 1.55 | 75.56 | 111,964 | 886.19 | |
| o1-minievaluation_protocol=CAT2026.02 | 42 | 1.43 | 88.1 | 30,748 | 158.63 | |
| kimi-k2evaluation_protocol=CAT2026.02 | 41 | 1.43 | 75.61 | 10,809 | 101.51 | |
| gemini-2.0-flashevaluation_protocol=CAT2026.02 | 41 | 1.43 | 75.61 | 10,811 | 36.51 | |
| gpt-4evaluation_protocol=CAT2026.02 | 38 | 1.31 | 78.95 | 9,688 | 65.76 | |
| gpt-oss-20bevaluation_protocol=CAT2026.02 | 37 | 1.37 | 78.38 | 27,882 | 138.6 | |
| qwen2.5-72b-instructevaluation_protocol=CAT2026.02 | 32 | 0.84 | 56.25 | 8,448 | 13.64 | |
| qwen-maxevaluation_protocol=CAT2026.02 | 31 | 0.73 | 64.52 | 8,076 | 22.05 | |
| glm-3-turboevaluation_protocol=CAT2026.02 | 30 | 1.03 | 73.33 | 7,693 | 32.33 | |
| yi-lightningevaluation_protocol=CAT2026.02 | 30 | 1 | 73.33 | 9,045 | 40.46 | |
| yi-sparkevaluation_protocol=CAT2026.02 | 30 | 0.85 | 63.33 | 9,045 | 43.31 | |
| yi-mediumevaluation_protocol=CAT2026.02 | 29 | 0.66 | 68.97 | 8,521 | 47.81 |