Accuracy and F1 on MedQA (Medical Reasoning)
83.76AccuracyCOTCAgent
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| COTCAgentBackbone=Baichuan-M22026.05 | 83.76 | 82.9 | |
| KAREBackbone=Baichuan-M22026.05 | 82.84 | 81.7 | |
| API toolsBackbone=Baichuan-M22026.05 | 82.33 | 80.6 | |
| COTCAgentBackbone=Gemini 2.5 Pro2026.05 | 81.76 | 80.2 | |
| Google'sBackbone=Baichuan-M22026.05 | 81.66 | 79.7 | |
| COTCAgentBackbone=Claude 3.7 Sonnet2026.05 | 81.48 | 80.1 | |
| COTCAgentBackbone=GPT-4o2026.05 | 81.14 | 79.9 | |
| DirPredBackbone=Baichuan-M22026.05 | 80.76 | 78.8 | |
| TimeCAPBackbone=Baichuan-M22026.05 | 79.38 | 77.7 | |
| KAREBackbone=Gemini 2.5 Pro2026.05 | 79.28 | 77.9 | |
| KAREBackbone=Claude 3.7 Sonnet2026.05 | 78.96 | 77.5 | |
| API toolsBackbone=Gemini 2.5 Pro2026.05 | 78.84 | 78.1 | |
| KAREBackbone=GPT-4o2026.05 | 78.76 | 76.8 | |
| API toolsBackbone=Claude 3.7 Sonnet2026.05 | 78.51 | 77.4 | |
| API toolsBackbone=GPT-4o2026.05 | 78.29 | 76.6 | |
| Google'sBackbone=Gemini 2.5 Pro2026.05 | 78.16 | 76.9 | |
| Google'sBackbone=Claude 3.7 Sonnet2026.05 | 77.73 | 76.1 | |
| Google'sBackbone=GPT-4o2026.05 | 77.59 | 75.8 | |
| DirPredBackbone=Gemini 2.5 Pro2026.05 | 77.51 | 75.8 | |
| DirPredBackbone=Claude 3.7 Sonnet2026.05 | 77.36 | 75.6 | |
| DirPredBackbone=GPT-4o2026.05 | 77.08 | 74.8 | |
| TimeCAPBackbone=Gemini 2.5 Pro2026.05 | 76.04 | 74.7 | |
| TimeCAPBackbone=Claude 3.7 Sonnet2026.05 | 75.98 | 74.6 | |
| TimeCAPBackbone=GPT-4o2026.05 | 75.71 | 73.6 | |
| COTCAgentBackbone=Qwen3-Next-80B-A3B2026.05 | 74.06 | 71.9 | |
| KAREBackbone=Qwen3-Next-80B-A3B2026.05 | 72.14 | 70.8 | |
| API toolsBackbone=Qwen3-Next-80B-A3B2026.05 | 71.53 | 70.3 | |
| Google'sBackbone=Qwen3-Next-80B-A3B2026.05 | 71.28 | 69.7 | |
| DirPredBackbone=Qwen3-Next-80B-A3B2026.05 | 70.62 | 69.1 | |
| TimeCAPBackbone=Qwen3-Next-80B-A3B2026.05 | 69.37 | 68.1 |