Medical Question Answering on OralQA-ZH (test)
82.71Endo AccuracyOralAgent
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OralAgentCore Orchestrator=Kimi-k2.5, Protocol=RAG2026.04 | 82.71 | 85.39 | 74.44 | 63.69 | 33.33 | 87.04 | 82.14 | 83.33 | 60.98 | 62.07 | 67.5 | 73.31 | |
| OralAgentCore Orchestrator=GPT-5.4, Protocol=RAG2026.04 | 81.2 | 92.13 | 85 | 77.98 | 41.67 | 88.89 | 85.71 | 75 | 78.05 | 79.31 | 87.5 | 82.08 | |
| OralAgentCore Orchestrator=MiniMax-M2.7-Guan, Protocol=RAG2026.04 | 81.2 | 91.01 | 84.44 | 69.05 | 25 | 92.59 | 85.71 | 75 | 75.61 | 86.21 | 80 | 79.45 | |
| OralAgentCore Orchestrator=Qwen3.5-27B, Protocol=RAG2026.04 | 79.7 | 92.13 | 80 | 71.43 | 41.67 | 88.89 | 78.57 | 83.33 | 65.85 | 86.21 | 77.5 | 78.32 | |
| OralAgentCore Orchestrator=Grok-4, Protocol=RAG2026.04 | 78.2 | 86.52 | 81.11 | 72.62 | 37.5 | 90.74 | 89.29 | 83.33 | 68.29 | 82.76 | 82.5 | 78.57 | |
| OralAgentCore Orchestrator=Gemini-3.1-pro-thinking, Protocol=RAG2026.04 | 74.53 | 88.05 | 81.71 | 49.52 | 34 | 86.05 | 82.07 | 41.04 | 44.39 | 57.01 | 53.08 | 69.02 | |
| GPT-5.4Protocol=Individual LLM Baseline2026.04 | 74.44 | 89.89 | 76.11 | 66.07 | 37.5 | 90.74 | 82.14 | 58.33 | 63.41 | 75.86 | 82.5 | 74.69 | |
| OralAgentCore Orchestrator=Lingshu-32B, Protocol=RAG2026.04 | 74.44 | 87.64 | 73.89 | 61.9 | 25 | 83.33 | 71.43 | 83.33 | 70.73 | 65.52 | 67.5 | 71.43 | |
| Grok-4Protocol=Individual LLM Baseline2026.04 | 73.68 | 85.39 | 77.78 | 65.48 | 45.83 | 81.48 | 78.57 | 58.33 | 60.98 | 82.76 | 80 | 73.81 | |
| Qwen3.5-27BProtocol=Individual LLM Baseline2026.04 | 72.93 | 71.91 | 82.78 | 76.19 | 33.33 | 88.89 | 78.57 | 58.33 | 60.98 | 86.21 | 75 | 75.56 | |
| OralAgentCore Orchestrator=Hulu-Med-14B, Protocol=RAG2026.04 | 72.18 | 86.52 | 74.44 | 60.12 | 41.67 | 81.48 | 57.14 | 66.67 | 70.73 | 68.97 | 75 | 70.8 | |
| OralAgentCore Orchestrator=MedDr-42B, Protocol=RAG2026.04 | 71.43 | 77.53 | 64.44 | 55.95 | 25 | 81.48 | 57.14 | 75 | 63.41 | 75.86 | 70 | 65.79 | |
| Lingshu-32BProtocol=Individual LLM Baseline2026.04 | 69.92 | 82.02 | 58.89 | 59.52 | 25 | 77.78 | 78.57 | 58.33 | 53.66 | 65.52 | 67.5 | 64.79 | |
| MedDr-42BProtocol=Individual LLM Baseline2026.04 | 68.42 | 67.42 | 56.67 | 54.76 | 25 | 70.37 | 60.71 | 50 | 56.1 | 65.52 | 70 | 60.4 | |
| OralAgentCore Orchestrator=GPT-5-nano, Protocol=RAG2026.04 | 66.92 | 83.15 | 65.56 | 57.74 | 29.17 | 79.63 | 57.14 | 50 | 68.29 | 79.31 | 82.5 | 66.92 | |
| OralAgentCore Orchestrator=HuatuoGPT-V-34B, Protocol=RAG2026.04 | 65.41 | 67.42 | 54.44 | 54.17 | 33.33 | 72.22 | 35.71 | 41.67 | 65.85 | 62.07 | 70 | 59.02 | |
| GPT-5-nanoProtocol=Individual LLM Baseline2026.04 | 63.91 | 80.9 | 60 | 51.19 | 25 | 77.78 | 60.71 | 50 | 58.54 | 72.41 | 80 | 62.53 | |
| Kimi-k2.5Protocol=Individual LLM Baseline2026.04 | 62.41 | 65.17 | 59.44 | 57.74 | 29.17 | 75.93 | 82.14 | 75 | 68.29 | 34.48 | 70 | 61.53 | |
| HuatuoGPT-V-34BProtocol=Individual LLM Baseline2026.04 | 58.65 | 67.42 | 47.78 | 45.83 | 25 | 61.11 | 35.71 | 33.33 | 51.22 | 62.07 | 72.5 | 52.88 | |
| MiniMax-M2.7-GuanProtocol=Individual LLM Baseline2026.04 | 57.89 | 62.92 | 56.11 | 50 | 25 | 59.26 | 53.57 | 33.33 | 34.15 | 48.28 | 57.5 | 53.38 | |
| Gemini-3.1-pro-thinkingProtocol=Individual LLM Baseline2026.04 | 55.64 | 71.91 | 69.44 | 31.55 | 0 | 90.74 | 71.43 | 58.33 | 19.51 | 3.45 | 12.5 | 50.88 | |
| GLM-4.7-GuanProtocol=Individual LLM Baseline2026.04 | 41.35 | 22.47 | 18.89 | 17.86 | 0 | 9.26 | 14.29 | 33.33 | 34.15 | 55.17 | 40 | 24.81 | |
| OralAgentCore Orchestrator=GLM-4.7-Guan, Protocol=RAG2026.04 | 36.84 | 58.43 | 53.89 | 41.67 | 25 | 74.07 | 50 | 50 | 51.22 | 31.03 | 45 | 47.87 | |
| Hulu-Med-14BProtocol=Individual LLM Baseline2026.04 | 30.83 | 31.46 | 47.22 | 54.17 | 29.17 | 24.07 | 10.71 | 58.33 | 53.66 | 75.86 | 62.5 | 43.11 |