Question Answering on ChineseSimpleQA
77.6AccuracyKimi-K2 Base
Evaluation results
| Method | Links | |
|---|---|---|
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 77.6 | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 70.9 | |
| DeepSeek-V3Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 68 | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 68 | |
| DeepSeek-R1Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 63.7 | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 61.5 | |
| GPT-4o-05132025.01 | 58.7 | |
| Claude-3.5-Sonnet-10222025.01 | 55.4 | |
| OpenAI-o1-mini2025.01 | 40.3 | |
| HaluSearch (MCTS)Backbone=Qwen2-7B-Instruct2025.01 | 36 | |
| BoNBackbone=Qwen2-7B-Instruct2025.01 | 35 | |
| HaluSearch (MCTS)Backbone=Llama3.1-8B-Instruct2025.01 | 30.5 | |
| SCBackbone=Qwen2-7B-Instruct2025.01 | 30.5 | |
| Self-RefineBackbone=Qwen2-7B-Instruct2025.01 | 30.5 | |
| BoNBackbone=Llama3.1-8B-Instruct2025.01 | 29 | |
| CoTBackbone=Qwen2-7B-Instruct2025.01 | 29 | |
| Direct GenerationBackbone=Qwen2-7B-Instruct2025.01 | 28.5 | |
| SCBackbone=Llama3.1-8B-Instruct2025.01 | 21 | |
| CoTBackbone=Llama3.1-8B-Instruct2025.01 | 19.5 | |
| Self-RefineBackbone=Llama3.1-8B-Instruct2025.01 | 17.5 | |
| Direct GenerationBackbone=Llama3.1-8B-Instruct2025.01 | 15.5 |