Scientific Reasoning on SuperGPQA (Accuracy)
44.7AccuracyKimi-K2 Base
Evaluation Results
| Method | Links | |
|---|---|---|
| Kimi-K2 Base# Shots=5-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 44.7 | |
| DeepSeek-V3.2 Exp Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 43.6 | |
| DeepSeek-V3.1 Base# Shots=5-shot, # Activated Params=37B, # Total Params=671B2026.01 | 42.3 | |
| MiMo-V2-Flash Base# Shots=5-shot, # Activated Params=15B, # Total Params=309B2026.01 | 41.1 | |
| Jet-RLModel=Qwen3-8B-Base, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 35.2 | |
| BF16Model=Qwen3-8B-Base, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 35.1 | |
| InitialModel=Qwen3-8B-Base, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 31.8 | |
| BF16-Train-FP8-RolloutModel=Qwen3-8B-Base, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 30.3 | |
| BF16Model=Qwen2.5-7B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 28.6 | |
| Jet-RLModel=Qwen2.5-7B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 28.5 | |
| InitialModel=Qwen2.5-7B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 25.5 | |
| Jet-RLModel=Llama3.1-8B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 19.9 | |
| BF16Model=Llama3.1-8B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 15.9 | |
| BF16-Train-FP8-RolloutModel=Llama3.1-8B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 14.7 | |
| InitialModel=Llama3.1-8B, Rollout Length=8k, Training Dataset (RL)=GSM8k + MATH2026.01 | 12.4 |