Mathematical Reasoning on AIME24 (Average Count of Unique Outputs)
12Average Unique Outputs CountBF16
Evaluation Results
| Method | Links | |
|---|---|---|
| BF16Model=Qwen3-8B, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 12 | |
| BF16Model=Mistral-7B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 12 | |
| BF16Model=Llama-3.1-8B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 9.6 | |
| BF16Model=Qwen3-32B, Configurations=9 runtime configurations (BS=8/16/32; TP=2/4/8)2025.11 | 9 | |
| BIOModel=Mistral-7B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 7.97 | |
| BIOModel=Qwen3-8B, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 7.87 | |
| BIOModel=Llama-3.1-8B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 7.2 | |
| BIOModel=Qwen3-32B, Configurations=9 runtime configurations (BS=8/16/32; TP=2/4/8)2025.11 | 6.9 | |
| BIO+TBIKModel=Qwen3-8B, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 1 | |
| BIO+TBIKModel=Mistral-7B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 1 | |
| BIO+TBIKModel=Llama-3.1-8B-Instruct, Configurations=12 runtime configurations (BS=8/16/32; TP=1/2/4/8)2025.11 | 1 | |
| BIO+TBIKModel=Qwen3-32B, Configurations=9 runtime configurations (BS=8/16/32; TP=2/4/8)2025.11 | 1 |