Language Modeling on HELM macro-averaged (test)
73.2AccuracyZero-Shot Predict
Evaluation Results
| Method | Links | |
|---|---|---|
| Zero-Shot PredictLanguage Model=o3 Mini2025.11 | 73.2 | |
| BFRSLanguage Model=o3 Mini, statistical_significance=true2025.11 | 73.1 | |
| MIPROv2Language Model=o3 Mini, statistical_significance=true2025.11 | 73.1 | |
| Zero-Shot CoTLanguage Model=o3 Mini, statistical_significance=true2025.11 | 72.7 | |
| HELM BaselineLanguage Model=o3 Mini2025.11 | 70.9 | |
| MIPROv2Language Model=Claude 3.7 Sonnet, statistical_significance=true2025.11 | 69.8 | |
| Zero-Shot CoTLanguage Model=Claude 3.7 Sonnet, statistical_significance=true2025.11 | 69.4 | |
| BFRSLanguage Model=Claude 3.7 Sonnet, statistical_significance=true2025.11 | 69.3 | |
| Zero-Shot CoTLanguage Model=Gemini 2.0 Flash, statistical_significance=true2025.11 | 66.2 | |
| BFRSLanguage Model=Gemini 2.0 Flash, statistical_significance=true2025.11 | 66.2 | |
| MIPROv2Language Model=Gemini 2.0 Flash, statistical_significance=true2025.11 | 66.2 | |
| BFRSLanguage Model=GPT 4o, statistical_significance=true2025.11 | 65.9 | |
| Zero-Shot CoTLanguage Model=GPT 4o, statistical_significance=true2025.11 | 65.7 | |
| MIPROv2Language Model=GPT 4o, statistical_significance=true2025.11 | 65.3 | |
| Zero-Shot PredictLanguage Model=Claude 3.7 Sonnet2025.11 | 65.1 | |
| HELM BaselineLanguage Model=Claude 3.7 Sonnet2025.11 | 64.8 | |
| BFRSLanguage Model=Llama 3.3 70B, statistical_significance=true2025.11 | 64.2 | |
| Zero-Shot CoTLanguage Model=Llama 3.3 70B, statistical_significance=true2025.11 | 64 | |
| MIPROv2Language Model=Llama 3.3 70B, statistical_significance=true2025.11 | 62.1 | |
| Zero-Shot PredictLanguage Model=Gemini 2.0 Flash2025.11 | 61.7 | |
| HELM BaselineLanguage Model=Gemini 2.0 Flash2025.11 | 61.4 | |
| HELM BaselineLanguage Model=GPT 4o2025.11 | 61 | |
| MIPROv2Language Model=Qwen3 4B, statistical_significance=true2025.11 | 60.4 | |
| BFRSLanguage Model=Qwen3 4B, statistical_significance=true2025.11 | 60.3 | |
| Zero-Shot PredictLanguage Model=GPT 4o2025.11 | 59.7 | |
| Zero-Shot PredictLanguage Model=Llama 3.3 70B2025.11 | 59.2 | |
| Zero-Shot CoTLanguage Model=Qwen3 4B, statistical_significance=true2025.11 | 59 | |
| HELM BaselineLanguage Model=Llama 3.3 70B2025.11 | 57.2 | |
| Zero-Shot PredictLanguage Model=Qwen3 4B2025.11 | 49.3 | |
| HELM BaselineLanguage Model=Qwen3 4B2025.11 | 47.8 |