Question Answering on LLAMA 1
88AccuracyFrontier
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FrontierModel=Claude Opus 4.72025.11 | 88 | — | |
| ReasonerModel=Gemini 3.1 Pro2025.11 | 87 | — | |
| FrontierModel=Gemini 3.1 Pro2025.11 | 86.3 | — | |
| ReasonerModel=GPT-5.52025.11 | 86.3 | — | |
| ConvFillBase Model=SmolLM2 1.7B, Talker=GPT-5.52025.11 | 86 | 98.5 | |
| ReasonerModel=Claude Opus 4.72025.11 | 85.7 | — | |
| ConvFillBase Model=SmolLM2 135M, Talker=Gemini 3.1 Pro2025.11 | 85.7 | 97.7 | |
| ConvFillBase Model=Qwen3 0.6B, Talker=Gemini 3.1 Pro2025.11 | 85.7 | 97.3 | |
| ConvFillBase Model=Gemma 3 1B, Talker=Claude Opus 4.72025.11 | 85.7 | 97.3 | |
| ConvFillBase Model=SmolLM2 360M, Talker=Gemini 3.1 Pro2025.11 | 85.3 | 97.7 | |
| FrontierModel=GPT-5.52025.11 | 85 | — | |
| ConvFillBase Model=SmolLM2 360M, Talker=GPT-5.52025.11 | 85 | 97.7 | |
| ConvFillBase Model=SmolLM2 1.7B, Talker=Claude Opus 4.72025.11 | 85 | 97.7 | |
| ConvFillBase Model=Qwen3 0.6B, Talker=Claude Opus 4.72025.11 | 84.7 | 96.9 | |
| ConvFillBase Model=Gemma 3 1B, Talker=GPT-5.52025.11 | 84.7 | 96.9 | |
| ConvFillBase Model=Gemma 3 1B, Talker=Gemini 3.1 Pro2025.11 | 84.7 | 96.2 | |
| ConvFillBase Model=Llama 3.2 1B, Talker=Claude Opus 4.72025.11 | 84.7 | 96.9 | |
| ConvFillBase Model=SmolLM2 1.7B, Talker=Gemini 3.1 Pro2025.11 | 84.7 | 96.2 | |
| ConvFillBase Model=SmolLM2 360M, Talker=Claude Opus 4.72025.11 | 84.3 | 97.7 | |
| ConvFillBase Model=Qwen3 0.6B, Talker=GPT-5.52025.11 | 84 | 96.1 | |
| ConvFillBase Model=Llama 3.2 1B, Talker=Gemini 3.1 Pro2025.11 | 84 | 95.8 | |
| ConvFillBase Model=SmolLM2 135M, Talker=Claude Opus 4.72025.11 | 83.7 | 95.7 | |
| ConvFillBase Model=Llama 3.2 1B, Talker=GPT-5.52025.11 | 83.3 | 95.4 | |
| ConvFillBase Model=SmolLM2 135M, Talker=GPT-5.52025.11 | 82.7 | 95.4 | |
| ConvFillBase Model=Gemma 3 270M, Talker=Gemini 3.1 Pro2025.11 | 81.7 | 92.7 | |
| ConvFillBase Model=Gemma 3 270M, Talker=GPT-5.52025.11 | 81.3 | 93.1 | |
| ConvFillBase Model=Gemma 3 270M, Talker=Claude Opus 4.72025.11 | 80.7 | 91.4 | |
| Base SLMModel=Llama 3.2 1B2025.11 | 72 | — | |
| Base SLMModel=SmolLM2 1.7B2025.11 | 72 | — | |
| Base SLMModel=Gemma 3 1B2025.11 | 63 | — | |
| Base SLMModel=SmolLM2 360M2025.11 | 51.3 | — | |
| Base SLMModel=Qwen3 0.6B2025.11 | 41.3 | — | |
| Base SLMModel=Gemma 3 270M2025.11 | 40.7 | — | |
| Base SLMModel=SmolLM2 135M2025.11 | 34.7 | — |