Multiple-choice Question Answering on MMLU Stem (val)
58.13AccuracySP-Struct-RWD1-full
Evaluation results
| Method | Links | |
|---|---|---|
| SP-Struct-RWD1-fullInference Protocol=Agentic-Independent, Prompting=zero-shot, Number of Parameters=3B2025.12 | 58.13 | |
| SP-Struct-RWD1-fullInference Protocol=Agentic-Internal, Prompting=zero-shot, Number of Parameters=3B2025.12 | 56.8 | |
| DeepSeekMath Base-7BPrompting=few-shot, Number of Parameters=7B2025.12 | 56.5 | |
| SP-Struct-RWD1-fullInference Protocol=Multiple-Try, Prompting=zero-shot, Number of Parameters=3B2025.12 | 53.6 | |
| Mistral 7BPrompting=5-shot, Number of Parameters=7B2025.12 | 51.1 | |
| SP-Struct-RWD1-fullInference Protocol=Single-Try, Prompting=zero-shot, Number of Parameters=3B2025.12 | 50.93 |