General Capability Evaluation on Tülu General Benchmarks 3
45MMLUSystem Prompt
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| System Promptdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 45 | 52 | 27 | 34 | |
| Llama-3.2-1B-SFTdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 42 | 50 | 25 | 24 | |
| VCLdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 42 | 51 | 26 | 25 | |
| DROdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 39 | 49 | 24 | 23 | |
| Self-CDdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 38 | 50 | 26 | 23 | |
| Vector Ablationdecoding_settings=greedy decoding, temperature=none, max_length=5122026.03 | 37 | 51 | 25 | 24 |