Knowledge Cutoff on Multi-cutoff Historical Event Benchmark (MHEB)
91MHEB Accuracy (Cutoff 0)w/ SR → QR
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| w/ SR → QRModel=Llama3.3-70b-Instruct2026.06 | 91 | 89.2 | 89.8 | 88.9 | 89.1 | 89 | 89.2 | 88.9 | 89.3 | 91 | 89.5 | |
| w/ ZS-CoTModel=Llama3.3-70b-Instruct2026.06 | 85.8 | 82.4 | 82 | 81.6 | 81.1 | 81.1 | 82.4 | 80.9 | 81.8 | 83.1 | 82.2 | |
| w/ QR → SRModel=Llama3.3-70b-Instruct2026.06 | 85.2 | 83.1 | 83.3 | 82.7 | 83.4 | 82.4 | 82.4 | 82.9 | 82.8 | 83.1 | 83.1 | |
| w/ QRModel=Llama3.3-70b-Instruct2026.06 | 82.7 | 80.6 | 78.7 | 76.1 | 76.8 | 76.9 | 78.7 | 76.3 | 77.8 | 77.9 | 78.2 | |
| w/ SRModel=Llama3.3-70b-Instruct2026.06 | 79.4 | 83.3 | 83 | 82.2 | 82.1 | 81.8 | 81.3 | 80.9 | 81.2 | 80.6 | 81.6 | |
| w/ ZS-PSModel=Llama3.3-70b-Instruct2026.06 | 76.8 | 77 | 76.3 | 75.6 | 74.8 | 75.2 | 74.8 | 75.7 | 74.6 | 73.9 | 75.5 | |
| P1Model=Llama3.3-70b-Instruct2026.06 | 67.3 | 74.9 | 73.8 | 73.9 | 74.2 | 72.3 | 71.6 | 71 | 71.3 | 69.7 | 72 |