Comparative Reasoning on COMPS 2-Choice
8.5Δ AccuracyBatch Calibration (BC)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Batch Calibration (BC)Model=Llama3.1-8B2026.02 | 8.5 | — | — | -0.173 | |
| RBCorrModel=Llama3.1-8B2026.02 | 8 | — | — | -0.174 | |
| Contextual Calibration (CC)Model=Llama3.1-8B2026.02 | 6.9 | — | — | -0.098 | |
| Batch Calibration (BC)Model=Llama3.1-8B-Instruct2026.02 | 0 | — | — | -0.003 | |
| RBCorrModel=Llama3.1-8B-Instruct2026.02 | 0 | — | — | -0.025 | |
| Contextual Calibration (CC)Model=Llama3.1-8B-Instruct2026.02 | -0.1 | — | — | -0.016 | |
| BaselineModel=Llama3.1-8B2026.02 | — | 0.79 | 0.185 | — | |
| BaselineModel=Llama3.1-8B-Instruct2026.02 | — | 0.863 | 0.036 | — |