Question Answering on BBQ
92.12AccuracyFairSteer
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FairSteerBackbone=Llama3-8B, Setting=Few-shot2025.04 | 92.12 | 4.39 | 2.32 | — | — | — | — | — | — | |
| FairSteerBackbone=Llama3-8B, Setting=Zero-shot2025.04 | 90.22 | 1.46 | 2.17 | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-13B, Setting=Few-shot2025.04 | 86.56 | 1.28 | 4.33 | — | — | — | — | — | — | |
| Llama3-8BSetting=Few-shot2025.04 | 84.74 | 13.53 | 2.42 | — | — | — | — | — | — | |
| CALBackbone=Llama3-8B, Setting=Few-shot2025.04 | 82.65 | 2.61 | 2.69 | — | — | — | — | — | — | |
| CALBackbone=Mistral-7B, Setting=Few-shot2025.04 | 81.89 | 8.43 | 4.68 | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-13B, Setting=Few-shot2025.04 | 80.26 | 1.58 | 3.68 | — | — | — | — | — | — | |
| FairSteerBackbone=Mistral-7B, Setting=Few-shot2025.04 | 79.93 | 8.04 | 4.87 | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-13B, Setting=Zero-shot2025.04 | 77.74 | 0.1 | 2.5 | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-13B, Setting=Zero-shot2025.04 | 74.02 | -0.82 | 0.84 | — | — | — | — | — | — | |
| FairSteerBackbone=Mistral-7B, Setting=Zero-shot2025.04 | 73.43 | 3.75 | 5.02 | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-7B, Setting=Few-shot2025.04 | 71.28 | 2.8 | 7.8 | — | — | — | — | — | — | |
| Llama3-8BSetting=Zero-shot2025.04 | 71 | 13.62 | 2.51 | — | — | — | — | — | — | |
| Mistral-7BSetting=Few-shot2025.04 | 70.53 | 16.68 | 5.1 | — | — | — | — | — | — | |
| CALBackbone=Mistral-7B, Setting=Zero-shot2025.04 | 68.32 | 2.28 | 4 | — | — | — | — | — | — | |
| FairSteerBackbone=Vicuna-7B, Setting=Zero-shot2025.04 | 65.38 | 1.47 | 5.16 | — | — | — | — | — | — | |
| Vicuna-13BSetting=Few-shot2025.04 | 64.74 | 15.72 | 5.49 | — | — | — | — | — | — | |
| CALBackbone=Vicuna-13B, Setting=Few-shot2025.04 | 63.72 | 12.11 | 5.93 | — | — | — | — | — | — | |
| Vicuna-13BSetting=Zero-shot2025.04 | 63.71 | 4.97 | 3.56 | — | — | — | — | — | — | |
| Mistral-7BSetting=Zero-shot2025.04 | 62.22 | 10.3 | 5.24 | — | — | — | — | — | — | |
| CALBackbone=Llama3-8B, Setting=Zero-shot2025.04 | 55.51 | 0.08 | 5.64 | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-7B, Setting=Few-shot2025.04 | 53.34 | 2.63 | 4.55 | — | — | — | — | — | — | |
| CALBackbone=Llama2-13B, Setting=Few-shot2025.04 | 53.27 | 9.82 | 5.47 | — | — | — | — | — | — | |
| CALBackbone=Llama2-13B, Setting=Zero-shot2025.04 | 51.29 | 1.41 | 2.46 | — | — | — | — | — | — | |
| Llama2-13BSetting=Zero-shot2025.04 | 48.6 | 5.86 | 2.91 | — | — | — | — | — | — | |
| CALBackbone=Vicuna-13B, Setting=Zero-shot2025.04 | 47.99 | 0.72 | 1.23 | — | — | — | — | — | — | |
| Llama2-13BSetting=Few-shot2025.04 | 47.94 | 16.31 | 5.55 | — | — | — | — | — | — | |
| FairSteerBackbone=Llama2-7B, Setting=Zero-shot2025.04 | 46.28 | 1.04 | 2.11 | — | — | — | — | — | — | |
| Vicuna-7BSetting=Few-shot2025.04 | 43.89 | 14.28 | 9.07 | — | — | — | — | — | — | |
| Llama2-7BSetting=Few-shot2025.04 | 42.67 | 6.43 | 5.36 | — | — | — | — | — | — | |
| CALBackbone=Llama2-7B, Setting=Zero-shot2025.04 | 41.4 | 0.87 | 1.62 | — | — | — | — | — | — | |
| Vicuna-7BSetting=Zero-shot2025.04 | 41.33 | 6.78 | 5.94 | — | — | — | — | — | — | |
| CALBackbone=Vicuna-7B, Setting=Few-shot2025.04 | 40.34 | 15.33 | 9.73 | — | — | — | — | — | — | |
| CALBackbone=Llama2-7B, Setting=Few-shot2025.04 | 38.75 | 6.23 | 6.99 | — | — | — | — | — | — | |
| CALBackbone=Vicuna-7B, Setting=Zero-shot2025.04 | 33.45 | -0.01 | -0.02 | — | — | — | — | — | — | |
| Llama2-7BSetting=Zero-shot2025.04 | 32.21 | 2.44 | 2.41 | — | — | — | — | — | — | |
| Direct-Prompting StandardBase Model=gpt-3.5-turbo, Prompting Strategy=Direct-Prompting (Standard)2026.01 | — | — | — | 86.82 | 64.24 | 75.53 | 0.0636 | 0.0517 | 0.0577 | |
| MADBase Model=gpt-3.5-turbo2026.01 | — | — | — | 92.92 | 41.16 | 67.04 | 0.0068 | 0.0528 | 0.0298 | |
| MPTBase Model=gpt-3.5-turbo2026.01 | — | — | — | 85.06 | 64.92 | 74.99 | 0.0115 | 0.0166 | 0.0141 | |
| Re-Prompting DebiasBase Model=gpt-3.5-turbo, Prompting Strategy=Re-Prompting (Debias)2026.01 | — | — | — | 96.36 | 55.25 | 75.8 | 0.0273 | 0.0132 | 0.0203 |