Question Answering on BBQ (Ambiguity/Disambiguation Metrics)
83.93Disambiguation TOP-1FARO-dp
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FARO-dpBackbone=Gemma-2-2b-it, Constraint=Demographic Parity (DP)2026.02 | 83.93 | 63.2 | 6.81 | -1.01 | 0.55 | — | — | |
| Gemma-2-2b-itBackbone=Gemma-2-2b-it, Optimization=Base2026.02 | 83.2 | 63.91 | 14.73 | -0.811 | — | — | — | |
| FARO-cfBackbone=Gemma-2-2b-it, Constraint=Counterfactual Fairness (CF)2026.02 | 83.1 | 62.86 | 10.55 | -0.965 | — | — | 0.41 | |
| FARO-eoBackbone=Gemma-2-2b-it, Constraint=Equalized Odds (EO)2026.02 | 82.71 | 63.72 | 12.96 | -0.822 | — | 0.44 | — | |
| FARO-dpBackbone=Qwen-2.5-1.5B, Constraint=Demographic Parity (DP)2026.02 | 75.11 | 59.18 | 9.11 | -0.104 | 0.26 | — | — | |
| Qwen-2.5-1.5BBackbone=Qwen-2.5-1.5B, Optimization=Base2026.02 | 74.14 | 58.97 | 11.44 | -0.092 | — | — | — | |
| FARO-eoBackbone=Qwen-2.5-1.5B, Constraint=Equalized Odds (EO)2026.02 | 74.06 | 57.66 | 10.87 | -0.1 | — | 0.05 | — | |
| FARO-cfBackbone=Qwen-2.5-1.5B, Constraint=Counterfactual Fairness (CF)2026.02 | 73.12 | 58.91 | 8.04 | -0.155 | — | — | 0.09 | |
| FARO-dpBackbone=Phi-3-Mini, Constraint=Demographic Parity (DP)2026.02 | 71.99 | 46.55 | 9.15 | 1.01 | 0.21 | — | — | |
| Phi-3-MiniBackbone=Phi-3-Mini, Optimization=Base2026.02 | 71.92 | 42.14 | 11.91 | 1.42 | — | — | — | |
| FARO-cfBackbone=Phi-3-Mini, Constraint=Counterfactual Fairness (CF)2026.02 | 71.73 | 45.92 | 9.01 | 0.93 | — | — | 0.37 | |
| FARO-eoBackbone=Phi-3-Mini, Constraint=Equalized Odds (EO)2026.02 | 70.05 | 44.01 | 10.86 | 1.04 | — | 0.18 | — |