# Question Answering on Fairness Evaluation Suite BBQ, CrP, GMO, SSt, WnQ Source: [Question Answering on Fairness Evaluation Suite BBQ, CrP, GMO, SSt, WnQ]() Collection index: [llms.txt]() Data updated: 2026-07-01T06:38:44.000Z Evaluation of Question Answering performance across a suite of fairness datasets, including BBQ, CrP, GMO, SSt, and WnQ components. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | Fairness Evaluation Suite BBQ, CrP, GMO, SSt, WnQ | | Metric | BBQ Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [BBQ Accuracy](): Higher is better; unit: PERCENT. - [CrP Accuracy](): Higher is better; unit: PERCENT. - [GMO Accuracy](): Higher is better; unit: PERCENT. - [SSt Accuracy](): Higher is better; unit: PERCENT. - [WnQ Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | BBQ Accuracy | Result date | Report paper | | --- | --- | --- | --- | | SD-E | 98.26 | Not recorded | 6a449f01e0224bd4f4992f95 | | ADBP | 97.99 | Not recorded | 6a449f01e0224bd4f4992f95 | | Fair-GCG | 97.9 | Not recorded | 6a449f01e0224bd4f4992f95 | | Luo et al. | 97.8 | Not recorded | 6a449f01e0224bd4f4992f95 | | Vanilla | 96.25 | Not recorded | 6a449f01e0224bd4f4992f95 | | IASC | 95.61 | Not recorded | 6a449f01e0224bd4f4992f95 | | SD-R | 94.51 | Not recorded | 6a449f01e0224bd4f4992f95 | | Vanilla | 93.93 | Not recorded | 6a449f01e0224bd4f4992f95 | | Fair-GCG | 93.87 | Not recorded | 6a449f01e0224bd4f4992f95 | | Vanilla | 93.41 | Not recorded | 6a449f01e0224bd4f4992f95 | | Fair-GCG | 93.1 | Not recorded | 6a449f01e0224bd4f4992f95 | | Luo et al. | 91.95 | Not recorded | 6a449f01e0224bd4f4992f95 | | SD-E | 90.85 | Not recorded | 6a449f01e0224bd4f4992f95 | | ADBP | 84.81 | Not recorded | 6a449f01e0224bd4f4992f95 | | SD-R | 82.53 | Not recorded | 6a449f01e0224bd4f4992f95 | | Fair-GCG | 81.34 | Not recorded | 6a449f01e0224bd4f4992f95 | | Vanilla | 74.38 | Not recorded | 6a449f01e0224bd4f4992f95 | | IASC | 72.92 | Not recorded | 6a449f01e0224bd4f4992f95 | ### SD-E BBQ Accuracy: 98.26. Result ID: 6a44b5f373d097d845bd9172. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### ADBP BBQ Accuracy: 97.99. Result ID: 6a44b5f373d097d845bd9170. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Fair-GCG BBQ Accuracy: 97.9. Result ID: 6a44b5f373d097d845bd9174. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Luo et al. BBQ Accuracy: 97.8. Result ID: 6a44b5f373d097d845bd916f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Vanilla BBQ Accuracy: 96.25. Result ID: 6a44b5f373d097d845bd916e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### IASC BBQ Accuracy: 95.61. Result ID: 6a44b5f373d097d845bd9171. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### SD-R BBQ Accuracy: 94.51. Result ID: 6a44b5f373d097d845bd9173. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B"}. This record does not assert independent reproduction. ### Vanilla BBQ Accuracy: 93.93. Result ID: 6a44b5f373d097d845bd9193. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Llama-70B"}. This record does not assert independent reproduction. ### Fair-GCG BBQ Accuracy: 93.87. Result ID: 6a44b5f373d097d845bd9192. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-OSS-20B"}. This record does not assert independent reproduction. ### Vanilla BBQ Accuracy: 93.41. Result ID: 6a44b5f373d097d845bd9191. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-OSS-20B"}. This record does not assert independent reproduction. ### Fair-GCG BBQ Accuracy: 93.1. Result ID: 6a44b5f373d097d845bd9194. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Llama-70B"}. This record does not assert independent reproduction. ### Luo et al. BBQ Accuracy: 91.95. Result ID: 6a44b5f373d097d845bd9168. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### SD-E BBQ Accuracy: 90.85. Result ID: 6a44b5f373d097d845bd916b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### ADBP BBQ Accuracy: 84.81. Result ID: 6a44b5f373d097d845bd9169. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### SD-R BBQ Accuracy: 82.53. Result ID: 6a44b5f373d097d845bd916c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### Fair-GCG BBQ Accuracy: 81.34. Result ID: 6a44b5f373d097d845bd916d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### Vanilla BBQ Accuracy: 74.38. Result ID: 6a44b5f373d097d845bd9167. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. ### IASC BBQ Accuracy: 72.92. Result ID: 6a44b5f373d097d845bd916a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama 3.1 8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.