# Question Answering on SQuAD 2.0 (Performance, Token Cost, Time Cost) Source: [Question Answering on SQuAD 2.0 (Performance, Token Cost, Time Cost)]() Collection index: [llms.txt]() Data updated: 2026-05-19T09:44:29.649Z Evaluation of Question Answering performance on the SQuAD 2.0 dataset, reporting standard performance alongside operational metrics like token cost and inference time. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | SQuAD 2.0 | | Metric | Performance (F1/EM) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Performance (F1/EM)](): Higher is better; unit: PERCENT. - [Token Cost](): Lower is better; unit: SCALAR. - [Time Cost (s)](): Lower is better; unit: TIME. - [Pass Rate (%)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Performance (F1/EM) | Result date | Report paper | | --- | --- | --- | --- | | LLM w/o cascade | 86.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Initial | 84 | Not recorded | 6a0bee98dbc836b87d82c429 | | Single-target attack | 70 | Not recorded | 6a0bee98dbc836b87d82c429 | | Random noise | 68.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 66 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 57.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Single-target attack | 53.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 49.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 41 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 29.5 | Not recorded | 6a0bee98dbc836b87d82c429 | | Joint-target attack | 16 | Not recorded | 6a0bee98dbc836b87d82c429 | ### LLM w/o cascade Performance (F1/EM): 86.5. Result ID: 6a0c30effbc373592c02aff4. [Reporting paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B"}. This record does not assert independent reproduction. ### Initial Performance (F1/EM): 84. Result ID: 6a0c30effbc373592c02aff3. [Reporting paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B"}. This record does not assert independent reproduction. ### Single-target attack Performance (F1/EM): 70. Result ID: 6a0c30effbc373592c02aff6. [Reporting paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"SLM only"}. This record does not assert independent reproduction. ### Random noise Performance (F1/EM): 68.5. Result ID: 6a0c30effbc373592c02aff5. [Reporting paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 66. Result ID: 6a0c30effbc373592c02affd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=1"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 57.5. Result ID: 6a0c30effbc373592c02affc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=0.8"}. This record does not assert independent reproduction. ### Single-target attack Performance (F1/EM): 53.5. Result ID: 6a0c30effbc373592c02aff7. [Reporting paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"DM only"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 49.5. Result ID: 6a0c30effbc373592c02affb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=0.6"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 41. Result ID: 6a0c30effbc373592c02affa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=0.4"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 29.5. Result ID: 6a0c30effbc373592c02aff9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=0.2"}. This record does not assert independent reproduction. ### Joint-target attack Performance (F1/EM): 16. Result ID: 6a0c30effbc373592c02aff8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Cascade System":"Qwen2.5-3B + DistilBERT-base + Phi-3.5-MoE 16x3.8B","Setting":"P=0"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.