# Question Answering on SQuAD (EM/F1)

Source: [Question Answering on SQuAD (EM/F1)](<https://www.sota2.com/research/sota/question-answering-on-squad-em-f1>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-12T12:13:50.003Z

Evaluation of model performance on the Stanford Question Answering Dataset (SQuAD) using Exact Match (EM) and F1 scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Question Answering |
| Dataset (including any stated split) | SQuAD |
| Metric | F1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 162 of 162 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [F1](<https://www.sota2.com/research/sota/question-answering-on-squad-em-f1.md?metric=F1>): Higher is better; unit: PERCENT.

- [EM](<https://www.sota2.com/research/sota/question-answering-on-squad-em-f1.md?metric=EM>): Higher is better; unit: PERCENT.

## Recorded results

| Method | F1 | Result date | Report paper |
| --- | --- | --- | --- |
| SMP-S | 89.8 | Not recorded | 68fa6f501e209131cea44f58 |
| SMP-L | 89.4 | Not recorded | 68fa6f501e209131cea44f58 |
| Prompt tuning | 88.41 | Not recorded | 6982d2fba8b01e76cc8e9a10 |
| Standard MHA | 88.4 | Not recorded | 69438a63423ea4226572927c |
| CAP | 88.2 | Not recorded | 68fa6f501e209131cea44f58 |
| MAHA | 88.2 | Not recorded | 69438a63423ea4226572927c |
| BERT base | 88.1 | Not recorded | 68fa6f501e209131cea44f58 |
| BigBird | 87.9 | Not recorded | 69438a63423ea4226572927c |
| Movement | 87.6 | Not recorded | 68fa6f501e209131cea44f58 |
| Longformer | 87.6 | Not recorded | 69438a63423ea4226572927c |
| SMP-S | 87.2 | Not recorded | 68fa6f501e209131cea44f58 |
| SMP-L | 86.9 | Not recorded | 68fa6f501e209131cea44f58 |
| Performer | 86.7 | Not recorded | 69438a63423ea4226572927c |
| CAP | 85.6 | Not recorded | 68fa6f501e209131cea44f58 |
| S-STE | 85.5 | Not recorded | 68fa766a1e209131cea6cd41 |
| Reformer | 85.2 | Not recorded | 69438a63423ea4226572927c |
| COIECD | 84.99 | Not recorded | 68fa74411e209131cea608ba |
| Soft-Movement | 84.9 | Not recorded | 68fa6f501e209131cea44f58 |
| Dense | 84.9 | Not recorded | 68fa766a1e209131cea6cd41 |
| SMP-S | 84.6 | Not recorded | 68fa6f501e209131cea44f58 |
| T-SR-STE+DF | 84.6 | Not recorded | 68fa766a1e209131cea6cd41 |
| SMP-L | 84.3 | Not recorded | 68fa6f501e209131cea44f58 |
| Movement | 84.3 | Not recorded | 68fa6f501e209131cea44f58 |
| SMP-S | 84.1 | Not recorded | 68fa6f501e209131cea44f58 |
| Dense | 83.6 | Not recorded | 68fa766a1e209131cea6cd41 |
| Regular | 83.53 | Not recorded | 68fa74411e209131cea608ba |
| SMP-L | 83.4 | Not recorded | 68fa6f501e209131cea44f58 |
| SC | 83.28 | Not recorded | 68fa74411e209131cea608ba |
| CD | 83.1 | Not recorded | 68fa74411e209131cea608ba |
| CAP | 83 | Not recorded | 68fa6f501e209131cea44f58 |
| DeBERTa-V3 base + PiFi | 82.83 | Not recorded | 68fa79451e209131cea7cf92 |
| S-STE | 82.7 | Not recorded | 68fa766a1e209131cea6cd41 |
| T-SR-STE | 82.6 | Not recorded | 68fa766a1e209131cea6cd41 |
| DeBERTa-V3 base | 82.49 | Not recorded | 68fa79451e209131cea7cf92 |
| T-SR-STE+DF | 82.4 | Not recorded | 68fa766a1e209131cea6cd41 |
| SR-STE | 82.4 | Not recorded | 68fa766a1e209131cea6cd41 |
| Soft-Movement | 82.3 | Not recorded | 68fa6f501e209131cea44f58 |
| L0-regularization | 81.9 | Not recorded | 68fa6f501e209131cea44f58 |
| CAD | 81.88 | Not recorded | 68fa74411e209131cea608ba |
| Movement | 81.7 | Not recorded | 68fa6f501e209131cea44f58 |
| DeBERTa base + PiFi | 81.52 | Not recorded | 68fa79451e209131cea7cf92 |
| Soft-Movement | 81.5 | Not recorded | 68fa6f501e209131cea44f58 |
| SMP-S | 81.4 | Not recorded | 68fa6f501e209131cea44f58 |
| Prompt tuning | 81.09 | Not recorded | 6982d2fba8b01e76cc8e9a10 |
| SMP-L | 81 | Not recorded | 68fa6f501e209131cea44f58 |
| DeBERTa base | 80.94 | Not recorded | 68fa79451e209131cea7cf92 |
| RoBERTa base + PiFi | 80.89 | Not recorded | 68fa79451e209131cea7cf92 |
| ELECTRA base + PiFi | 80.45 | Not recorded | 68fa79451e209131cea7cf92 |
| ELSA | 80.44 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| Magnitude | 80.1 | Not recorded | 68fa6f501e209131cea44f58 |
| RoBERTa base | 80.06 | Not recorded | 68fa79451e209131cea7cf92 |
| FedCAda | 80.03 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| Soft-Movement | 79.9 | Not recorded | 68fa6f501e209131cea44f58 |
| w/ Context | 79.81 | Not recorded | 6993f6059ebc499c97409ae4 |
| RoFed | 79.33 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| ELSA | 79.24 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| OURS (PIC) | 79.03 | Not recorded | 6993f6059ebc499c97409ae4 |
| FedProx | 79.01 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| RaSA | 78.99 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| Dense | 78.8 | Not recorded | 68fa766a1e209131cea6cd41 |
| S-STE | 78.8 | Not recorded | 68fa766a1e209131cea6cd41 |
| FedCAda | 78.52 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| T-SR-STE+DF | 78.5 | Not recorded | 68fa766a1e209131cea6cd41 |
| RoFed | 78.5 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| RaSA | 78.38 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| FedProx | 78.33 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| BERT base + PiFi | 78.09 | Not recorded | 68fa79451e209131cea7cf92 |
| FedAMS | 78.02 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| Movement | 78 | Not recorded | 68fa6f501e209131cea44f58 |
| PCC-Large | 77.76 | Not recorded | 6993f6059ebc499c97409ae4 |
| Llama 3.3 70B | 77.7 | Not recorded | 68fa79061e209131cea7bc30 |
| FedAMS | 77.66 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| SR-STE | 77.5 | Not recorded | 68fa766a1e209131cea6cd41 |
| FedAvg | 77.22 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| T-SR-STE | 77.2 | Not recorded | 68fa766a1e209131cea6cd41 |
| FedAvg | 76.92 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| FedAvg (Random) | 76.52 | Not recorded | 69705dc4e25ad2fa9d1ed51f |
| Movement | 76.3 | Not recorded | 68fa6f501e209131cea44f58 |
| BERT base | 76.06 | Not recorded | 68fa79451e209131cea7cf92 |
| PCC-Lite | 75.83 | Not recorded | 6993f6059ebc499c97409ae4 |

### SMP-S

F1: 89.8. Result ID: 6990a2f53a6c314849020735.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"50%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### SMP-L

F1: 89.4. Result ID: 6990a2f53a6c314849020734.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"50%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### Prompt tuning

F1: 88.41. Result ID: 69964e605698665ec492fa39.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2fba8b01e76cc8e9a10.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6aa51e209131cea296c9.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.2-3B-Base","Evaluation Protocol":"Prompt tuning","Context Setting":"w/ context"}. This record does not assert independent reproduction.

### Standard MHA

F1: 88.4. Result ID: 6996bd665698665ec408529d.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49db1.md>)

[Implementation](<https://github.com/tensorflow/tensor2tensor>)

Recorded evaluation configuration: {"Memory (GB)":"15.2"}. This record does not assert independent reproduction.

### CAP

F1: 88.2. Result ID: 6990a2f53a6c314849020733.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b6e1e209131cea2e511.md>)

Recorded evaluation configuration: {"Remaining Weights":"50%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### MAHA

F1: 88.2. Result ID: 6996bd665698665ec40852a2.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

Recorded evaluation configuration: {"Memory (GB)":"6.7"}. This record does not assert independent reproduction.

### BERT base

F1: 88.1. Result ID: 6990a2f53a6c314849020729.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Remaining Weights":"100%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### BigBird

F1: 87.9. Result ID: 6996bd665698665ec408529f.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa690b1e209131cea1faf6.md>)

[Implementation](<http://goo.gle/bigbird-transformer>)

Recorded evaluation configuration: {"Memory (GB)":"10.3"}. This record does not assert independent reproduction.

### Movement

F1: 87.6. Result ID: 6990a2f53a6c314849020732.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"50%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### Longformer

F1: 87.6. Result ID: 6996bd665698665ec408529e.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68d41e209131cea1e701.md>)

[Implementation](<https://github.com/allenai/longformer>)

Recorded evaluation configuration: {"Memory (GB)":"9.1"}. This record does not assert independent reproduction.

### SMP-S

F1: 87.2. Result ID: 6990a2f53a6c31484902073c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### SMP-L

F1: 86.9. Result ID: 6990a2f53a6c31484902073b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### Performer

F1: 86.7. Result ID: 6996bd665698665ec40852a1.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e031e209131cea3d699.md>)

[Implementation](<https://github.com/google-research/google-research/tree/master/performer>)

Recorded evaluation configuration: {"Memory (GB)":"8.5"}. This record does not assert independent reproduction.

### CAP

F1: 85.6. Result ID: 6990a2f53a6c31484902073a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b6e1e209131cea2e511.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### S-STE

F1: 85.5. Result ID: 69943b6d3a6c3148491bee9d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"774M","Pre-train":"S-STE","Fine-tune":"S-STE"}. This record does not assert independent reproduction.

### Reformer

F1: 85.2. Result ID: 6996bd665698665ec40852a0.

[Reporting paper](<https://www.sota2.com/research/paper/69438a63423ea4226572927c.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa671b1e209131cea143ed.md>)

[Implementation](<https://github.com/google/trax/tree/master/trax/models/reformer>)

Recorded evaluation configuration: {"Memory (GB)":"7.8"}. This record does not assert independent reproduction.

### COIECD

F1: 84.99. Result ID: 699343e33a6c31484914cea1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

[Implementation](<https://github.com/Stacy027/COIECD>)

Recorded evaluation configuration: {"Model":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### Soft-Movement

F1: 84.9. Result ID: 6990a2f53a6c314849020739.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### Dense

F1: 84.9. Result ID: 69943b6d3a6c3148491bee9b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"774M","Pre-train":"Dense","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### SMP-S

F1: 84.6. Result ID: 6990a2f53a6c31484902072d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### T-SR-STE+DF

F1: 84.6. Result ID: 69943b6d3a6c3148491bee9c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756e1e209131cea67db7.md>)

Recorded evaluation configuration: {"Params":"774M","Pre-train":"T-SR-STE+DF \[20\]","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### SMP-L

F1: 84.3. Result ID: 6990a2f53a6c31484902072c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### Movement

F1: 84.3. Result ID: 6990a2f53a6c314849020738.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### SMP-S

F1: 84.1. Result ID: 6990a2f53a6c314849020741.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### Dense

F1: 83.6. Result ID: 69943b6d3a6c3148491bee96.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"350M","Pre-train":"Dense","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### Regular

F1: 83.53. Result ID: 699343e33a6c31484914ce9d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

Recorded evaluation configuration: {"Model":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### SMP-L

F1: 83.4. Result ID: 6990a2f53a6c314849020740.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### SC

F1: 83.28. Result ID: 699343e33a6c31484914ce9e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eb61e209131cea41b33.md>)

Recorded evaluation configuration: {"Model":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### CD

F1: 83.1. Result ID: 699343e33a6c31484914ce9f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70091e209131cea48c24.md>)

[Implementation](<https://github.com/XiangLi1999/ContrastiveDecoding>)

Recorded evaluation configuration: {"Model":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### CAP

F1: 83. Result ID: 6990a2f53a6c31484902073f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6b6e1e209131cea2e511.md>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### DeBERTa-V3 base + PiFi

F1: 82.83. Result ID: 6995d82b3badfaffe6de39ed.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Implementation](<https://github.com/khyun8072/PiFi>)

Recorded evaluation configuration: {"Model Backbone":"DeBERTa-V3-base","PiFi Integration":true,"Source LLM":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### S-STE

F1: 82.7. Result ID: 69943b6d3a6c3148491bee9a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"350M","Pre-train":"S-STE","Fine-tune":"S-STE"}. This record does not assert independent reproduction.

### T-SR-STE

F1: 82.6. Result ID: 69943b6d3a6c3148491bee98.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756e1e209131cea67db7.md>)

Recorded evaluation configuration: {"Params":"350M","Pre-train":"T-SR-STE","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### DeBERTa-V3 base

F1: 82.49. Result ID: 6995d82b3badfaffe6de39ec.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ee11e209131cea4297b.md>)

[Implementation](<https://github.com/microsoft/DeBERTa>)

Recorded evaluation configuration: {"Model Backbone":"DeBERTa-V3-base"}. This record does not assert independent reproduction.

### T-SR-STE+DF

F1: 82.4. Result ID: 69943b6d3a6c3148491bee97.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756e1e209131cea67db7.md>)

Recorded evaluation configuration: {"Params":"350M","Pre-train":"T-SR-STE+DF \[20\]","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### SR-STE

F1: 82.4. Result ID: 69943b6d3a6c3148491bee99.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e921e209131cea40bdc.md>)

Recorded evaluation configuration: {"Params":"350M","Pre-train":"SR-STE","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### Soft-Movement

F1: 82.3. Result ID: 6990a2f53a6c31484902073e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### L0-regularization

F1: 81.9. Result ID: 6990a2f53a6c314849020737.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa647d1e209131cea04d19.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### CAD

F1: 81.88. Result ID: 699343e33a6c31484914cea0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa74411e209131cea608ba.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f6e1e209131cea45813.md>)

Recorded evaluation configuration: {"Model":"FLAN-T5-3B"}. This record does not assert independent reproduction.

### Movement

F1: 81.7. Result ID: 6990a2f53a6c31484902072a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### DeBERTa base + PiFi

F1: 81.52. Result ID: 6995d82b3badfaffe6de39eb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Implementation](<https://github.com/khyun8072/PiFi>)

Recorded evaluation configuration: {"Model Backbone":"DeBERTa-base","PiFi Integration":true,"Source LLM":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### Soft-Movement

F1: 81.5. Result ID: 6990a2f53a6c31484902072b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### SMP-S

F1: 81.4. Result ID: 6990a2f53a6c314849020731.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### Prompt tuning

F1: 81.09. Result ID: 69964e605698665ec492fa32.

[Reporting paper](<https://www.sota2.com/research/paper/6982d2fba8b01e76cc8e9a10.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6aa51e209131cea296c9.md>)

Recorded evaluation configuration: {"Backbone":"Llama-3.2-1B-Base","Evaluation Protocol":"Prompt tuning","Context Setting":"w/ context"}. This record does not assert independent reproduction.

### SMP-L

F1: 81. Result ID: 6990a2f53a6c314849020730.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Implementation](<https://github.com/kongds/SMP>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### DeBERTa base

F1: 80.94. Result ID: 6995d82b3badfaffe6de39ea.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6aef1e209131cea2b2a6.md>)

[Implementation](<https://github.com/microsoft/DeBERTa111>)

Recorded evaluation configuration: {"Model Backbone":"DeBERTa-base"}. This record does not assert independent reproduction.

### RoBERTa base + PiFi

F1: 80.89. Result ID: 6995d82b3badfaffe6de39e7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Implementation](<https://github.com/khyun8072/PiFi>)

Recorded evaluation configuration: {"Model Backbone":"RoBERTa-base","PiFi Integration":true,"Source LLM":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### ELECTRA base + PiFi

F1: 80.45. Result ID: 6995d82b3badfaffe6de39e9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Implementation](<https://github.com/khyun8072/PiFi>)

Recorded evaluation configuration: {"Model Backbone":"ELECTRA-base","PiFi Integration":true,"Source LLM":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### ELSA

F1: 80.44. Result ID: 69992a5c5698665ec4925e40.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### Magnitude

F1: 80.1. Result ID: 6990a2f53a6c314849020736.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa622d1e209131ce9f7f15.md>)

Recorded evaluation configuration: {"Remaining Weights":"10%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### RoBERTa base

F1: 80.06. Result ID: 6995d82b3badfaffe6de39e6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66131e209131cea0e18b.md>)

[Implementation](<https://github.com/pytorch/fairseq>)

Recorded evaluation configuration: {"Model Backbone":"RoBERTa-base"}. This record does not assert independent reproduction.

### FedCAda

F1: 80.03. Result ID: 69992a5c5698665ec4925e3d.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73491e209131cea5b8ff.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### Soft-Movement

F1: 79.9. Result ID: 6990a2f53a6c31484902072f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### w/ Context

F1: 79.81. Result ID: 699a80af5698665ec40a566f.

[Reporting paper](<https://www.sota2.com/research/paper/6993f6059ebc499c97409ae4.md>)

Recorded evaluation configuration: {"Compression Rate":"1x"}. This record does not assert independent reproduction.

### RoFed

F1: 79.33. Result ID: 69992a5c5698665ec4925e3e.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### ELSA

F1: 79.24. Result ID: 69992a5c5698665ec4925e38.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### OURS (PIC)

F1: 79.03. Result ID: 699a80af5698665ec40a5677.

[Reporting paper](<https://www.sota2.com/research/paper/6993f6059ebc499c97409ae4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6993f6059ebc499c97409ae4.md>)

Recorded evaluation configuration: {"Compression Rate":"4x","Parameter Size":"0.5B"}. This record does not assert independent reproduction.

### FedProx

F1: 79.01. Result ID: 69992a5c5698665ec4925e3a.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67701e209131cea163fe.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### RaSA

F1: 78.99. Result ID: 69992a5c5698665ec4925e3c.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### Dense

F1: 78.8. Result ID: 69943b6d3a6c3148491bee91.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"124M","Pre-train":"Dense","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### S-STE

F1: 78.8. Result ID: 69943b6d3a6c3148491bee95.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

Recorded evaluation configuration: {"Params":"124M","Pre-train":"S-STE","Fine-tune":"S-STE"}. This record does not assert independent reproduction.

### FedCAda

F1: 78.52. Result ID: 69992a5c5698665ec4925e35.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73491e209131cea5b8ff.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### T-SR-STE+DF

F1: 78.5. Result ID: 69943b6d3a6c3148491bee92.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756e1e209131cea67db7.md>)

Recorded evaluation configuration: {"Params":"124M","Pre-train":"T-SR-STE+DF \[20\]","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### RoFed

F1: 78.5. Result ID: 69992a5c5698665ec4925e36.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### RaSA

F1: 78.38. Result ID: 69992a5c5698665ec4925e34.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### FedProx

F1: 78.33. Result ID: 69992a5c5698665ec4925e32.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa67701e209131cea163fe.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### BERT base + PiFi

F1: 78.09. Result ID: 6995d82b3badfaffe6de39e5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Implementation](<https://github.com/khyun8072/PiFi>)

Recorded evaluation configuration: {"Model Backbone":"BERT-base","PiFi Integration":true,"Source LLM":"Llama-3.1-8B"}. This record does not assert independent reproduction.

### FedAMS

F1: 78.02. Result ID: 69992a5c5698665ec4925e3b.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5a541e209131ce9becf5.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### Movement

F1: 78. Result ID: 6990a2f53a6c31484902073d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":true}. This record does not assert independent reproduction.

### PCC-Large

F1: 77.76. Result ID: 699a80af5698665ec40a5676.

[Reporting paper](<https://www.sota2.com/research/paper/6993f6059ebc499c97409ae4.md>)

Recorded evaluation configuration: {"Compression Rate":"4x","Parameter Size":"8B"}. This record does not assert independent reproduction.

### Llama 3.3 70B

F1: 77.7. Result ID: 6995b99f3badfaffe6bada4d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79061e209131cea7bc30.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75d11e209131cea6a028.md>)

[Implementation](<https://github.com/meta-llama/llama-agentic-system>)

Recorded evaluation configuration: {"Model Size":"70B"}. This record does not assert independent reproduction.

### FedAMS

F1: 77.66. Result ID: 69992a5c5698665ec4925e33.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa5a541e209131ce9becf5.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### SR-STE

F1: 77.5. Result ID: 69943b6d3a6c3148491bee94.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e921e209131cea40bdc.md>)

Recorded evaluation configuration: {"Params":"124M","Pre-train":"SR-STE","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### FedAvg

F1: 77.22. Result ID: 69992a5c5698665ec4925e39.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e761e209131cea40072.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### T-SR-STE

F1: 77.2. Result ID: 69943b6d3a6c3148491bee93.

[Reporting paper](<https://www.sota2.com/research/paper/68fa766a1e209131cea6cd41.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa756e1e209131cea67db7.md>)

Recorded evaluation configuration: {"Params":"124M","Pre-train":"T-SR-STE","Fine-tune":"Dense"}. This record does not assert independent reproduction.

### FedAvg

F1: 76.92. Result ID: 69992a5c5698665ec4925e31.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e761e209131cea40072.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.1"}. This record does not assert independent reproduction.

### FedAvg (Random)

F1: 76.52. Result ID: 69992a5c5698665ec4925e3f.

[Reporting paper](<https://www.sota2.com/research/paper/69705dc4e25ad2fa9d1ed51f.md>)

Recorded evaluation configuration: {"alpha\_hat":"0.2"}. This record does not assert independent reproduction.

### Movement

F1: 76.3. Result ID: 6990a2f53a6c31484902072e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa6f501e209131cea44f58.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68a01e209131cea1d35c.md>)

Recorded evaluation configuration: {"Remaining Weights":"3%","Knowledge Distillation":false}. This record does not assert independent reproduction.

### BERT base

F1: 76.06. Result ID: 6995d82b3badfaffe6de39e4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa79451e209131cea7cf92.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa65c31e209131cea0c4ed.md>)

[Implementation](<https://github.com/google-research/bert>)

Recorded evaluation configuration: {"Model Backbone":"BERT-base"}. This record does not assert independent reproduction.

### PCC-Lite

F1: 75.83. Result ID: 699a80af5698665ec40a5675.

[Reporting paper](<https://www.sota2.com/research/paper/6993f6059ebc499c97409ae4.md>)

Recorded evaluation configuration: {"Compression Rate":"4x","Parameter Size":"0.77B"}. This record does not assert independent reproduction.

Page 1 of 3. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/question-answering-on-squad-em-f1.md?metric=F1&page=2>): Follow this link to discover more pages.
