Question Answering on SQUAD (Acc, Rouge-L, PPL)
93.8Accuracy+Scrambler
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| +ScramblerModel=Qwen 2.5 32B2026.05 | 93.8 | 91.02 | 2.35 | |
| +DistributedModel=Qwen 2.5 32B2026.05 | 93.75 | 91 | 2.35 | |
| BaselineModel=Qwen 2.5 32B2026.05 | 93.72 | 90.94 | 2.35 | |
| +DistributedModel=Qwen 2 7B2026.05 | 89.23 | 88.23 | 6.43 | |
| BaselineModel=Qwen 2 7B2026.05 | 89.21 | 88.22 | 6.43 | |
| +ScramblerModel=Qwen 2 7B2026.05 | 89.18 | 88.24 | 6.43 | |
| BaselineModel=Llama 3.1 8B2026.05 | 88.33 | 89.75 | 1.85 | |
| +ScramblerModel=Llama 3.1 8B2026.05 | 88.3 | 89.72 | 1.85 | |
| +DistributedModel=Llama 3.1 8B2026.05 | 88.29 | 89.74 | 1.85 | |
| BaselineModel=Mistral 7B2026.05 | 86.44 | 89.76 | 1.6 | |
| +ScramblerModel=Mistral 7B2026.05 | 86.41 | 89.74 | 1.6 | |
| +DistributedModel=Mistral 7B2026.05 | 86.4 | 89.75 | 1.6 |