Reading Comprehension on QuAC
74.4F1 ScoreFine-tuned SOTA
Evaluation Results
| Method | Links | |
|---|---|---|
| Fine-tuned SOTASetting=Fine-tuned2020.05 | 74.4 | |
| Llama 2Size=70B, shot=1-shot2023.07 | 49.3 | |
| Llama 2Size=13B, shot=1-shot2023.07 | 44.8 | |
| Llama 2Size=34B, shot=1-shot2023.07 | 44.4 | |
| GPT-3Setting=Few-Shot2020.05 | 44.3 | |
| Llama 1Size=33B, shot=0-shot2023.07 | 44.1 | |
| GPT-3Setting=One-Shot2020.05 | 43.3 | |
| FalconSize=40B, shot=1-shot2023.07 | 43.3 | |
| Llama 2Size=34B, shot=0-shot2023.07 | 42.9 | |
| Llama 2Size=13B, shot=0-shot2023.07 | 42.7 | |
| Llama 2Size=70B, shot=0-shot2023.07 | 42.4 | |
| GPT-3Setting=Zero-Shot2020.05 | 41.5 | |
| FalconSize=40B, shot=0-shot2023.07 | 41.2 | |
| MPTSize=30B, shot=1-shot2023.07 | 41.1 | |
| Llama 1Size=65B, shot=0-shot2023.07 | 41 | |
| MPTSize=30B, shot=0-shot2023.07 | 40.4 | |
| Llama 1Size=33B, shot=1-shot2023.07 | 40.3 | |
| Llama 1Size=13B, shot=0-shot2023.07 | 39.9 | |
| Llama 1Size=65B, shot=1-shot2023.07 | 39.8 | |
| Llama 2Size=7B, shot=1-shot2023.07 | 39.7 | |
| Llama 2Size=7B, shot=0-shot2023.07 | 39.4 | |
| Llama 1Size=7B, shot=0-shot2023.07 | 38.9 | |
| MPTSize=7B, shot=0-shot2023.07 | 38 | |
| MPTSize=7B, shot=1-shot2023.07 | 37.7 | |
| Llama 1Size=13B, shot=1-shot2023.07 | 36.5 | |
| Llama 1Size=7B, shot=1-shot2023.07 | 32 | |
| FalconSize=7B, shot=0-shot2023.07 | 24 | |
| FalconSize=7B, shot=1-shot2023.07 | 18.8 |