Generative Question Answering on MsMARCO (dev)
57.2ROUGE ScoreRAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RAGRole=Published SotA2022.04 | 57.2 | — | |
| KIDBackbone=BART-large, Parameters=406M, Evaluation Protocol=fine-tuned, Decoding Strategy=KID2022.04 | 56.9 | 51.9 | |
| KIDBackbone=GPT2-medium, Parameters=345M, Evaluation Protocol=fine-tuned, Decoding Strategy=KID2022.04 | 53.9 | 47.4 | |
| BART-largeBackbone=BART-large, Parameters=406M, Evaluation Protocol=fine-tuned, Decoding Strategy=Sampling2022.04 | 53.5 | 48.4 | |
| GPT2-mediumBackbone=GPT2-medium, Parameters=345M, Evaluation Protocol=fine-tuned, Decoding Strategy=Sampling2022.04 | 51.2 | 46.2 | |
| GPT2-mediumBackbone=GPT2-medium, Parameters=345M, Evaluation Protocol=fine-tuned, Decoding Strategy=Beam2022.04 | 50.6 | 44.1 | |
| BART-largeBackbone=BART-large, Parameters=406M, Evaluation Protocol=fine-tuned, Decoding Strategy=Beam2022.04 | 50.5 | 44.5 | |
| FastQAExtk=52017.03 | 34.4 | 35 | |
| FastQAk=52017.03 | 33 | 34.9 | |
| GPT2-mediumBackbone=GPT2-medium, Parameters=345M, Evaluation Protocol=zero-shot2022.04 | 30.1 | 28.8 | |
| BART-largeBackbone=BART-large, Parameters=406M, Evaluation Protocol=zero-shot2022.04 | 19.7 | 19.1 |