Grounded Generation on QAMPARI (test)
30.28Correctness PrecisionCaLM
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CaLMMethod Type=Postprocess, # Main LM Call=<4, # Verifier LM Call=<3, Main LM=text-unicorn, Retriever=GTR2024.06 | 30.28 | 22.79 | 28.53 | 28.34 | 27.49 | |
| Snippet + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=GTR2024.06 | 27.26 | 25.07 | 25.96 | 25.55 | 25.96 | |
| Summ + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=GTR2024.06 | 27.11 | 26.14 | 24.61 | 24.98 | 25.71 | |
| CaLMMethod Type=Postprocess, # Main LM Call=<4, # Verifier LM Call=<3, Main LM=GPT-3.5-Turbo-1106, Retriever=GTR2024.06 | 26.71 | 18.55 | 25.16 | 24.39 | 23.7 | |
| ICLCite + USCMethod Type=Postprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=GTR2024.06 | 26.34 | 22.1 | 25.48 | 25.19 | 24.78 | |
| ICLCiteMethod Type=Single Run, # Main LM Call=1, Main LM=text-unicorn, Retriever=GTR2024.06 | 26.27 | 21.42 | 25.52 | 25.21 | 24.61 | |
| Snippet + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=GTR2024.06 | 20.48 | 17.18 | 25.45 | 23.91 | 21.76 | |
| CaLMMethod Type=Postprocess, # Main LM Call=<4, # Verifier LM Call=<3, Main LM=text-unicorn, Retriever=DPR2024.06 | 19.62 | 16.71 | 15.9 | 15.95 | 17.05 | |
| ICLCiteMethod Type=Single Run, # Main LM Call=1, Main LM=GPT-3.5-Turbo-1106, Retriever=GTR2024.06 | 19.23 | 17.32 | 21.75 | 20.77 | 19.77 | |
| Snippet + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=DPR2024.06 | 18.81 | 15.11 | 16.1 | 15.83 | 16.46 | |
| Summ + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=DPR2024.06 | 18.61 | 15.17 | 15.61 | 15.36 | 16.19 | |
| CaLMMethod Type=Postprocess, # Main LM Call=<4, # Verifier LM Call=<3, Main LM=GPT-3.5-Turbo-1106, Retriever=DPR2024.06 | 17.65 | 13.61 | 13.93 | 12.99 | 14.55 | |
| ICLCiteMethod Type=Single Run, # Main LM Call=1, Main LM=text-unicorn, Retriever=DPR2024.06 | 17.06 | 12.55 | 15.77 | 15.59 | 15.24 | |
| ICLCite + USCMethod Type=Postprocess, # Main LM Call=10, Main LM=text-unicorn, Retriever=DPR2024.06 | 17.04 | 12.47 | 15.81 | 15.62 | 15.24 | |
| Summ + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=GTR2024.06 | 15.06 | 13.1 | 21.89 | 21 | 17.76 | |
| ICLCite + USCMethod Type=Postprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=GTR2024.06 | 14.07 | 12.45 | 17.34 | 16.6 | 15.12 | |
| ICLCiteMethod Type=Single Run, # Main LM Call=1, Main LM=GPT-3.5-Turbo-1106, Retriever=DPR2024.06 | 12.47 | 10.28 | 12.6 | 11.62 | 11.74 | |
| Snippet + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=DPR2024.06 | 11.67 | 7.87 | 14.14 | 12.64 | 11.58 | |
| ICLCite + USCMethod Type=Postprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=DPR2024.06 | 9.88 | 8.36 | 10.14 | 9.28 | 9.42 | |
| Summ + ICLCiteMethod Type=Preprocess, # Main LM Call=10, Main LM=GPT-3.5-Turbo-1106, Retriever=DPR2024.06 | 6.65 | 4.97 | 10.56 | 9.43 | 7.9 |