Question Answering on QAMPARI (test)
18.86Correctness Rec@5LLAMA-2-7B + f.g.RL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLAMA-2-7B + f.g.RLBackbone=LLAMA-2-7B, Setting=Separate Setting2024.02 | 18.86 | 17.66 | 16.95 | 18.36 | |
| LLAMA-2-7B + f.g.RLBackbone=LLAMA-2-7B, Setting=Combined Setting2024.02 | 18.39 | 18.81 | 17.82 | 19.07 | |
| +f.g.RLreward=fine-grained, opt=RL2024.02 | 18.39 | 18.81 | 17.82 | 19.07 | |
| ChatGPTSetting=In-Context Learning2024.02 | 18.34 | 18.57 | 17.65 | 20.6 | |
| ICL ChatGPTmode=ICL2024.02 | 18.34 | 18.57 | 17.65 | 20.6 | |
| T5 + f.g.RLBackbone=T5, Setting=Separate Setting2024.02 | 18.1 | 14.56 | 15.75 | 17.97 | |
| LLAMA-2-7B + f.g.RSBackbone=LLAMA-2-7B, Setting=Separate Setting2024.02 | 17.48 | 20.67 | 20.62 | 21.65 | |
| +h.RLreward=heuristic, opt=RL2024.02 | 17.3 | 16.44 | 16.39 | 18.92 | |
| Mdistdescription=Base model2024.02 | 17.26 | 16.18 | 18.69 | 18.94 | |
| LLAMA-2-7B + f.g.(RS+RL)Backbone=LLAMA-2-7B, Setting=Combined Setting2024.02 | 16.65 | 19.54 | 19.5 | 20.03 | |
| +f.g.(RS+RL)reward=fine-grained, opt=RS+RL2024.02 | 16.65 | 19.54 | 19.5 | 20.03 | |
| LLAMA-2-7B + f.g.RSBackbone=LLAMA-2-7B, Setting=Combined Setting2024.02 | 16.14 | 18.95 | 18.56 | 19.5 | |
| +f.g.RSreward=fine-grained, opt=RS2024.02 | 16.14 | 18.95 | 18.56 | 19.5 | |
| LLAMA-2-7B + f.g.(RS+RL)Backbone=LLAMA-2-7B, Setting=Separate Setting2024.02 | 15.76 | 21.6 | 21.36 | 21.92 | |
| +h.(RS+RL)reward=heuristic, opt=RS+RL2024.02 | 15.02 | 15.67 | 16.82 | 18.09 | |
| +h.RSreward=heuristic, opt=RS2024.02 | 14.62 | 15.21 | 16.71 | 17.98 | |
| ICL LLaMA-2-7Bmode=ICL2024.02 | 8.24 | 9.23 | 7.51 | 7.65 |