Attributable Text Generation on ExpertQA v1 (test)
0.6612AutoAISf.g. (RS+RL)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| f.g. (RS+RL)Reward Granularity=Fine-grained, Learning Method=RS + RL, Base Model=Mdist2024.02 | 0.6612 | 0.8378 | 6,256 | |
| f.g. RSReward Granularity=Fine-grained, Learning Method=Rejection Sampling (RS), Base Model=Mdist2024.02 | 0.6349 | 0.8385 | 7,450 | |
| h.(RS+RL)Reward Granularity=Holistic, Learning Method=RS + RL, Base Model=Mdist2024.02 | 0.5942 | 0.8371 | 8,012 | |
| h.RSReward Granularity=Holistic, Learning Method=Rejection Sampling (RS), Base Model=Mdist2024.02 | 0.5788 | 0.8347 | 8,436 | |
| ICL ChatGPTMode=In-context learning, Base Model=ChatGPT2024.02 | 0.5698 | 0.8583 | 8,145 | |
| f.g. RLReward Granularity=Fine-grained, Learning Method=Reinforcement Learning (RL), Base Model=Mdist2024.02 | 0.5615 | 0.8389 | 7,322 | |
| h.RLReward Granularity=Holistic, Learning Method=Reinforcement Learning (RL), Base Model=Mdist2024.02 | 0.5364 | 0.8382 | 8,139 | |
| MdistMode=SFT / Distilled2024.02 | 0.5133 | 0.8346 | 7,293 | |
| ICL LLaMA-2-7BMode=In-context learning, Base Model=LLaMA-2-7B2024.02 | 0.1963 | 0.8277 | 10,053 |