Knowledge-intensive Question Answering on CRAG
42.3TruthfulnessGPT-5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-52025.09 | 42.3 | 24.7 | — | |
| TruthRLBackbone=Llama3.3-70B-Instruct2025.09 | 40.6 | 17.3 | — | |
| TruthRLRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 37.2 | 19.4 | 56.6 | |
| OpenAI o32025.09 | 33.2 | 33.1 | — | |
| TruthRLRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 33.1 | 17.3 | 50.4 | |
| PromptingBackbone=Llama3.3-70B-Instruct2025.09 | 32.1 | 26.4 | — | |
| RLKFRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 25.2 | 21.9 | 47.1 | |
| RFTRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 22.6 | 31.4 | 54 | |
| TruthRLRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 22.4 | 16.3 | 38.7 | |
| TruthRL-BinaryRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 20.8 | 39.5 | 60.3 | |
| RLKFRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 18.9 | 30.6 | 49.5 | |
| RLHFRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 16.5 | 35.9 | 52.4 | |
| TruthRLRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 16.2 | 8.7 | 24.9 | |
| R-TuningRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 15.2 | 33.1 | 48.4 | |
| R-TuningRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 13.4 | 35 | 48.4 | |
| RLHFRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 13.1 | 39.7 | 52.7 | |
| PromptingRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 10.6 | 38.4 | 49 | |
| TruthRL-BinaryRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | 8.4 | 45.3 | 53.7 | |
| PromptingRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 5.3 | 43.5 | 48.8 | |
| RLHFRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 1.4 | 35.3 | 36.7 | |
| SFTRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | 1.4 | 49.3 | 50.7 | |
| RLKFRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -0.6 | 32.8 | 32.2 | |
| SFTRetrieval Setting=With Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -2.3 | 51.2 | 48.8 | |
| RFTRetrieval Setting=With Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -3.7 | 48.8 | 45.1 | |
| PromptingRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -4.4 | 44.5 | 40.1 | |
| RLKFRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -6.1 | 28.1 | 22.1 | |
| R-TuningRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -7.5 | 21.9 | 14.5 | |
| RFTRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -7.6 | 48.1 | 40.4 | |
| R-TuningRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -13.7 | 39.5 | 25.8 | |
| TruthRL-BinaryRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -14.5 | 57.2 | 42.8 | |
| RFTRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -16.8 | 46.7 | 29.9 | |
| PromptingRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -17.4 | 48.1 | 30.6 | |
| RLHFRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -17.4 | 45.7 | 28.3 | |
| TruthRL-BinaryRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -29.2 | 64.5 | 35.3 | |
| SFTRetrieval Setting=Without Retrieval, Backbone=Llama3.1-8B-Inst2025.09 | -42.1 | 71.1 | 28.9 | |
| SFTRetrieval Setting=Without Retrieval, Backbone=Qwen2.5-7B-Inst2025.09 | -51.5 | 75.7 | 24.3 |