Faithfulness Hallucination on FollowRAG Faithfulness+
49.5Faithfulness (NaturalQA)NOVA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 49.5 | 56.5 | 18.5 | 55 | 44.9 | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 48.5 | 68 | 25 | 52 | 48.4 | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 46.5 | 60 | 19 | 53.5 | 44.8 | |
| NOVABackbone=Qwen-2, Selection Ratio=5%2025.02 | 46 | 59.6 | 23.5 | 55.5 | 46.1 | |
| NOVAInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 46 | 63 | 20 | 59 | 47 | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 46 | 54.5 | 15 | 52 | 41.9 | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 45.5 | 61.5 | 22 | 48 | 44.3 | |
| NOVABackbone=Qwen-2, Selection Ratio=10%2025.02 | 45 | 62 | 21.5 | 53.5 | 45.5 | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 45 | 62.5 | 21 | 49 | 44.4 | |
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 45 | 62 | 20.5 | 53.5 | 45.3 | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 45 | 52 | 16 | 53 | 41.5 | |
| NOVABackbone=LLaMA-1, Selection Ratio=5%2025.02 | 44.5 | 58.5 | 24 | 55.5 | 45.6 | |
| FLAME-DPOBackbone=Qwen-2, Selection Ratio=100%2025.02 | 44.5 | 58 | 20.5 | 53 | 44 | |
| CaRBackbone=Qwen-2, Selection Ratio=5%2025.02 | 44.5 | 55.5 | 21 | 52 | 43.3 | |
| NOVABackbone=Qwen-2, Selection Ratio=15%2025.02 | 44.5 | 64.5 | 22 | 54 | 46.3 | |
| NOVAInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 44.5 | 59 | 18 | 51 | 43.1 | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 44 | 59.5 | 18 | 48.5 | 42.5 | |
| NOVABackbone=LLaMA-1, Selection Ratio=10%2025.02 | 43.5 | 59.5 | 22.5 | 53 | 44.6 | |
| SELF-EVALBackbone=Qwen-2, Selection Ratio=100%2025.02 | 43.5 | 59 | 21 | 53 | 44.1 | |
| FLAME-DPO factInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 43.5 | 57 | 17.5 | 52 | 42.5 | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=15%2025.02 | 43.5 | 63 | 23 | 50 | 44.9 | |
| SELF-EVALInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 43.5 | 59 | 15.5 | 51.5 | 42.4 | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 43.5 | 57.5 | 17 | 51.5 | 42.4 | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=15%2025.02 | 43.5 | 55 | 18 | 53.5 | 42.5 | |
| NuggetsBackbone=Qwen-2, Selection Ratio=5%2025.02 | 43 | 57.5 | 21.5 | 52.5 | 43.6 | |
| CaRBackbone=Qwen-2, Selection Ratio=15%2025.02 | 43 | 62.5 | 19.5 | 53 | 44.5 | |
| SELF-EVALInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 43 | 58 | 16.5 | 52.5 | 42.5 | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 43 | 58.5 | 17 | 52.5 | 42.8 | |
| CaRInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 43 | 55 | 15.5 | 48 | 40.4 | |
| SELF-EVALBackbone=LLaMA-1, Selection Ratio=100%2025.02 | 42.5 | 56.5 | 22.5 | 53.5 | 43.8 | |
| NOVABackbone=LLaMA-1, Selection Ratio=15%2025.02 | 42.5 | 56.5 | 23.5 | 54.5 | 44.3 | |
| IFDBackbone=Qwen-2, Selection Ratio=5%2025.02 | 42.5 | 56.5 | 20.5 | 53.5 | 43.3 | |
| CaRBackbone=Qwen-2, Selection Ratio=10%2025.02 | 42.5 | 60 | 18.5 | 53 | 43.5 | |
| CaRInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 42.5 | 58 | 16.5 | 51 | 42 | |
| NuggetsInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 42.5 | 56 | 16.5 | 51 | 41.5 | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 42.5 | 58 | 16.5 | 52 | 42.3 | |
| NuggetsBackbone=Qwen-2, Selection Ratio=10%2025.02 | 42 | 60 | 20 | 51.5 | 43.4 | |
| IFDBackbone=Qwen-2, Selection Ratio=15%2025.02 | 42 | 61.5 | 18.5 | 52 | 43.5 | |
| FLAME-DPO factInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 42 | 55.5 | 16.5 | 52 | 41.5 | |
| FLAME-DPOBackbone=LLaMA-1, Selection Ratio=100%2025.02 | 41.5 | 55 | 21.5 | 52.5 | 42.6 | |
| IFDBackbone=Qwen-2, Selection Ratio=10%2025.02 | 41.5 | 59.5 | 19.5 | 51 | 42.9 | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=5%2025.02 | 41.5 | 57 | 15.5 | 51.5 | 41.4 | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 41.5 | 54.5 | 16.5 | 50 | 40.6 | |
| CaRBackbone=LLaMA-1, Selection Ratio=10%2025.02 | 41 | 52 | 18 | 49.5 | 40.1 | |
| NuggetsInstruction Dataset=Alpaca - GPT4, Sample Ratio=5%2025.02 | 41 | 56 | 17 | 52 | 41.5 | |
| NuggetsBackbone=Qwen-2, Selection Ratio=15%2025.02 | 40.5 | 62.5 | 20 | 52.5 | 43.9 | |
| VanillaInstruction Dataset=Alpaca, Sample Ratio=100%2025.02 | 40.5 | 53.5 | 16 | 49.5 | 39.9 | |
| IFDInstruction Dataset=Alpaca, Sample Ratio=10%2025.02 | 40.5 | 60 | 17.5 | 53.5 | 42.9 | |
| IFDInstruction Dataset=Alpaca - GPT4, Sample Ratio=10%2025.02 | 40.5 | 56 | 16 | 49.5 | 40.5 | |
| IFDBackbone=LLaMA-1, Selection Ratio=10%2025.02 | 40 | 54.5 | 20 | 51 | 41.4 | |
| NuggetsBackbone=LLaMA-1, Selection Ratio=15%2025.02 | 40 | 52.5 | 15.5 | 50.5 | 39.6 | |
| NuggetsBackbone=LLaMA-1, Selection Ratio=5%2025.02 | 39.5 | 54.5 | 20 | 50 | 41 | |
| NuggetsBackbone=LLaMA-1, Selection Ratio=10%2025.02 | 39.5 | 53 | 17.5 | 51 | 40.3 | |
| IFDBackbone=LLaMA-1, Selection Ratio=15%2025.02 | 39.5 | 52 | 17.5 | 49.5 | 39.6 | |
| VanillaBackbone=Qwen-2, Selection Ratio=100%2025.02 | 39.5 | 57.5 | 18.5 | 49 | 41.1 | |
| VanillaInstruction Dataset=Alpaca - GPT4, Sample Ratio=100%2025.02 | 39.5 | 49.5 | 14.5 | 49 | 38.1 | |
| IFDBackbone=LLaMA-1, Selection Ratio=5%2025.02 | 38 | 53.5 | 18.5 | 49 | 39.8 | |
| CaRBackbone=LLaMA-1, Selection Ratio=5%2025.02 | 38 | 53 | 19 | 50.5 | 40.1 | |
| CaRBackbone=LLaMA-1, Selection Ratio=15%2025.02 | 38 | 51.5 | 17 | 48 | 38.6 | |
| VanillaBackbone=LLaMA-1, Selection Ratio=100%2025.02 | 37.5 | 50.5 | 16 | 47.5 | 37.9 |