Direct Question Answering on 5% (forget set)
36ROUGE-L ScoreNPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NPOModel=Gemma-2B-IT, Variant=Context-aware2025.10 | 36 | 25 | |
| UNDIALModel=Gemma-2B-IT, Variant=Context-aware2025.10 | 34 | 38 | |
| UNDIALModel=Gemma-2B-IT, Variant=Vanilla2025.10 | 33 | 39 | |
| UNDIALModel=Qwen3-8B, Variant=Context-aware2025.10 | 33 | 39 | |
| UNDIALModel=Qwen3-8B, Variant=Vanilla2025.10 | 32 | 38 | |
| NPOModel=Gemma-2B-IT, Variant=Vanilla2025.10 | 31 | 19 | |
| NPOModel=Qwen3-8B, Variant=Context-aware2025.10 | 29 | 20 | |
| NPOModel=Qwen3-8B, Variant=Vanilla2025.10 | 27 | 14 | |
| RMUModel=Gemma-2B-IT, Variant=Context-aware2025.10 | 13 | 1 | |
| RMUModel=Qwen3-8B, Variant=Context-aware2025.10 | 13 | 1 | |
| RMUModel=Qwen3-8B, Variant=Vanilla2025.10 | 10 | 0 | |
| RMUModel=Gemma-2B-IT, Variant=Vanilla2025.10 | 4 | 0 |