Faithfulness Evaluation on MEMERAG
79.7Balanced Accuracy (DE)UCS (EN)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| UCS (EN)Backbone=Qwen-235B, Training Supervision Language=English2026.03 | 79.7 | 84.3 | 80 | 82.2 | 81.6 | |
| Zero-shotBackbone=Qwen-235B, Prompting Strategy=Zero-shot2026.03 | 77.2 | 77.5 | 77.5 | 74.4 | 76.7 | |
| AGBackbone=Qwen-235B, Prompting Strategy=Annotation Guidelines2026.03 | 77.2 | 80.6 | 76.1 | 78 | 78 | |
| RocketEvalBackbone=Qwen-235B, Evaluator Type=Checklist-based2026.03 | 76.4 | 81 | 75.9 | 79.1 | 78.1 | |
| CheckEvalBackbone=Qwen-235B, Evaluator Type=Checklist-based2026.03 | 76.1 | 80.7 | 74.8 | 78.4 | 77.5 | |
| CoTBackbone=Qwen-235B, Prompting Strategy=Chain-of-Thought2026.03 | 73 | 76.2 | 76.2 | 74.1 | 74.9 | |
| ChatEvalBackbone=Qwen-235B, Evaluator Type=Multi-agent debate2026.03 | 71.8 | 73 | 60.9 | 62.4 | 67 |