Multimodal RAG Evaluation on MAGMaR Extrinsic Quality Judgment (EQJ)
65Information Preservation (Retrieval)INFOF1
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| INFOF1Judgment Type=EQJ, Annotator=22025.10 | 65 | -12 | 38 | — | — | — | — | — | |
| INFOF1Judgment Type=EQJ, Annotator=32025.10 | 52 | 14 | 25 | — | — | — | — | — | |
| INFOF1Judgment Type=EQJ, Annotator=12025.10 | 41 | 4 | 31 | — | — | — | — | — | |
| BERTScoreJudgment Type=EQJ, Annotator=12025.10 | — | — | — | — | — | — | — | 9 | |
| BERTScoreJudgment Type=EQJ, Annotator=22025.10 | — | — | — | — | — | — | — | 7 | |
| BERTScoreJudgment Type=EQJ, Annotator=32025.10 | — | — | — | — | — | — | — | 15 | |
| CITEF1Judgment Type=EQJ, Annotator=12025.10 | — | — | — | 26 | 26 | 28 | — | — | |
| CITEF1Judgment Type=EQJ, Annotator=22025.10 | — | — | — | 50 | 3 | 48 | — | — | |
| CITEF1Judgment Type=EQJ, Annotator=32025.10 | — | — | — | 38 | 22 | 18 | — | — | |
| ROUGE-LJudgment Type=EQJ, Annotator=12025.10 | — | — | — | — | — | — | -4 | — | |
| ROUGE-LJudgment Type=EQJ, Annotator=22025.10 | — | — | — | — | — | — | -1 | — | |
| ROUGE-LJudgment Type=EQJ, Annotator=32025.10 | — | — | — | — | — | — | 16 | — |