Report Generation on IU-XRay (Factual Consistency, Completeness)
85.71Factual Consistency (FC)clean RAG
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| clean RAGModel=Gemini-2.5, Retriever=clean RAG, Top-K=12026.05 | 85.71 | 82.86 | |
| clean RAGModel=GPT-5, Retriever=clean RAG, Top-K=12026.05 | 84.76 | 77.14 | |
| clean RAGModel=LLaVA-Med, Retriever=clean RAG, Top-K=12026.05 | 84.76 | 14.29 | |
| LIARModel=GPT-5, Retriever=SigLIP, Top-K=12026.05 | 83.54 | 75.08 | |
| LIARModel=Gemini-2.5, Retriever=SigLIP, Top-K=12026.05 | 82.82 | 80.09 | |
| M3AttModel=GPT-5, Retriever=SigLIP, Top-K=12026.05 | 81.7 | 72.83 | |
| M3AttModel=Gemini-2.5, Retriever=SigLIP, Top-K=12026.05 | 79.8 | 77.08 | |
| clean RAGModel=GPT-4o, Retriever=clean RAG, Top-K=12026.05 | 79.05 | 69.52 | |
| LIARModel=GPT-4o, Retriever=SigLIP, Top-K=12026.05 | 76.88 | 67.07 | |
| clean RAGModel=Claude-4.5, Retriever=clean RAG, Top-K=12026.05 | 76.19 | 80.95 | |
| LIARModel=LLaVA-Med, Retriever=SigLIP, Top-K=12026.05 | 76.06 | 12.77 | |
| LIARModel=Claude-4.5, Retriever=SigLIP, Top-K=12026.05 | 74.52 | 78.02 | |
| M3AttModel=GPT-4o, Retriever=SigLIP, Top-K=12026.05 | 74.36 | 64.52 | |
| M3AttModel=Claude-4.5, Retriever=SigLIP, Top-K=12026.05 | 72.52 | 74.89 | |
| M3AttModel=LLaVA-Med, Retriever=SigLIP, Top-K=12026.05 | 68.4 | 11.44 | |
| w/o RAGModel=Claude-4.5, Retriever=w/o RAG, Top-K=12026.05 | 66.11 | 53.89 | |
| w/o RAGModel=GPT-5, Retriever=w/o RAG, Top-K=12026.05 | 61.11 | 50.56 | |
| w/o RAGModel=Gemini-2.5, Retriever=w/o RAG, Top-K=12026.05 | 46.67 | 46.11 | |
| w/o RAGModel=LLaVA-Med, Retriever=w/o RAG, Top-K=12026.05 | 20 | 0 | |
| w/o RAGModel=GPT-4o, Retriever=w/o RAG, Top-K=12026.05 | 8.33 | 2.78 |