False Premise Detection on CREAK
92.8TP (%)G-retriever
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| G-retrieverRetrieval/Query Configuration=Logical Form for Both Stages2025.04 | 92.8 | 83.8 | 16.2 | 7.2 | 88.9 | 88.3 | |
| GraphRAG/ToGRetrieval/Query Configuration=Logical Form for Both Stages2025.04 | 92.8 | 91.2 | 8.8 | 7.2 | 92.1 | 92 | |
| GraphRAG/ToGRetrieval/Query Configuration=Original Query for Both Stages2025.04 | 89.9 | 91.2 | 8.8 | 10.1 | 90.5 | 90.5 | |
| GraphRAG/ToGRetrieval/Query Configuration=Logical Form for Retrieval and Original Query for False Premise Detection2025.04 | 89.9 | 88.2 | 11.8 | 10.1 | 89.2 | 89.1 | |
| with RAGRetrieval/Query Configuration=Logical Form for Both Stages2025.04 | 88.4 | 92.6 | 7.4 | 11.6 | 90.4 | 90.5 | |
| with RAGRetrieval/Query Configuration=Logical Form for Retrieval and Original Query for False Premise Detection2025.04 | 76.8 | 92.6 | 7.4 | 23.2 | 83.5 | 84.7 | |
| Direct ClaimRetrieval/Query Configuration=Original Query for Both Stages2025.04 | 72.5 | 89.7 | 10.3 | 27.5 | 79.4 | 81 | |
| Direct ClaimRetrieval/Query Configuration=Logical Form for Retrieval and Original Query for False Premise Detection2025.04 | 72.5 | 89.7 | 10.3 | 27.5 | 79.4 | 81 | |
| Direct ClaimRetrieval/Query Configuration=Logical Form for Both Stages2025.04 | 72.5 | 89.7 | 10.3 | 27.5 | 79.4 | 81 | |
| with RAGRetrieval/Query Configuration=Original Query for Both Stages2025.04 | 62.3 | 86.8 | 13.2 | 37.7 | 71.1 | 74.5 | |
| G-retrieverRetrieval/Query Configuration=Logical Form for Retrieval and Original Query for False Premise Detection2025.04 | 36.2 | 83.8 | 16.2 | 63.8 | 47.6 | 59.9 | |
| G-retrieverRetrieval/Query Configuration=Original Query for Both Stages2025.04 | 24.6 | 92.6 | 7.4 | 75.4 | 37.4 | 58.4 |