Question Answering on SQuAD (test) (GPT-Judge Metrics)
89GPT Judge AccuracyCoT with SSRAG
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| CoT with SSRAGModel=LLaMA 22026.01 | 89 | 79 | 80 | 0.2 | |
| CoVe with SSRAGModel=LLaMA 22026.01 | 89 | 81.2 | 86 | 0.198 | |
| SSRAGModel=GPT-42026.01 | 84 | 82.3 | 86 | 0.15 | |
| CoVe with SSRAGModel=GPT-42026.01 | 84 | 77 | 83 | 0.13 | |
| CoT with SSRAGModel=GPT-42026.01 | 83 | 83.4 | 82 | 0.13 | |
| SSRAGModel=Gemini 1.52026.01 | 83 | 78.9 | 93 | 0.27 | |
| SSRAGModel=LLaMA 22026.01 | 82 | 82.3 | 90 | 0.21 | |
| SSRAGModel=IBM Granite2026.01 | 81 | 81.2 | 94 | 0.21 | |
| CoT with SSRAGModel=Gemini 1.52026.01 | 81 | 71 | 87 | 0.28 | |
| CoVe with SSRAGModel=Gemini 1.52026.01 | 78 | 72 | 86 | 0.28 | |
| CoVe with SSRAGModel=IBM Granite2026.01 | 77 | 74 | 87 | 0.22 | |
| CoT with SSRAGModel=IBM Granite2026.01 | 76 | 76.5 | 85 | 0.23 | |
| SSRAGModel=TinyLLaMA2026.01 | 74 | 85.2 | 96 | 0.29 | |
| CoT with SSRAGModel=TinyLLaMA2026.01 | 74 | 77 | 84 | 0.29 | |
| CoVe with SSRAGModel=TinyLLaMA2026.01 | 69 | 78 | 87 | 0.3 | |
| Graph RAGModel=GPT-42026.01 | 66 | 58 | 65 | 0.37 | |
| Graph RAGModel=Gemini 1.52026.01 | 65 | 56 | 57 | 0.43 | |
| CoVe with GraphModel=GPT-42026.01 | 63 | 57.7 | 58 | 0.39 | |
| Graph RAGModel=IBM Granite2026.01 | 63 | 57 | 59 | 0.39 | |
| CoT with Graph RAGModel=GPT-42026.01 | 62 | 54.7 | 63 | 0.33 | |
| Graph RAGModel=LLaMA 22026.01 | 62 | 54 | 63 | 0.32 | |
| CoVeModel=GPT-42026.01 | 58 | 48 | 41 | 0.47 | |
| CoVe with GraphModel=LLaMA 22026.01 | 58 | 56 | 60 | 0.38 | |
| CoT with Graph RAGModel=LLaMA 22026.01 | 57 | 58.2 | 66 | 0.39 | |
| CoVe with GraphModel=IBM Granite2026.01 | 57 | 56 | 59 | 0.37 | |
| CoTModel=GPT-42026.01 | 56 | 47 | 45 | 0.43 | |
| CoT with Graph RAGModel=IBM Granite2026.01 | 56 | 59.9 | 69 | 0.32 | |
| CoVe with GraphModel=Gemini 1.52026.01 | 56 | 54 | 60 | 0.31 | |
| CoT with Graph RAGModel=Gemini 1.52026.01 | 55 | 59.8 | 69 | 0.3 | |
| Baseline RAGModel=GPT-42026.01 | 54 | 43.2 | 47 | 0.52 | |
| Baseline RAGModel=LLaMA 22026.01 | 53 | 49.9 | 51 | 0.53 | |
| Baseline RAGModel=Gemini 1.52026.01 | 53 | 49.7 | 56 | 0.41 | |
| CoVeModel=LLaMA 22026.01 | 52 | 52.3 | 50 | 0.52 | |
| Graph RAGModel=TinyLLaMA2026.01 | 52 | 49 | 58 | 0.41 | |
| CoTModel=LLaMA 22026.01 | 51 | 47.8 | 51 | 0.49 | |
| Baseline RAGModel=IBM Granite2026.01 | 51 | 52 | 54 | 0.42 | |
| CoVeModel=IBM Granite2026.01 | 51 | 53 | 50 | 0.511 | |
| CoVe with GraphModel=TinyLLaMA2026.01 | 50 | 57 | 58 | 0.39 | |
| CoTModel=IBM Granite2026.01 | 50 | 54.3 | 50 | 0.43 | |
| CoVeModel=Gemini 1.52026.01 | 50 | 47.7 | 51 | 0.54 | |
| Baseline RAGModel=TinyLLaMA2026.01 | 49 | 31.5 | 47 | 0.47 | |
| CoT with Graph RAGModel=TinyLLaMA2026.01 | 49 | 59.9 | 66 | 0.41 | |
| CoTModel=Gemini 1.52026.01 | 49 | 49.8 | 51 | 0.46 | |
| CoVeModel=TinyLLaMA2026.01 | 44 | 51 | 54 | 0.59 | |
| CoTModel=TinyLLaMA2026.01 | 43 | 49.1 | 54 | 0.52 |