Citation-augmented Question Answering on GT, PK 1.0 (test)
90.55AccuracyFOOTNOTE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FOOTNOTEModel=DeepSeek-R12025.04 | 90.55 | 56.78 | 50.33 | 51.25 | 3.2 | 3.87 | 0.17 | |
| INTRALIGNModel=Llama-3.1-70B-Instruct2025.04 | 85.72 | 68.45 | 88.1 | 78.79 | 3.69 | 4.42 | 0.1 | |
| GUIDED-RAELModel=DeepSeek-R12025.04 | 84.77 | 53.89 | 49.47 | 51.45 | 3.28 | 4.04 | 0.15 | |
| FOOTNOTEModel=GPT-4o2025.04 | 82.19 | 57.22 | 52.14 | 51.75 | 3.42 | 3.61 | 0.18 | |
| POSTCITEModel=GPT-4o2025.04 | 81.81 | 39.54 | 81.95 | 56.32 | 2.94 | 2.95 | 0.2 | |
| Guided-RAELModel=GPT-4o2025.04 | 81.59 | 62.94 | 66.67 | 58.84 | 3.58 | 4 | 0.1 | |
| FOOTNOTEModel=o1-mini2025.04 | 77.53 | 53.92 | 43.56 | 49.01 | 3.26 | 3.89 | 0.15 | |
| FRONTModel=Llama-3.1-70B-Instruct, Knowledge Restriction=External knowledge only2025.04 | 76.65 | 72.82 | — | 57.61 | 3.26 | 2.53 | — | |
| INTRALIGNModel=Llama-3.1-8B-Instruct2025.04 | 75.9 | 67.63 | 85.8 | 63.72 | 3.61 | 4.05 | 0.1 | |
| FOOTNOTEModel=Llama-3.1-70B-Instruct2025.04 | 75.83 | 52.91 | 32.14 | 30.36 | 3.43 | 4.1 | 0.23 | |
| Guided-RAELModel=Llama-3.1-70B-Instruct2025.04 | 73.06 | 62 | 66.67 | 59.79 | 3.32 | 3.97 | 0.13 | |
| CONTEXTCITEModel=Llama-3.1-70B-Instruct, Knowledge Restriction=External knowledge only2025.04 | 72.49 | 33.08 | — | 33.06 | 3.51 | 3.19 | — | |
| FRONTModel=Llama-3.1-8B-Instruct, Knowledge Restriction=External knowledge only2025.04 | 72.08 | 71.84 | — | 48 | 3.42 | 2.01 | — | |
| POSTCITEModel=Llama-3.1-8B-Instruct2025.04 | 71.64 | 31.01 | 30.77 | 29.82 | 3.47 | 2.25 | 0.14 | |
| CONTEXTCITEModel=Llama-3.1-8B-Instruct, Knowledge Restriction=External knowledge only2025.04 | 71.1 | 25.25 | — | 25.08 | 3.43 | 3.47 | — | |
| GUIDED-RAELModel=o1-mini2025.04 | 70.88 | 56.25 | 47.92 | 52.17 | 3.23 | 4 | 0.13 | |
| FOOTNOTEModel=Llama-3.1-8B-Instruct2025.04 | 65.91 | 51.2 | 44.38 | 16.03 | 3.32 | 3.96 | 0.2 | |
| POSTCITEModel=Llama-3.1-70B-Instruct2025.04 | 64.82 | 23.86 | 66.29 | 43.71 | 3.69 | 1.81 | 0.19 | |
| Guided-RAELModel=Llama-3.1-8B-Instruct2025.04 | 62.87 | 61.54 | 74.46 | 57.33 | 3.14 | 3.97 | 0.12 | |
| RECITEModel=Llama-3.1-8B-Instruct, Knowledge Restriction=Parameter knowledge only2025.04 | 61.38 | — | 65.71 | 60.23 | 3.37 | 3.36 | 0.29 | |
| RECITEModel=Llama-3.1-70B-Instruct, Knowledge Restriction=Parameter knowledge only2025.04 | 45.05 | — | 81.62 | 77.86 | 3.54 | 3.86 | 0.22 |