Question Answering on PopQA v1.0 (test)
83.07A1EigV + MCH
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| EigV + MCHBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=MCH2025.05 | 83.07 | 12.35 | 96.31 | 37.18 | 70.39 | |
| ECC + MCHBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=MCH2025.05 | 81.77 | 11.11 | 96.08 | 36.53 | 72.71 | |
| SE + MCHBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=MCH2025.05 | 80.21 | 12.35 | 94.01 | 42.31 | 70.1 | |
| PE + NLIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=NLI2025.05 | 74.23 | 24.69 | 93.09 | 46.8 | 68.65 | |
| PE + MCHBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=MCH2025.05 | 74.22 | 17.28 | 91.01 | 48.72 | 68.63 | |
| SEBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=None2025.05 | 74.22 | 16.05 | 95.39 | 33.97 | 68.53 | |
| PE + CTIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=CTI2025.05 | 72.91 | 14.82 | 89.63 | 46.8 | 67.14 | |
| PEBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=None2025.05 | 70.57 | 22.22 | 89.17 | 44.23 | 65.72 | |
| SE + CTIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=CTI2025.05 | 70.57 | 12.35 | 93.55 | 37.18 | 67.1 | |
| SE + NLIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=NLI2025.05 | 69.01 | 25.93 | 94.24 | 39.74 | 70.53 | |
| EigV + CTIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=CTI2025.05 | 68.23 | 16.05 | 93.55 | 39.74 | 65.23 | |
| ECC + CTIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=CTI2025.05 | 67.45 | 18.52 | 94.24 | 37.82 | 68.36 | |
| EigV + NLIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=NLI2025.05 | 65.88 | 32.1 | 95.16 | 39.1 | 68.4 | |
| ECC + NLIBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=NLI2025.05 | 65.88 | 29.63 | 95.62 | 36.53 | 69.91 | |
| EigVBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=None2025.05 | 41.15 | 6.17 | 93.08 | 26.92 | 66.35 | |
| ECCBackbone=Mistral-v0.3, Retriever=Contriver, Calibration=None2025.05 | 40.62 | 4.93 | 92.16 | 23.08 | 66.28 |