Factuality-based Question Answering on FreshQA 2025/11/24
44CBase
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BaseBackbone=Gemma-2-9b, Size Class=Mid-Size Models (7B-12B)2026.02 | 44 | 44 | 12 | 88 | 50 | — | — | — | |
| BaseBackbone=Gemma-3-12b, Size Class=Mid-Size Models (7B-12B)2026.02 | 44 | 56 | — | — | 44 | — | — | — | |
| VeriFY-MMBackbone=Gemma-2-9b, Size Class=Mid-Size Models (7B-12B)2026.02 | 38.4 | 27.6 | 5.4 | 28.6 | 66 | 58.2 | 87.7 | 69.9 | |
| BaseBackbone=Llama-3.1-8B, Size Class=Mid-Size Models (7B-12B)2026.02 | 37.4 | 44.8 | 17.8 | 82.2 | 45.5 | — | — | — | |
| VeriFY-NMBackbone=Gemma-3-12b, Size Class=Mid-Size Models (7B-12B)2026.02 | 36.4 | 27 | 7 | 29.6 | 63.4 | 57.4 | 83.9 | 68.2 | |
| VeriFY-SMBackbone=Gemma-2-9b, Size Class=Mid-Size Models (7B-12B)2026.02 | 36 | 27.8 | 6.8 | 29.4 | 63.8 | 56.4 | 84.1 | 67.5 | |
| BaseBackbone=Llama-3.2-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 35.8 | 47.2 | 17 | 83 | 43.1 | — | — | — | |
| VeriFY-NMBackbone=Llama-3.1-8B, Size Class=Mid-Size Models (7B-12B)2026.02 | 35.6 | 38.6 | 4.4 | 21.4 | 74.2 | 48 | 89 | 62.4 | |
| VeriFY-MMBackbone=Gemma-3-12b, Size Class=Mid-Size Models (7B-12B)2026.02 | 35.6 | 29.6 | 7 | 27.8 | 65.2 | 54.6 | 83.6 | 66 | |
| VeriFY-NMBackbone=Gemma-2-9b, Size Class=Mid-Size Models (7B-12B)2026.02 | 35.2 | 24 | 7.6 | 33.2 | 59.2 | 59.5 | 82.2 | 69 | |
| VeriFY-SMBackbone=Gemma-3-12b, Size Class=Mid-Size Models (7B-12B)2026.02 | 35.2 | 24.6 | 7.6 | 32.6 | 59.8 | 58.9 | 82.2 | 68.6 | |
| VeriFY-MMBackbone=Llama-3.1-8B, Size Class=Mid-Size Models (7B-12B)2026.02 | 35 | 36.2 | 5.2 | 23.6 | 71.2 | 49.2 | 87.1 | 62.8 | |
| BaseBackbone=Gemma-2-2b, Size Class=Small-Size Models (2B-4B)2026.02 | 34.6 | 58.2 | 7.2 | 92.8 | 37.3 | — | — | — | |
| VeriFY-SMBackbone=Llama-3.1-8B, Size Class=Mid-Size Models (7B-12B)2026.02 | 34.2 | 38.6 | 3.8 | 23.4 | 72.8 | 47 | 90 | 61.7 | |
| BaseBackbone=Gemma-3-4b, Size Class=Small-Size Models (2B-4B)2026.02 | 33.4 | 66.6 | — | — | 33.4 | — | — | — | |
| BaseBackbone=Qwen2.5-7B, Size Class=Mid-Size Models (7B-12B)2026.02 | 32.2 | 61.6 | 6.2 | 93.8 | 34.3 | — | — | — | |
| VeriFY-NMBackbone=Llama-3.2-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 31.4 | 35.8 | 6.2 | 26.6 | 67.2 | 46.7 | 83.5 | 59.9 | |
| VeriFY-MMBackbone=Gemma-2-2b, Size Class=Small-Size Models (2B-4B)2026.02 | 28.8 | 29.6 | 6.4 | 35.2 | 58.4 | 49.3 | 81.8 | 61.5 | |
| VeriFY-SMBackbone=Llama-3.2-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 28.6 | 33.4 | 6.8 | 31.2 | 62 | 46.1 | 80.8 | 58.7 | |
| VeriFY-MMBackbone=Llama-3.2-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 28.4 | 33.6 | 8.2 | 29.8 | 62 | 45.8 | 77.6 | 57.6 | |
| VeriFY-SMBackbone=Gemma-2-2b, Size Class=Small-Size Models (2B-4B)2026.02 | 28 | 34 | 7.6 | 30.4 | 62 | 45.2 | 78.7 | 57.4 | |
| BaseBackbone=Qwen2.5-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 28 | 54.6 | 17.4 | 82.6 | 33.9 | — | — | — | |
| KPBackbone=Gemma-3-12b, Size Class=Mid-Size Models (7B-12B)2026.02 | 26.4 | 15 | 17.2 | 41.4 | 41.4 | 63.8 | 60.5 | 62.1 | |
| KPBackbone=Gemma-2-9b, Size Class=Mid-Size Models (7B-12B)2026.02 | 26.2 | 12.8 | 16.6 | 44.4 | 39 | 67.2 | 61.2 | 64.1 | |
| KPBackbone=Llama-3.1-8B, Size Class=Mid-Size Models (7B-12B)2026.02 | 25.8 | 20.8 | 12.4 | 41 | 46.6 | 55.4 | 67.5 | 60.9 | |
| VeriFY-MMBackbone=Gemma-3-4b, Size Class=Small-Size Models (2B-4B)2026.02 | 25.6 | 27 | 7.4 | 40 | 52.6 | 48.7 | 77.6 | 59.8 | |
| VeriFY-SMBackbone=Gemma-3-4b, Size Class=Small-Size Models (2B-4B)2026.02 | 25.4 | 19.8 | 9 | 45.8 | 45.2 | 56.2 | 73.8 | 63.8 | |
| VeriFY-MMBackbone=Qwen2.5-7B, Size Class=Mid-Size Models (7B-12B)2026.02 | 24.4 | 23.6 | 8.4 | 43.6 | 48 | 50.8 | 74.4 | 60.4 | |
| VeriFY-SMBackbone=Qwen2.5-7B, Size Class=Mid-Size Models (7B-12B)2026.02 | 24.4 | 24.4 | 9.2 | 42 | 48.8 | 50 | 72.6 | 59.2 | |
| KPBackbone=Gemma-2-2b, Size Class=Small-Size Models (2B-4B)2026.02 | 23.6 | 20 | 10.4 | 46 | 43.6 | 54.1 | 69.4 | 60.8 | |
| VeriFY-NMBackbone=Gemma-2-2b, Size Class=Small-Size Models (2B-4B)2026.02 | 23.6 | 27.6 | 10.8 | 38 | 51.2 | 46.1 | 68.6 | 55.1 | |
| VeriFY-NMBackbone=Gemma-3-4b, Size Class=Small-Size Models (2B-4B)2026.02 | 23.2 | 18.8 | 10 | 48 | 42 | 55.2 | 69.9 | 61.7 | |
| VeriFY-NMBackbone=Qwen2.5-7B, Size Class=Mid-Size Models (7B-12B)2026.02 | 23 | 23.2 | 9.4 | 44.4 | 46.2 | 49.8 | 71 | 58.5 | |
| KPBackbone=Llama-3.2-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 21.6 | 21.4 | 12.4 | 44.6 | 43 | 50.2 | 63.5 | 56.1 | |
| VeriFY-SMBackbone=Qwen2.5-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 19.8 | 29.6 | 7.6 | 43 | 49.4 | 40.1 | 72.3 | 51.6 | |
| KPBackbone=Gemma-3-4b, Size Class=Small-Size Models (2B-4B)2026.02 | 19.4 | 14.8 | 14 | 51.8 | 34.2 | 56.7 | 58.1 | 57.4 | |
| VeriFY-MMBackbone=Qwen2.5-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 18.8 | 21.6 | 9.4 | 50.2 | 40.4 | 46.5 | 66.7 | 54.8 | |
| VeriFY-NMBackbone=Qwen2.5-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 17.8 | 19.8 | 11.4 | 51 | 37.6 | 47.3 | 61 | 53.3 | |
| KPBackbone=Qwen2.5-7B, Size Class=Mid-Size Models (7B-12B)2026.02 | 17.4 | 13.4 | 14.4 | 54.8 | 30.8 | 56.5 | 54.7 | 55.6 | |
| KPBackbone=Qwen2.5-3B, Size Class=Small-Size Models (2B-4B)2026.02 | 9 | 8 | 18 | 65 | 17 | 52.9 | 33.3 | 40.9 |