Long-form QA Factuality Detection on Long-form QA Benchmark
17.3PR-AUCFRANQ condition-calibrated
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FRANQ condition-calibratedLLM Backbone=Falcon 3B Base2025.05 | 17.3 | 0.354 | |
| CCPLLM Backbone=Falcon 3B Base2025.05 | 16.2 | 0.181 | |
| FRANQ calibratedLLM Backbone=Gemma 4B Instruct2025.05 | 15 | 0.401 | |
| FRANQ condition-calibratedLLM Backbone=Llama 3B Instruct2025.05 | 14 | 0.223 | |
| FRANQ no calibrationLLM Backbone=Falcon 3B Base2025.05 | 13.5 | 0.362 | |
| Max Token EntropyLLM Backbone=Falcon 3B Base2025.05 | 13 | 0.219 | |
| Max Claim Prob.LLM Backbone=Falcon 3B Base2025.05 | 12.6 | 0.258 | |
| XGBoost (all UQ features)LLM Backbone=Llama 3B Instruct2025.05 | 12.4 | 0.206 | |
| P(True)LLM Backbone=Llama 3B Instruct2025.05 | 11.7 | 0.207 | |
| Parametric KnowledgeLLM Backbone=Gemma 4B Instruct2025.05 | 11.2 | 0.183 | |
| XGBoost (FRANQ features)LLM Backbone=Llama 3B Instruct2025.05 | 11.1 | 0.149 | |
| Max Token EntropyLLM Backbone=Llama 3B Instruct2025.05 | 10.9 | 0.115 | |
| AlignScoreLLM Backbone=Falcon 3B Base2025.05 | 10.4 | 0.233 | |
| FRANQ calibratedLLM Backbone=Llama 3B Instruct2025.05 | 10.3 | 0.256 | |
| Max Token EntropyLLM Backbone=Llama 8B Instruct2025.05 | 10.2 | 0.138 | |
| FRANQ no calibrationLLM Backbone=Llama 3B Instruct2025.05 | 10 | 0.181 | |
| P(True)LLM Backbone=Gemma 4B Instruct2025.05 | 9.6 | 0.148 | |
| PerplexityLLM Backbone=Falcon 3B Base2025.05 | 9 | 0.165 | |
| XGBoost (FRANQ features)LLM Backbone=Gemma 4B Instruct2025.05 | 9 | 0.158 | |
| FRANQ condition-calibratedLLM Backbone=Gemma 4B Instruct2025.05 | 9 | 0.208 | |
| XGBoost (all UQ features)LLM Backbone=Falcon 3B Base2025.05 | 8.8 | 0.198 | |
| CCPLLM Backbone=Gemma 4B Instruct2025.05 | 8.7 | 0.216 | |
| CCPLLM Backbone=Llama 3B Instruct2025.05 | 8.5 | 0.169 | |
| FRANQ condition-calibratedLLM Backbone=Llama 8B Instruct2025.05 | 8.1 | 0.184 | |
| XGBoost (FRANQ features)LLM Backbone=Falcon 3B Base2025.05 | 8 | 0.086 | |
| FRANQ no calibrationLLM Backbone=Gemma 4B Instruct2025.05 | 8 | 0.2 | |
| P(True)LLM Backbone=Falcon 3B Base2025.05 | 7.7 | 0.17 | |
| PerplexityLLM Backbone=Llama 8B Instruct2025.05 | 7.5 | 0.09 | |
| AlignScoreLLM Backbone=Llama 3B Instruct2025.05 | 7.5 | 0.108 | |
| FRANQ calibratedLLM Backbone=Falcon 3B Base2025.05 | 7.4 | 0.09 | |
| XGBoost (all UQ features)LLM Backbone=Gemma 4B Instruct2025.05 | 7.3 | 0.085 | |
| P(True)LLM Backbone=Llama 8B Instruct2025.05 | 7.1 | 0.112 | |
| AlignScoreLLM Backbone=Llama 8B Instruct2025.05 | 6.8 | 0.119 | |
| Parametric KnowledgeLLM Backbone=Falcon 3B Base2025.05 | 6.7 | 0.029 | |
| Parametric KnowledgeLLM Backbone=Llama 3B Instruct2025.05 | 6.4 | 0.018 | |
| FRANQ no calibrationLLM Backbone=Llama 8B Instruct2025.05 | 6.3 | 0.162 | |
| Max Claim Prob.LLM Backbone=Gemma 4B Instruct2025.05 | 6.1 | 0 | |
| CCPLLM Backbone=Llama 8B Instruct2025.05 | 6.1 | 0.108 | |
| AlignScoreLLM Backbone=Gemma 4B Instruct2025.05 | 6.1 | 0.058 | |
| Parametric KnowledgeLLM Backbone=Llama 8B Instruct2025.05 | 5.9 | 0.047 | |
| Max Claim Prob.LLM Backbone=Llama 3B Instruct2025.05 | 5.8 | -0.029 | |
| PerplexityLLM Backbone=Llama 3B Instruct2025.05 | 5.6 | -0.081 | |
| Max Claim Prob.LLM Backbone=Llama 8B Instruct2025.05 | 5.5 | 0.118 | |
| Max Token EntropyLLM Backbone=Gemma 4B Instruct2025.05 | 5.1 | -0.003 | |
| PerplexityLLM Backbone=Gemma 4B Instruct2025.05 | 4.8 | -0.071 | |
| XGBoost (FRANQ features)LLM Backbone=Llama 8B Instruct2025.05 | 4.8 | 0.017 | |
| XGBoost (all UQ features)LLM Backbone=Llama 8B Instruct2025.05 | 4.4 | — | |
| FRANQ calibratedLLM Backbone=Llama 8B Instruct2025.05 | 4.3 | -0.047 |