Factual Question Answering on TVQA ID
82.1PrecisionUALIGN
Evaluation results
| Method | Links | ||
|---|---|---|---|
| UALIGNBackbone=Mistral-7B2024.12 | 82.1 | 59.05 | |
| RLKFBackbone=Mistral-7B2024.12 | 80.43 | 56.92 | |
| UALIGN (w/o Entro.)Backbone=Mistral-7B2024.12 | 80.18 | 57.64 | |
| RL-PPOBackbone=Mistral-7B2024.12 | 79.23 | 55.08 | |
| UALIGNBackbone=Llama-3-8B2024.12 | 79.14 | 57.04 | |
| UALIGN (w/o Entro.)Backbone=Llama-3-8B2024.12 | 78.43 | 57.69 | |
| ICLBackbone=Mistral-7B2024.12 | 77.92 | 55.14 | |
| RLKFBackbone=Llama-3-8B2024.12 | 77.12 | 56.07 | |
| ICL-COTBackbone=Mistral-7B2024.12 | 76.73 | 54.78 | |
| UALIGN (w/o Conf.)Backbone=Mistral-7B2024.12 | 76.44 | 55.13 | |
| RL-PPOBackbone=Llama-3-8B2024.12 | 76.32 | 55.19 | |
| ICLBackbone=Llama-3-8B2024.12 | 76.15 | 56.55 | |
| ITIBackbone=Mistral-7B2024.12 | 74.65 | 55.16 | |
| SFTBackbone=Mistral-7B2024.12 | 74.57 | 54.77 | |
| UALIGN (w/o Conf.)Backbone=Llama-3-8B2024.12 | 74.13 | 54.45 | |
| R-TuningBackbone=Llama-3-8B2024.12 | 72.93 | 55.44 | |
| ICL-IDKBackbone=Mistral-7B2024.12 | 72.59 | 51.37 | |
| RL-DPOBackbone=Mistral-7B2024.12 | 72.2 | 52.98 | |
| RL-DPOBackbone=Llama-3-8B2024.12 | 72.08 | 53.96 | |
| ITIBackbone=Llama-3-8B2024.12 | 71.09 | 53.97 | |
| SFTBackbone=Llama-3-8B2024.12 | 70.8 | 52.57 | |
| ICL-IDKBackbone=Llama-3-8B2024.12 | 69.17 | 54.1 | |
| R-TuningBackbone=Mistral-7B2024.12 | 67.7 | 52.25 | |
| ICL-COTBackbone=Llama-3-8B2024.12 | 66.68 | 53.37 |