Probabilistic Multiple-Choice on TruthfulQA (single info)
59.2MC1 ScoreMistral-Instruct-v0.2 + TACS-T
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Mistral-Instruct-v0.2 + TACS-TBackbone=Mistral-Instruct-v0.2, Granularity=token-level (TACS-T)2024.03 | 59.2 | 69 | 44.8 | 57.7 | |
| Mistral-Instruct-v0.2 + TACS-SBackbone=Mistral-Instruct-v0.2, Granularity=sentence-level (TACS-S)2024.03 | 55.8 | 59.4 | 39.9 | 51.7 | |
| Mistral-Instruct-v0.2Backbone=Mistral-Instruct-v0.22024.03 | 53.6 | 56.4 | 37 | 49 | |
| Llama 2-Chat + TACS-SBackbone=Llama 2-Chat, Granularity=sentence-level (TACS-S)2024.03 | 50.8 | 57.8 | 33.7 | 47.5 | |
| Llama 2-ChatBackbone=Llama 2-Chat2024.03 | 50.6 | 51.7 | 31.1 | 44.5 | |
| Llama 2-Chat + ITIBackbone=Llama 2-Chat, Enhancement=ITI2024.03 | 50.6 | 51.2 | 30.5 | 44.1 | |
| Llama 2-Chat + TACS-TBackbone=Llama 2-Chat, Granularity=token-level (TACS-T)2024.03 | 48.8 | 56.7 | 33.4 | 46.3 |