Truthfulness Evaluation on TruthfulQA medical (test)
83.6Health ScoreBioMistral 7B TIES
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BioMistral 7B TIESPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 83.6 | 75 | 42.1 | 44.4 | 61.3 | |
| GPT-3.5 Turbo 1106Prompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 80 | 68.8 | 42.1 | 44.4 | 58.8 | |
| BioMistral 7BPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 78.2 | 75 | 36.8 | 55.6 | 61.4 | |
| BioMistral 7B EnsemblePrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 74.5 | 71.6 | 60 | 56.1 | 65.6 | |
| BioMistral 7BPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 72.7 | 68.8 | 31.6 | 33.3 | 51.6 | |
| BioMistral 7B DAREPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 70.9 | 75 | 36.8 | 33.3 | 54 | |
| BioMistral 7B EnsemblePrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 69.1 | 59.5 | 52 | 50.1 | 57.6 | |
| BioMistral 7B TIESPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 69.1 | 68.8 | 36.8 | 33.3 | 52 | |
| BioMistral 7B SLERPPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 69.1 | 81.2 | 36.8 | 33.3 | 55.1 | |
| BioMistral 7B DAREPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 67.3 | 50 | 36.8 | 44.4 | 49.6 | |
| GPT-3.5 Turbo 1106Prompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 65.5 | 62.5 | 42.1 | 44.4 | 53.6 | |
| BioMistral 7B SLERPPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 63.6 | 68.8 | 36.8 | 44.4 | 53.4 | |
| Mistral 7B InstructPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 61.8 | 56.2 | 31.6 | 44.4 | 48.5 | |
| Mistral 7B InstructPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 60 | 43.8 | 42.1 | 44.4 | 47.5 | |
| MedAlpaca 7BPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 41.8 | 18.8 | 26.3 | 22.2 | 27.3 | |
| BioMedGPT-LM-7BPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 40 | 18.8 | 26.3 | 44.4 | 32.37 | |
| BioMedGPT-LM-7BPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 36.4 | 25 | 15.8 | 33.3 | 27.62 | |
| MedAlpaca 7BPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 34.5 | 12.5 | 15.8 | 33.3 | 24 | |
| MediTron-7BPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 16.4 | 18.8 | 5.3 | 0 | 10.1 | |
| MediTron-7BPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 14.5 | 25 | 0 | 0 | 9.8 | |
| PMC-LLaMA 7BPrompt=Prompt 2 - Truthful answer prompt, Evaluation Protocol=zero-shot2024.02 | 10.9 | 25 | 10.5 | 0 | 11.6 | |
| PMC-LLaMa 7BPrompt=Prompt 1 - QA prompt, Evaluation Protocol=zero-shot2024.02 | 9.1 | 25 | 10.5 | 0 | 11.1 |