List your company
ResearchFormerly WizWand

Factuality Question Answering on TruthfulQA (score)

66.92ScoreUNA-score & binary
38.621645.968353.31560.6617Aug 27, 2024
Updated 5mo ago

Evaluation results

MethodLinks
UNA-score & binaryBase Model=Qwen 8B, Alignment Strategy=UNA-score & binary, Feedback Type=Score-based & Binary2024.0866.92
UNA-score & binaryBase Model=Mistral 7B, Alignment Strategy=UNA-score & binary, Feedback Type=Score-based & Binary2024.0864.62
UNA-score (MSE)Base Model=Qwen 8B, Alignment Strategy=UNA-score (MSE), Feedback Type=Score-based2024.0864.44
UNA-score & binaryBase Model=Llama 8B, Alignment Strategy=UNA-score & binary, Feedback Type=Score-based & Binary2024.0861.24
UNA-score (MSE)Base Model=Llama 8B, Alignment Strategy=UNA-score (MSE), Feedback Type=Score-based2024.0860.46
UNA-score (MSE)Base Model=Mistral 7B, Alignment Strategy=UNA-score (MSE), Feedback Type=Score-based2024.0855.09
KTOBase Model=Llama 8B, Alignment Strategy=KTO, Feedback Type=Binary2024.0855.07
UNA-binary (BCE)Base Model=Llama 8B, Alignment Strategy=UNA-binary (BCE), Feedback Type=Binary2024.0854.75
DPOBase Model=Llama 8B, Alignment Strategy=DPO, Feedback Type=Pairwise2024.0853.47
UNA-pairwiseBase Model=Llama 8B, Alignment Strategy=UNA-pairwise, Feedback Type=Pairwise2024.0853.47
Qwen 8BBase Model=Qwen 8B, Alignment Strategy=Baseline2024.0852.19
UNA-binary (BCE)Base Model=Qwen 8B, Alignment Strategy=UNA-binary (BCE), Feedback Type=Binary2024.0852.1
KTOBase Model=Qwen 8B, Alignment Strategy=KTO, Feedback Type=Binary2024.0852.07
DPOBase Model=Qwen 8B, Alignment Strategy=DPO, Feedback Type=Pairwise2024.0851.78
UNA-pairwiseBase Model=Qwen 8B, Alignment Strategy=UNA-pairwise, Feedback Type=Pairwise2024.0851.78
UNA-binary (BCE)Base Model=Mistral 7B, Alignment Strategy=UNA-binary (BCE), Feedback Type=Binary2024.0848.33
KTOBase Model=Mistral 7B, Alignment Strategy=KTO, Feedback Type=Binary2024.0847.72
UNA-score & binaryBase Model=Gemma 4B, Alignment Strategy=UNA-score & binary, Feedback Type=Score-based & Binary2024.0846.69
UNA-score (MSE)Base Model=Gemma 4B, Alignment Strategy=UNA-score (MSE), Feedback Type=Score-based2024.0845.74
Llama 8BBase Model=Llama 8B, Alignment Strategy=Baseline2024.0845.16
DPOBase Model=Mistral 7B, Alignment Strategy=DPO, Feedback Type=Pairwise2024.0844.75
UNA-pairwiseBase Model=Mistral 7B, Alignment Strategy=UNA-pairwise, Feedback Type=Pairwise2024.0844.75
Mistral 7BBase Model=Mistral 7B, Alignment Strategy=Baseline2024.0842.58
UNA-binary (BCE)Base Model=Gemma 4B, Alignment Strategy=UNA-binary (BCE), Feedback Type=Binary2024.0840.79
DPOBase Model=Gemma 4B, Alignment Strategy=DPO, Feedback Type=Pairwise2024.0840.37
UNA-pairwiseBase Model=Gemma 4B, Alignment Strategy=UNA-pairwise, Feedback Type=Pairwise2024.0840.37
Gemma 4BBase Model=Gemma 4B, Alignment Strategy=Baseline2024.0839.73
KTOBase Model=Gemma 4B, Alignment Strategy=KTO, Feedback Type=Binary2024.0839.71