List your company
ResearchFormerly WizWand

Correctness detection on TruthfulQA

0.97AUCLlama-3B
0.469760.599630.72950.85937Aug 11, 2025Sep 10, 2025Oct 10, 2025Nov 9, 2025Dec 9, 2025Jan 8, 2026Feb 8, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Llama-3Btuning=Instruction-tuned, Size=3B, Layer=L12/28, Depth=43%2026.020.97
Qwen2-7Btuning=Instruction-tuned, Size=7B, Layer=L20/28, Depth=75%2026.020.94
Llama-1Btuning=Instruction-tuned, Size=1B, Layer=L8/16, Depth=56%2026.020.93
Gemma-2Btuning=Instruction-tuned, Size=2B, Layer=L15/26, Depth=62%2026.020.93
Mistral-7Btuning=Instruction-tuned, Size=7B, Layer=L23/32, Depth=75%2026.020.92
Qwen2-1.5Btuning=Instruction-tuned, Size=1.5B, Layer=L16/28, Depth=61%2026.020.91
GPT-2-Medtuning=Base model, Size=355M, Layer=L23/24, Depth=100%2026.020.84
GPT-2-Largetuning=Base model, Size=774M, Layer=L35/36, Depth=100%2026.020.84
GPT-2tuning=Base model, Size=124M, Layer=L11/12, Depth=100%2026.020.8
Probe(EOS)Model=Qwen2.5-7B2025.080.794
Probe(AVG)Model=Qwen2.5-7B2025.080.761
Probe(EU)Model=Qwen2.5-7B2025.080.759
Prob(Exact)Model=Qwen2.5-7B2025.080.758
Probe(AVG)Model=Fanar1-9b2025.080.734
Probe(AVG)Model=Gemma3-12B2025.080.733
Prob(Exact)Model=Gemma3-12B2025.080.728
Probe(EOS)Model=Gemma3-12B2025.080.728
Prob(Exact)Model=Fanar1-9b2025.080.711
Probe(EU)Model=Fanar1-9b2025.080.709
Probe(EOS)Model=Fanar1-9b2025.080.706
Probe(EU)Model=Gemma3-12B2025.080.687
LogTokUModel=Qwen2.5-7B2025.080.642
P(true)Model=Gemma3-12B2025.080.598
LogProbModel=Fanar1-9b2025.080.597
LogProbModel=Qwen2.5-7B2025.080.591
LogProbModel=Gemma3-12B2025.080.545
LogTokUModel=Fanar1-9b2025.080.541
P(true)Model=Qwen2.5-7B2025.080.537
P(true)Model=Fanar1-9b2025.080.53
LogTokUModel=Gemma3-12B2025.080.489