Correctness detection on TruthfulQA
0.97AUCLlama-3B
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama-3Btuning=Instruction-tuned, Size=3B, Layer=L12/28, Depth=43%2026.02 | 0.97 | |
| Qwen2-7Btuning=Instruction-tuned, Size=7B, Layer=L20/28, Depth=75%2026.02 | 0.94 | |
| Llama-1Btuning=Instruction-tuned, Size=1B, Layer=L8/16, Depth=56%2026.02 | 0.93 | |
| Gemma-2Btuning=Instruction-tuned, Size=2B, Layer=L15/26, Depth=62%2026.02 | 0.93 | |
| Mistral-7Btuning=Instruction-tuned, Size=7B, Layer=L23/32, Depth=75%2026.02 | 0.92 | |
| Qwen2-1.5Btuning=Instruction-tuned, Size=1.5B, Layer=L16/28, Depth=61%2026.02 | 0.91 | |
| GPT-2-Medtuning=Base model, Size=355M, Layer=L23/24, Depth=100%2026.02 | 0.84 | |
| GPT-2-Largetuning=Base model, Size=774M, Layer=L35/36, Depth=100%2026.02 | 0.84 | |
| GPT-2tuning=Base model, Size=124M, Layer=L11/12, Depth=100%2026.02 | 0.8 | |
| Probe(EOS)Model=Qwen2.5-7B2025.08 | 0.794 | |
| Probe(AVG)Model=Qwen2.5-7B2025.08 | 0.761 | |
| Probe(EU)Model=Qwen2.5-7B2025.08 | 0.759 | |
| Prob(Exact)Model=Qwen2.5-7B2025.08 | 0.758 | |
| Probe(AVG)Model=Fanar1-9b2025.08 | 0.734 | |
| Probe(AVG)Model=Gemma3-12B2025.08 | 0.733 | |
| Prob(Exact)Model=Gemma3-12B2025.08 | 0.728 | |
| Probe(EOS)Model=Gemma3-12B2025.08 | 0.728 | |
| Prob(Exact)Model=Fanar1-9b2025.08 | 0.711 | |
| Probe(EU)Model=Fanar1-9b2025.08 | 0.709 | |
| Probe(EOS)Model=Fanar1-9b2025.08 | 0.706 | |
| Probe(EU)Model=Gemma3-12B2025.08 | 0.687 | |
| LogTokUModel=Qwen2.5-7B2025.08 | 0.642 | |
| P(true)Model=Gemma3-12B2025.08 | 0.598 | |
| LogProbModel=Fanar1-9b2025.08 | 0.597 | |
| LogProbModel=Qwen2.5-7B2025.08 | 0.591 | |
| LogProbModel=Gemma3-12B2025.08 | 0.545 | |
| LogTokUModel=Fanar1-9b2025.08 | 0.541 | |
| P(true)Model=Qwen2.5-7B2025.08 | 0.537 | |
| P(true)Model=Fanar1-9b2025.08 | 0.53 | |
| LogTokUModel=Gemma3-12B2025.08 | 0.489 |