Bias Evaluation on SOCT
0.054DR (Female Stereotype)Llama 3.1 8B - LFT w. SH-N (baseline 3)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama 3.1 8B - LFT w. SH-N (baseline 3)Model=Llama 3.1 8B, Variant=LFT w. SH-N (baseline 3)2025.12 | 0.054 | 0.269 | |
| Llama 3.1 8B - LFT w. SH (baseline 2)Model=Llama 3.1 8B, Variant=LFT w. SH (baseline 2)2025.12 | 0.0996 | 0.3036 | |
| Llama 3.2 1B - LFT w. SH-N (baseline 3)Model=Llama 3.2 1B, Variant=LFT w. SH-N (baseline 3)2025.12 | 0.1631 | 0.2776 | |
| Llama 3.1 8B - Pretrained model (baseline 1)Model=Llama 3.1 8B, Variant=Pretrained model (baseline 1)2025.12 | 0.2111 | 0.2904 | |
| Llama 3.2 1B - LFT w. SH (baseline 2)Model=Llama 3.2 1B, Variant=LFT w. SH (baseline 2)2025.12 | 0.2207 | 0.3098 | |
| Llama 3.2 1B - Pretrained model (baseline 1)Model=Llama 3.2 1B, Variant=Pretrained model (baseline 1)2025.12 | 0.2272 | 0.2522 | |
| Llama 3.1 8B - LFT w. SH-Dgender (GC-CDA)Model=Llama 3.1 8B, Variant=LFT w. SH-Dgender (GC-CDA)2025.12 | 0.2446 | 0.0264 | |
| Qwen 3 0.6B - Pretrained model (baseline 1)Model=Qwen 3 0.6B, Variant=Pretrained model (baseline 1)2025.12 | 0.2798 | 0.1426 | |
| Qwen 3 0.6B - LFT w. SH (baseline 2)Model=Qwen 3 0.6B, Variant=LFT w. SH (baseline 2)2025.12 | 0.3452 | 0.2059 | |
| Llama 3.1 8B - LFT w. SH-Dgender (BaseCDA)Model=Llama 3.1 8B, Variant=LFT w. SH-Dgender (BaseCDA)2025.12 | 0.3488 | 0.1384 | |
| Qwen 3 0.6B - LFT w. SH-N (baseline 3)Model=Qwen 3 0.6B, Variant=LFT w. SH-N (baseline 3)2025.12 | 0.3592 | 0.0434 | |
| Llama 3.2 1B - LFT w. SH-Dgender (GC-CDA)Model=Llama 3.2 1B, Variant=LFT w. SH-Dgender (GC-CDA)2025.12 | 0.3701 | 0.008 | |
| Llama 3.2 1B - LFT w. SH-Dgender (BaseCDA)Model=Llama 3.2 1B, Variant=LFT w. SH-Dgender (BaseCDA)2025.12 | 0.379 | 0.0879 | |
| Qwen 3 0.6B - LFT w. SH-Dgender (GC-CDA)Model=Qwen 3 0.6B, Variant=LFT w. SH-Dgender (GC-CDA)2025.12 | 0.4493 | 0.1461 | |
| Qwen 3 0.6B - LFT w. SH-Dgender(BaseCDA)Model=Qwen 3 0.6B, Variant=LFT w. SH-Dgender(BaseCDA)2025.12 | 0.4845 | 0.345 |