Hate Speech Detection on POPQUORN
76.8Accuracygated demographic residual model
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| gated demographic residual modelBackbone=BERTweet, Input=Ours, Framework=Finetuned PLMs2026.05 | 76.8 | 67.51 | 59.28 | 60.19 | 75.71 | 4.16 | |
| gated demographic residual modelBackbone=ToxDect-RoBERTa, Input=Ours, Framework=Finetuned PLMs2026.05 | 76.12 | 65.96 | 59.2 | 59.98 | 73.73 | 2.27 | |
| BERTweetBackbone=BERTweet, Input=+Demo, Framework=Finetuned PLMs2026.05 | 75.89 | 65.92 | 58.48 | 58.81 | 73.94 | 2.39 | |
| TID-DeBERTa-v3-baseBackbone=TID-DeBERTa-v3-base, Input=+Demo, Framework=Annotator modeling2026.05 | 75.78 | 64.87 | 56.42 | 56.33 | 72.3 | 0.81 | |
| TID-DeBERTa-v3-baseBackbone=TID-DeBERTa-v3-base, Input=Text, Framework=Annotator modeling2026.05 | 75.51 | 64.37 | 56.19 | 55.67 | 71.49 | — | |
| Mistral-7B-v0.3Backbone=Mistral-7B-v0.3, Input=+Demo, Framework=Zero-shot2026.05 | 75.42 | 47.78 | 49.97 | 43.2 | 46.61 | -8.08 | |
| LLaMA-3.2-1B-LoRABackbone=LLaMA-3.2-1B-LoRA, Input=+Demo, Framework=Finetuned LLMs2026.05 | 75.18 | 63.63 | 57.88 | 58.47 | 68.61 | 2.43 | |
| Qwen2.5-0.5B-LoRABackbone=Qwen2.5-0.5B-LoRA, Input=+Demo, Framework=Finetuned LLMs2026.05 | 74.78 | 63.25 | 58.08 | 58.74 | 68.97 | 7.53 | |
| LLaMA-3.2-1B-SPTBackbone=LLaMA-3.2-1B-SPT, Input=Text+Demo, Framework=Finetuned LLMs2026.05 | 74.75 | 51.76 | 52.47 | 48.21 | 64.72 | — | |
| BERTweetBackbone=BERTweet, Input=Text, Framework=Finetuned PLMs2026.05 | 74.6 | 61.49 | 55.15 | 54.1 | 71.55 | — | |
| ToxDect-RoBERTaBackbone=ToxDect-RoBERTa, Input=+Demo, Framework=Finetuned PLMs2026.05 | 74.38 | 62.68 | 57.78 | 58.02 | 71.46 | -0.22 | |
| ToxDect-RoBERTaBackbone=ToxDect-RoBERTa, Input=Text, Framework=Finetuned PLMs2026.05 | 74.32 | 62.21 | 56.53 | 56.49 | 71.68 | — | |
| LLaMA-3.2-1B-LoRABackbone=LLaMA-3.2-1B-LoRA, Input=Text, Framework=Finetuned LLMs2026.05 | 73.64 | 59.73 | 55.36 | 55.29 | 66.18 | — | |
| Qwen2.5-0.5B-LoRABackbone=Qwen2.5-0.5B-LoRA, Input=Text, Framework=Finetuned LLMs2026.05 | 71.14 | 56.9 | 55.54 | 55.81 | 61.44 | — | |
| Qwen2-7BBackbone=Qwen2-7B, Input=Text, Framework=Zero-shot2026.05 | 53.78 | 58.53 | 61.19 | 52.44 | 66.14 | — | |
| Qwen2-7BBackbone=Qwen2-7B, Input=+Demo, Framework=Zero-shot2026.05 | 52.11 | 56.88 | 59.03 | 50.74 | 63.91 | -2.23 | |
| Mistral-7B-v0.3Backbone=Mistral-7B-v0.3, Input=Text, Framework=Zero-shot2026.05 | 45 | 53.12 | 53.83 | 44.41 | 54.69 | — | |
| LLaMA-2-7B-chat-hfBackbone=LLaMA-2-7B-chat-hf, Input=Text, Framework=Zero-shot2026.05 | 32.35 | 56.1 | 53.2 | 31.08 | 60.71 | — | |
| LLaMA-2-7B-chat-hfBackbone=LLaMA-2-7B-chat-hf, Input=+Demo, Framework=Zero-shot2026.05 | 27.43 | 51.62 | 50.43 | 24.65 | 53.81 | -6.9 |