Offensiveness Rating on Offensiveness Rating 20% workers' responses (test)
0.45MAE (Mean)CrowdLLM
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CrowdLLMBackbone=Gemma 3-12B2025.12 | 0.45 | 0.48 | 0.43 | 1 | 0.53 | 0.59 | 0.82 | 1.1 | 1.53 | 1.23 | 0.85 | 0.51 | |
| LLMBackbone=Gemma 3-12B, Mode=multi-persona2025.12 | 0.65 | 0.64 | 0.67 | 0.69 | 0.68 | 0.86 | 1 | 1.16 | 1.15 | 1.16 | 0.69 | 0.78 | |
| VAE2025.12 | 0.71 | 0.81 | 0.6 | 0.96 | 0.92 | 0.94 | 1.15 | 1.24 | 1.37 | 1.54 | 0.76 | 0.79 | |
| LLMBackbone=Gemma 3-12B, Mode=zero-shot2025.12 | 0.86 | 1.03 | 1.23 | 1.06 | 1.23 | 1.08 | 1.32 | 1.53 | 1.38 | 1.52 | 0.39 | 1.13 | |
| LLMBackbone=Gemma 3-12B, Mode=SC2025.12 | 0.99 | 1.26 | 1.48 | 1.22 | 1.49 | 1.16 | 1.48 | 1.68 | 1.52 | 1.69 | 0.34 | 1.24 | |
| Random2025.12 | 1.27 | 1.61 | 1.57 | 1.73 | 1.89 | 1.44 | 1.86 | 2.05 | 2.17 | 2.31 | 0.56 | 1.31 |