Content Moderation on Multi-category Chinese Content Moderation Dataset 1.0 (test)
89.7Politics AccuracyCARO (Llama3-8B)
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CARO (Llama3-8B)Category=Our Model, Base Model=Llama3-8B2026.04 | 89.7 | 94.5 | 97.4 | 81.4 | 97.2 | 72.3 | 88.8 | |
| CARO (Qwen2.5-7B)Category=Our Model, Base Model=Qwen2.5-7B2026.04 | 89.5 | 93.5 | 97.8 | 81.2 | 98.4 | 74.6 | 89.2 | |
| Naive SFTCategory=Post-Training2026.04 | 82.8 | 94.4 | 93.2 | 70.7 | 98.6 | 63.7 | 84.2 | |
| DeepSeek V3Category=General LLMs2026.04 | 79 | 90.3 | 89.8 | 70.5 | 95 | 62.5 | 80.3 | |
| CoT SFT+RLCategory=Post-Training2026.04 | 78.3 | 90 | 91 | 72.3 | 95.4 | 66.3 | 81.6 | |
| CoT SFTCategory=Post-Training2026.04 | 77.3 | 90.6 | 89.2 | 65.9 | 96.7 | 65.4 | 80 | |
| QwQ-32BCategory=General LLMs2026.04 | 75.4 | 69.6 | 72 | 60.7 | 84.9 | 54.6 | 69.1 | |
| Class-RAGCategory=Post-Training2026.04 | 74.9 | 83.7 | 91 | 68 | 90 | 56 | 75.5 | |
| DeepSeek R1Category=General LLMs2026.04 | 72.7 | 91.4 | 86.1 | 64.6 | 94.3 | 59.7 | 77.1 | |
| Agentic ICLCategory=Post-Training2026.04 | 66.8 | 87 | 77.7 | 67.6 | 91.8 | 54.6 | 72.9 | |
| Qwen2.5-32B-InstructCategory=General LLMs2026.04 | 59.1 | 91.1 | 84.4 | 67.9 | 95.4 | 54.2 | 74.3 | |
| GPT-4Category=General LLMs2026.04 | 58.6 | 88.7 | 79.8 | 64.3 | 92.7 | 56.8 | 72.3 | |
| LLaMA3-8BCategory=General LLMs2026.04 | 58.5 | 55.9 | 81 | 65.2 | 90.6 | 44.2 | 67.5 | |
| Qwen2.5-7B-InstructCategory=General LLMs2026.04 | 54.9 | 81.9 | 70 | 60.1 | 84.3 | 48.8 | 64.3 | |
| LLaMA-Guard-3-8BCategory=Specific LLMs2026.04 | 12 | 74.1 | 41.8 | 45.7 | 29.4 | 35.6 | 39.7 |