Character Probing on ARCANE In-world The Underdogs, East Lynne (low-popularity unvalidated)
72.5APFARCANE 32B Arc
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ARCANE 32B ArcFamily=ARCANE, Size=32B, Mode=Arc2026.06 | 72.5 | 70.1 | 66.8 | 66.8 | |
| DeepSeek-V4 Pro ArcFamily=DeepSeek-V4, Size=Pro, Mode=Arc2026.06 | 64.3 | 64.1 | 58.8 | 57.7 | |
| DeepSeek-V4 Flash ArcFamily=DeepSeek-V4, Size=Flash, Mode=Arc2026.06 | 62.8 | 62.4 | 54.9 | 57 | |
| ARCANE 8B ArcFamily=ARCANE, Size=8B, Mode=Arc2026.06 | 60.8 | 58.8 | 52.1 | 55.4 | |
| DeepSeek-V4 Flash VanillaFamily=DeepSeek-V4, Size=Flash, Mode=Vanilla2026.06 | 55.6 | 56.6 | 47.9 | 47.1 | |
| ARCANE 32B VanillaFamily=ARCANE, Size=32B, Mode=Vanilla2026.06 | 55.3 | 55.5 | 47 | 41.9 | |
| Qwen3 32B ArcFamily=Qwen3, Size=32B, Mode=Arc2026.06 | 55.1 | 54.7 | 45.6 | 44.2 | |
| DeepSeek-V4 Pro VanillaFamily=DeepSeek-V4, Size=Pro, Mode=Vanilla2026.06 | 53 | 54.2 | 45.6 | 45.3 | |
| ARCANE 8B VanillaFamily=ARCANE, Size=8B, Mode=Vanilla2026.06 | 46.1 | 46.2 | 37.2 | 34.6 | |
| Qwen3 8B ArcFamily=Qwen3, Size=8B, Mode=Arc2026.06 | 44.6 | 45.2 | 33.9 | 37.6 | |
| Qwen3 32B VanillaFamily=Qwen3, Size=32B, Mode=Vanilla2026.06 | 40.8 | 41.8 | 31.8 | 30.2 | |
| Qwen3 8B VanillaFamily=Qwen3, Size=8B, Mode=Vanilla2026.06 | 35.6 | 36.8 | 26.7 | 25.9 |