Character Probing on ARCANE Out-of-world The Underdogs, East Lynne (low-popularity unvalidated)
73.5APFARCANE 32B Arc
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ARCANE 32B ArcFamily=ARCANE, Size=32B, Mode=Arc2026.06 | 73.5 | 70.7 | 68.1 | 70 | |
| DeepSeek-V4 Pro ArcFamily=DeepSeek-V4, Size=Pro, Mode=Arc2026.06 | 63.9 | 64 | 56.9 | 56.2 | |
| DeepSeek-V4 Flash ArcFamily=DeepSeek-V4, Size=Flash, Mode=Arc2026.06 | 62.4 | 62.3 | 54.9 | 53.6 | |
| ARCANE 8B ArcFamily=ARCANE, Size=8B, Mode=Arc2026.06 | 62 | 59.7 | 54.4 | 51.6 | |
| ARCANE 32B VanillaFamily=ARCANE, Size=32B, Mode=Vanilla2026.06 | 55.9 | 55.8 | 47.5 | 42.1 | |
| Qwen3 32B ArcFamily=Qwen3, Size=32B, Mode=Arc2026.06 | 53.1 | 52.6 | 43.3 | 42.3 | |
| DeepSeek-V4 Pro VanillaFamily=DeepSeek-V4, Size=Pro, Mode=Vanilla2026.06 | 52.5 | 53.5 | 44 | 43 | |
| DeepSeek-V4 Flash VanillaFamily=DeepSeek-V4, Size=Flash, Mode=Vanilla2026.06 | 52.4 | 53.4 | 43.7 | 47.8 | |
| Qwen3 8B ArcFamily=Qwen3, Size=8B, Mode=Arc2026.06 | 46.2 | 46.3 | 35.9 | 35.3 | |
| ARCANE 8B VanillaFamily=ARCANE, Size=8B, Mode=Vanilla2026.06 | 43.1 | 43 | 34 | 35 | |
| Qwen3 32B VanillaFamily=Qwen3, Size=32B, Mode=Vanilla2026.06 | 41.4 | 42.8 | 31.9 | 31.6 | |
| Qwen3 8B VanillaFamily=Qwen3, Size=8B, Mode=Vanilla2026.06 | 36 | 37.2 | 26.6 | 28.6 |