Coding on WildChat 5,000 conversations
0.26KL Divergence (Forward)Gemini 3.1 Pro
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 3.1 ProModel Category=Closed-Source2026.05 | 0.26 | 0.256 | 0.246 | |
| GPT-5.4Model Category=Closed-Source2026.05 | 0.261 | 0.265 | 0.248 | |
| humanlm-opinionModel Category=Trained Simulators2026.05 | 0.268 | 0.257 | 0.247 | |
| gemma-4-31B-itModel Category=Open-Source2026.05 | 0.311 | 0.296 | 0.297 | |
| UserLM-8bModel Category=Trained Simulators2026.05 | 0.328 | 0.388 | 0.349 | |
| GPT-5.4 nanoModel Category=Closed-Source2026.05 | 0.353 | 0.364 | 0.353 | |
| Gemini 3 FlashModel Category=Closed-Source2026.05 | 0.355 | 0.328 | 0.334 | |
| gpt-oss-120bModel Category=Open-Source2026.05 | 0.358 | 0.353 | 0.343 | |
| GPT-5.4 miniModel Category=Closed-Source2026.05 | 0.363 | 0.388 | 0.364 | |
| Qwen3.5-122B-A10BModel Category=Open-Source2026.05 | 0.394 | 0.378 | 0.387 | |
| gemma-4-26B-A4B-itModel Category=Open-Source2026.05 | 0.397 | 0.373 | 0.379 | |
| gpt-oss-20bModel Category=Open-Source2026.05 | 0.411 | 0.435 | 0.409 | |
| gemma-4-E4B-itModel Category=Open-Source2026.05 | 0.422 | 0.383 | 0.402 | |
| Gemini 3.1 Flash-LiteModel Category=Closed-Source2026.05 | 0.432 | 0.406 | 0.421 | |
| Llama-3.1-8B-InstructModel Category=Open-Source2026.05 | 0.435 | 0.427 | 0.423 | |
| Qwen3.5-27BModel Category=Open-Source2026.05 | 0.437 | 0.421 | 0.433 | |
| Claude Haiku 4.5Model Category=Closed-Source2026.05 | 0.438 | 0.43 | 0.427 | |
| gemma-4-E2B-itModel Category=Open-Source2026.05 | 0.446 | 0.435 | 0.435 | |
| Qwen3.5-9BModel Category=Open-Source2026.05 | 0.465 | 0.457 | 0.454 | |
| Llama-3.3-70B-InstructModel Category=Open-Source2026.05 | 0.471 | 0.456 | 0.469 | |
| Qwen3.5-35B-A3BModel Category=Open-Source2026.05 | 0.48 | 0.467 | 0.478 | |
| Qwen3.5-4BModel Category=Open-Source2026.05 | 0.482 | 0.471 | 0.472 | |
| Qwen3.5 .8BModel Category=Open-Source2026.05 | 0.547 | 0.568 | 0.579 | |
| Qwen3.5-2BModel Category=Open-Source2026.05 | 0.584 | 0.591 | 0.608 |