Social Intelligence Reasoning on Cognitive Genome In-domain 1.0 (test)
36.56Item SelectionHumanLLM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| HumanLLMBackbone=Llama-3.1-8B-Instruct2026.01 | 36.56 | 15.33 | 16.33 | 14.67 | 26.06 | 21.79 | |
| HumanLLMBackbone=Qwen2.5-7B-Instruct, Training Variant=gen2026.01 | 35.73 | 38 | 18.33 | 15 | 28.54 | 27.12 | |
| HumanLLMBackbone=Qwen2.5-7B-Instruct2026.01 | 33.16 | 39.56 | 27.33 | 11 | 25.84 | 27.38 | |
| HumanLLMBackbone=Qwen3-8B2026.01 | 32.2 | 31.78 | 22 | 12.67 | 13.36 | 22.4 | |
| HumanLLMBackbone=Qwen3-8B, Training Variant=t22026.01 | 31.97 | 4.58 | 8.36 | 6.58 | 13.58 | 13.01 | |
| HumanLLMBackbone=Phi-3-mini-128k-instruct2026.01 | 30.86 | 11.11 | 15.33 | 12.33 | 11.14 | 16.15 | |
| HumanLLMBackbone=Qwen2.5-3B-Instruct2026.01 | 28.86 | 16.89 | 12.67 | 6.67 | 8.02 | 14.62 | |
| GPT-4o2026.01 | 25.54 | 5.75 | 7.49 | 5.83 | 7.28 | 10.38 | |
| HumanLLMBackbone=Qwen2.5-7B-Instruct, Training Variant=raw2026.01 | 19.44 | 8.67 | 10.67 | 1.33 | 8.02 | 9.63 | |
| Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct2026.01 | 16.86 | 3.33 | 6.67 | 1.67 | 4.01 | 6.51 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct2026.01 | 15.23 | 4.22 | 9.33 | 1.67 | 3.12 | 6.71 | |
| HumanLLMBackbone=Qwen3-8B, Training Variant=t12026.01 | 15.01 | 23.25 | 14.9 | 11.75 | 10.77 | 15.13 | |
| Qwen3-8BBackbone=Qwen3-8B2026.01 | 14.5 | 5.33 | 10.33 | 1.67 | 3.79 | 7.12 | |
| Qwen2.5-3B-InstructBackbone=Qwen2.5-3B-Instruct2026.01 | 12.56 | 1.78 | 3.67 | 2 | 1.78 | 4.36 | |
| BE.FM2026.01 | 11.57 | 11.83 | 14.2 | 9.5 | 10.39 | 11.49 | |
| Phi-3-mini-128k-instructBackbone=Phi-3-mini-128k-instruct2026.01 | 10.2 | 10.89 | 12.67 | 8 | 5.12 | 9.38 | |
| Centaur2026.01 | 4.36 | 3.58 | 5.4 | 0.17 | 1.14 | 2.93 |