Transitivity Reasoning on Transitivity (test)
70.4Average AccuracyTransitivity Sem V4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Transitivity Sem V4Prediction Pattern=Context-dependent (17-6.5k), Collapse=No2026.05 | 70.4 | 26.8 | |
| Standard GemmaPrediction Pattern=Task-specific heuristics, Collapse=No2026.05 | 70.1 | 23.5 | |
| Transitivity V1Prediction Pattern=Always ”Yes” (10k/0), Collapse=Yes2026.05 | 27.7 | 31.9 |