Theory-of-Mind Reasoning on ancient Egyptian
86Belief Inference Accuracy8B ↬ 405B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| 8B ↬ 405BSolution Type=Ours, Backbone=Llama3.1, Scaling Setting=weak-to-strong control2025.06 | 86 | 76.3 | 80.4 | |
| 4B-depth ↬ 70BSolution Type=Ours, Backbone=Llama3.1, Scaling Setting=weak-to-strong control2025.06 | 85.3 | 71.3 | 77.9 | |
| 70B-post-trainedSolution Type=Raw, Backbone=Llama3.1, Scaling Setting=direct post-training2025.06 | 84.6 | 66.3 | 75.3 | |
| 70B-zero-shotSolution Type=Raw, Backbone=Llama3.1, Scaling Setting=zero-shot2025.06 | 83.6 | 60.6 | 69.3 | |
| 8B ↬ 70BSolution Type=Ours, Backbone=Llama3.1, Scaling Setting=weak-to-strong control2025.06 | 83.6 | 76 | 77.7 | |
| 4B-wide ↬ 70BSolution Type=Ours, Backbone=Llama3.1, Scaling Setting=weak-to-strong control2025.06 | 83 | 75.3 | 79.1 |