Planning on Blocksworld (accuracy)
97Blocksworld AccuracyPT-SFT
Evaluation results
| Method | Links | |
|---|---|---|
| PT-SFTBase model=Mistral, Transfer cost=full retrain2026.05 | 97 | |
| PT-SFTBase model=Qwen2.5, Transfer cost=full retrain2026.05 | 96 | |
| PT-SFTBase model=GPT-OSS, Transfer cost=full retrain2026.05 | 93 | |
| SoftCoTBase model=Qwen2.5, Transfer cost=full retrain2026.05 | 82 | |
| SoftCoTBase model=GPT-OSS, Transfer cost=full retrain2026.05 | 82 | |
| OVMBase model=Qwen2.5, Transfer cost=full retrain2026.05 | 81.4 | |
| HyperGuideBase model=GPT-OSS, Transfer cost=small MLP2026.05 | 80 | |
| HyperGuideBase model=Qwen2.5, Transfer cost=small MLP2026.05 | 79 | |
| OVMBase model=GPT-OSS, Transfer cost=full retrain2026.05 | 77.4 | |
| CARLBase Model=Qwen3-8B, Strategy=RL Training2026.07 | 77.2 | |
| SoftCoTBase model=Mistral, Transfer cost=full retrain2026.05 | 71 | |
| HyperGuideBase model=Mistral, Transfer cost=small MLP2026.05 | 63 | |
| OVMBase model=Mistral, Transfer cost=full retrain2026.05 | 62 | |
| Self-ConsistencyBase model=Qwen2.5, Transfer cost=N/A2026.05 | 60 | |
| Tree of ThoughtsBase model=Qwen2.5, Transfer cost=N/A2026.05 | 58 | |
| Few-shotBase model=Mistral, Transfer cost=N/A2026.05 | 57 | |
| Tree of ThoughtsBase model=Mistral, Transfer cost=N/A2026.05 | 55 | |
| Self-ConsistencyBase model=Mistral, Transfer cost=N/A2026.05 | 53 | |
| Oracle FeedbackBase Model=GPT-4o, Strategy=Prompting2026.07 | 52 | |
| ReferenceBase Model=GPT-4o, Strategy=Prompting2026.07 | 51.4 | |
| Behavioral CloningBase Model=GPT-4o, Strategy=Prompting2026.07 | 51.2 | |
| DirectBase Model=GPT-4o, Strategy=Prompting2026.07 | 42.4 | |
| Few-shotBase model=Qwen2.5, Transfer cost=N/A2026.05 | 41 | |
| Tree of ThoughtsBase model=GPT-OSS, Transfer cost=N/A2026.05 | 37 | |
| Few-shotBase model=GPT-OSS, Transfer cost=N/A2026.05 | 9 | |
| Self-ConsistencyBase model=GPT-OSS, Transfer cost=N/A2026.05 | 2 |