Multi-Hop Question Answering on Bamboogle (F1, LasJ)
61.6F1CARL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CARLModel=4B Reasoning, Max actions=322025.12 | 61.6 | 64 | |
| CARL-LiteModel=4B Reasoning, Max actions=102025.12 | 61.1 | 61.6 | |
| CARLModel=4B Reasoning, Max actions=102025.12 | 60.7 | 63.2 | |
| ARPOModel=4B Reasoning, Max actions=102025.12 | 59.6 | 61.6 | |
| GRPOModel=4B Reasoning, Max actions=322025.12 | 59.5 | 63.2 | |
| BranPOBackbone=Qwen3-4B-Instruct-25072026.02 | 57.8 | 68 | |
| TreeRLModel=4B Reasoning, Max actions=102025.12 | 57.7 | 58.4 | |
| TreeRPOModel=4B Reasoning, Max actions=102025.12 | 55.7 | 57.6 | |
| GRPOModel=4B Reasoning, Max actions=102025.12 | 55.1 | 58.4 | |
| GRPOBackbone=Qwen3-4B-Instruct-25072026.02 | 52.9 | 64.8 | |
| BranPOBackbone=Qwen2.5-7B-Instruct2026.02 | 52 | 62.4 | |
| Tree-GRPOBackbone=Qwen3-4B-Instruct-25072026.02 | 51.2 | 64 | |
| Tree-GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 51 | 62.4 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 50.1 | 61.6 | |
| GiGPOBackbone=Qwen3-4B-Instruct-25072026.02 | 49.7 | 61.9 | |
| SFTBackbone=Qwen3-4B-Instruct-25072026.02 | 49.1 | 60 | |
| CARLModel=7B Non-Reasoning, Max actions=322025.12 | 48.6 | 49.6 | |
| GiGPOBackbone=Qwen2.5-7B-Instruct2026.02 | 48.1 | 55.2 | |
| SFTBackbone=Qwen2.5-7B-Instruct2026.02 | 44.2 | 54.4 | |
| GRPOModel=7B Non-Reasoning, Max actions=322025.12 | 40.7 | 44.8 | |
| Zero-ShotModel=4B Reasoning, Max actions=102025.12 | 28.5 | 46.4 | |
| CARLModel=3B Non-Reasoning, Max actions=322025.12 | 25.7 | 22.4 | |
| GRPOModel=3B Non-Reasoning, Max actions=322025.12 | 24.5 | 20.8 | |
| Zero-ShotModel=3B Non-Reasoning, Max actions=322025.12 | 19 | 23.2 | |
| Zero-ShotModel=7B Non-Reasoning, Max actions=322025.12 | 16.5 | 36.8 |