Operations Research Problem Solving on StructuredOR (test)
30Correct Examples CountToT-Fully-Traverse
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ToT-Fully-TraverseModel=GPT-4o2024.11 | 30 | — | — | |
| ToT-RethinkModel=GPT-4o2024.11 | 23 | — | — | |
| ToT-Fully-TraverseModel=Llama-3-70B2024.11 | 22 | — | — | |
| SCModel=GPT-4o2024.11 | 21 | — | — | |
| ToT-Randomly-ChosenModel=GPT-4o2024.11 | 21 | — | — | |
| ToT-Fully-TraverseModel=GPT-4o-mini2024.11 | 21 | — | — | |
| SCModel=Llama-3.1-70B2024.11 | 21 | — | — | |
| CoT-BMLDModel=GPT-4o2024.11 | 19 | — | — | |
| CoT-SPVOCModel=GPT-4o2024.11 | 19 | — | — | |
| SCModel=GPT-4o-mini2024.11 | 19 | — | — | |
| ToT-RethinkModel=GPT-4o-mini2024.11 | 19 | — | — | |
| SCModel=Llama-3-70B2024.11 | 19 | — | — | |
| CoT-BMLDModel=Llama-3.1-70B2024.11 | 18 | — | — | |
| CoT-SPVOCModel=Llama-3-70B2024.11 | 17 | — | — | |
| ToT-Randomly-ChosenModel=Llama-3-70B2024.11 | 17 | — | — | |
| ToT-RethinkModel=Llama-3-70B2024.11 | 17 | — | — | |
| ToT-Fully-TraverseModel=Llama-3.1-70B2024.11 | 17 | — | — | |
| CoT-SPVOCModel=GPT-4o-mini2024.11 | 14 | — | — | |
| ToT-RethinkModel=Llama-3.1-70B2024.11 | 14 | — | — | |
| CoT-BMLDModel=GPT-4o-mini2024.11 | 13 | — | — | |
| ToT-Randomly-ChosenModel=GPT-4o-mini2024.11 | 12 | — | — | |
| CoT-BMLDModel=Llama-3-70B2024.11 | 10 | — | — | |
| CoT-SPVOCModel=Llama-3.1-70B2024.11 | 10 | — | — | |
| SCModel=Qwen-2.5-72B-Instruct2024.11 | 9 | — | — | |
| ToT-Fully-TraverseModel=Llama-3.2-11B2024.11 | 8 | — | — | |
| CoT-BMLDModel=Qwen-2.5-72B-Instruct2024.11 | 6 | — | — | |
| ToT-Randomly-ChosenModel=Llama-3.1-70B2024.11 | 5 | — | — | |
| ToT-Fully-TraverseModel=Qwen-2-72B-Instruct2024.11 | 5 | — | — | |
| CoT-SPVOCModel=Qwen-2.5-72B-Instruct2024.11 | 5 | — | — | |
| SCModel=Qwen-2-72B-Instruct2024.11 | 4 | — | — | |
| ToT-Fully-TraverseModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 4 | — | — | |
| ToT-Randomly-ChosenModel=Qwen-2-72B-Instruct2024.11 | 3 | — | — | |
| CoT-BMLDModel=Llama-3.2-11B2024.11 | 2 | — | — | |
| CoT-BMLDModel=Qwen-2-72B-Instruct2024.11 | 2 | — | — | |
| CoT-SPVOCModel=Qwen-2-72B-Instruct2024.11 | 2 | — | — | |
| CoT-BMLDModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 2 | — | — | |
| CoT-SPVOCModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 2 | — | — | |
| ToT-Randomly-ChosenModel=Qwen-2.5-72B-Instruct2024.11 | 2 | — | — | |
| ToT-RethinkModel=Qwen-2.5-72B-Instruct2024.11 | 2 | — | — | |
| ToT-Fully-TraverseModel=Qwen-2.5-72B-Instruct2024.11 | 2 | — | — | |
| SCModel=Llama-3.2-11B2024.11 | 1 | — | — | |
| ToT-RethinkModel=Qwen-2-72B-Instruct2024.11 | 1 | — | — | |
| ToT-RethinkModel=Mixtral-8x7B-v0.12024.11 | 1 | — | — | |
| ToT-Fully-TraverseModel=Mixtral-8x7B-v0.12024.11 | 1 | — | — | |
| CoT-SPVOCModel=Llama-3.2-11B2024.11 | 0 | — | — | |
| ToT-Randomly-ChosenModel=Llama-3.2-11B2024.11 | 0 | — | — | |
| ToT-RethinkModel=Llama-3.2-11B2024.11 | 0 | — | — | |
| SCModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 0 | — | — | |
| ToT-Randomly-ChosenModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 0 | — | — | |
| ToT-RethinkModel=Qwen-2.5-MATH-72B-Instruct2024.11 | 0 | — | — | |
| CoT-BMLDModel=Mixtral-8x7B-v0.12024.11 | 0 | — | — | |
| CoT-SPVOCModel=Mixtral-8x7B-v0.12024.11 | 0 | — | — | |
| SCModel=Mixtral-8x7B-v0.12024.11 | 0 | — | — | |
| ToT-Randomly-ChosenModel=Mixtral-8x7B-v0.12024.11 | 0 | — | — | |
| Beam Search VariantPolicy Model=GPT-4o2024.11 | — | 80 | 15 | |
| BPP-Search VariantPolicy Model=GPT-4o2024.11 | — | 93.3 | 15 | |
| CoTPolicy Model=GPT-4o2024.11 | — | 63.3 | 1 | |
| Greedy Search VariantPolicy Model=GPT-4o2024.11 | — | 83.3 | 9 | |
| SCPolicy Model=GPT-4o2024.11 | — | 70 | 4 | |
| ToT-Randomly-ChosenPolicy Model=GPT-4o2024.11 | — | 70 | 39 | |
| ToT-RethinkPolicy Model=GPT-4o2024.11 | — | 76.6 | 40 |