Logical Reasoning on ProofWriter (held-out)
0.5483PerformanceENVISIONS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ENVISIONSBackbone=LLaMA2-Chat (13B), Approach=Env-guided Self-Training2024.06 | 0.5483 | 0.6356 | |
| ENVISIONSBackbone=LLaMA2-Chat (7B), Approach=Env-guided Self-Training2024.06 | 0.5283 | 0.5869 | |
| STaR + Env.Backbone=LLaMA2-Chat (13B), Approach=Env-guided Self-Training2024.06 | 0.5233 | 0.606 | |
| Iterative SFT+DPOBackbone=LLaMA2-Chat (13B), Approach=Reinforced Self-Training2024.06 | 0.51 | 0.5977 | |
| STaR + Env.Backbone=LLaMA2-Chat (7B), Approach=Env-guided Self-Training2024.06 | 0.5017 | 0.5591 | |
| Iterative SFT+DPOBackbone=LLaMA2-Chat (7B), Approach=Reinforced Self-Training2024.06 | 0.49 | 0.5473 | |
| Self-RewardingBackbone=LLaMA2-Chat (13B), Approach=Reinforced Self-Training2024.06 | 0.3733 | 0.525 | |
| Claude-2 + LLaMA2-ChatBackbone=LLaMA2-Chat (7B), Approach=Distill-then-Finetune2024.06 | 0.3617 | 0.5303 | |
| Claude-2 + LLaMA2-ChatBackbone=LLaMA2-Chat (13B), Approach=Distill-then-Finetune2024.06 | 0.36 | 0.5717 | |
| LLaMA2-Chat (few-shot)Backbone=LLaMA2-Chat (13B), Mode=few-shot2024.06 | 0.3583 | 0.3861 | |
| LLaMA2-Chat (few-shot)Backbone=LLaMA2-Chat (7B), Mode=few-shot2024.06 | 0.3483 | 0.2569 | |
| GPT-4 + LLaMA2-ChatBackbone=LLaMA2-Chat (7B), Approach=Distill-then-Finetune2024.06 | 0.3433 | 0.5246 | |
| GPT-4 + LLaMA2-ChatBackbone=LLaMA2-Chat (13B), Approach=Distill-then-Finetune2024.06 | 0.3417 | 0.5649 | |
| Self-RewardingBackbone=LLaMA2-Chat (7B), Approach=Reinforced Self-Training2024.06 | 0.3217 | 0.4422 |