Compositional Reasoning Dataset
43.3Correction Score (C)CREME
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CREMEBackbone=OpenAlpaca-3B2024.02 | 43.3 | 23.71 | 3.61 | 1.24 | |
| CREMEBackbone=LLAMA-2-7B2024.02 | 17 | 7.99 | 1.27 | 0.86 | |
| OpenAlpaca-3BModel Type=Base Model2024.02 | 7.2 | 7 | 13.5 | 0.6 | |
| LLAMA-2-7BModel Type=Base Model2024.02 | 3.2 | 2.3 | 13.1 | 0.3 | |
| Memory InjectionBackbone=LLAMA-2-7B2024.02 | 2.21 | 0.3 | 0.32 | 26.72 | |
| CoT-PatchScopesBackbone=LLAMA-2-7B2024.02 | 1.2 | — | — | — | |
| Memory InjectionBackbone=OpenAlpaca-3B2024.02 | 0.98 | 0.45 | 0.75 | 2.93 | |
| CoT-PatchScopesBackbone=OpenAlpaca-3B2024.02 | 0.91 | — | — | — |