Scientific Reasoning on AI2-ARC Scientific
79RecallCorDA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CorDABackbone=Mistral-7B, # Facts=1002026.05 | 79 | 80.3 | |
| CorDA# Facts=2502026.05 | 71.2 | 81.63 | |
| CorDABackbone=Mistral-7B, # Facts=2502026.05 | 71.2 | 80 | |
| CorDABackbone=Mistral-7B, # Facts=10002026.05 | 70.5 | 78.96 | |
| CorDABackbone=Mistral-7B, # Facts=5002026.05 | 70.4 | 77.53 | |
| CorDA# Facts=10002026.05 | 68.2 | 50.96 | |
| LoRABackbone=Mistral-7B, # Facts=2502026.05 | 66.8 | 79.08 | |
| OPLoRA# Facts=5002026.05 | 66.2 | 88.05 | |
| OPLoRA# Facts=1002026.05 | 64 | 88.13 | |
| OPLoRA# Facts=2502026.05 | 64 | 87.54 | |
| LoRABackbone=Mistral-7B, # Facts=10002026.05 | 63.9 | 77.82 | |
| LoRA# Facts=2502026.05 | 63.2 | 81.14 | |
| CorDA# Facts=1002026.05 | 63 | 83.29 | |
| LoRABackbone=Mistral-7B, # Facts=1002026.05 | 63 | 80.09 | |
| LoRABackbone=Mistral-7B, # Facts=5002026.05 | 63 | 77.9 | |
| OPLoRA# Facts=10002026.05 | 62.3 | 88.8 | |
| LoRA# Facts=5002026.05 | 62.2 | 87.37 | |
| LoRA# Facts=10002026.05 | 61.7 | 87.67 | |
| LoRA# Facts=1002026.05 | 61 | 86.03 | |
| PALoRABackbone=Mistral-7B, # Facts=5002026.05 | 60.4 | 84.47 | |
| PALoRABackbone=Mistral-7B, # Facts=10002026.05 | 60.3 | 84.01 | |
| PALoRA# Facts=5002026.05 | 60.2 | 89.1 | |
| PALoRA# Facts=1002026.05 | 60 | 87.96 | |
| PALoRABackbone=Mistral-7B, # Facts=1002026.05 | 60 | 84.89 | |
| PALoRA# Facts=10002026.05 | 59.3 | 88.43 | |
| PALoRA# Facts=2502026.05 | 58.4 | 88.3 | |
| OPLoRABackbone=Mistral-7B, # Facts=2502026.05 | 58.4 | 79.46 | |
| OPLoRABackbone=Mistral-7B, # Facts=1002026.05 | 58 | 81.52 | |
| PALoRABackbone=Mistral-7B, # Facts=2502026.05 | 56.8 | 85.19 | |
| OPLoRABackbone=Mistral-7B, # Facts=10002026.05 | 52.7 | 79.92 | |
| OPLoRABackbone=Mistral-7B, # Facts=5002026.05 | 52 | 78.54 | |
| CorDA# Facts=5002026.05 | 51.8 | 60.31 |