Coding Reasoning on MBPP
80Knowledge RecallCorDA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CorDABackbone=Mistral-7B, # Facts=1002026.05 | 80 | 49.49 | |
| CorDABackbone=Mistral-7B, # Facts=10002026.05 | 77.9 | 42.42 | |
| CorDABackbone=Mistral-7B, # Facts=2502026.05 | 74.4 | 45.45 | |
| CorDABackbone=Mistral-7B, # Facts=5002026.05 | 72.6 | 42.42 | |
| PALoRABackbone=Mistral-7B, # Facts=1002026.05 | 67 | 47.47 | |
| LoRABackbone=Mistral-7B, # Facts=2502026.05 | 66.8 | 48.48 | |
| PALoRABackbone=Mistral-7B, # Facts=2502026.05 | 64 | 48.48 | |
| LoRABackbone=Mistral-7B, # Facts=10002026.05 | 63.9 | 51.52 | |
| PALoRABackbone=Mistral-7B, # Facts=5002026.05 | 63.6 | 46.46 | |
| LoRABackbone=Mistral-7B, # Facts=1002026.05 | 63 | 48.48 | |
| LoRABackbone=Mistral-7B, # Facts=5002026.05 | 63 | 44.44 | |
| PALoRABackbone=Mistral-7B, # Facts=10002026.05 | 62.2 | 49.49 | |
| OPLoRABackbone=Mistral-7B, # Facts=2502026.05 | 58.4 | 47.47 | |
| OPLoRABackbone=Mistral-7B, # Facts=1002026.05 | 58 | 48.48 | |
| OPLoRABackbone=Mistral-7B, # Facts=10002026.05 | 52.7 | 47.47 | |
| OPLoRABackbone=Mistral-7B, # Facts=5002026.05 | 52 | 44.44 |