General Language Understanding on Open LLM Leaderboard (test)
62.03ARCFalcon Mamba-7B
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Falcon Mamba-7BType=Instruct-tuned2024.08 | 62.03 | 80.82 | 62.11 | 73.64 | 53.42 | — | |
| Mamba2-Llama3Attention percentage=50%2024.08 | 60.41 | 77.97 | 56.67 | 71.35 | 66.6 | — | |
| Mamba2-Llama3Attention percentage=25%2024.08 | 59.22 | 76.88 | 53.94 | 64.88 | 64.64 | — | |
| Mamba-Llama3Attention percentage=50%2024.08 | 56.57 | 78.99 | 59.26 | 69.06 | 58.85 | — | |
| Mamba-Llama3Attention percentage=25%2024.08 | 55.03 | 75.66 | 52.68 | 62.83 | 55.03 | — | |
| Mamba2-Llama3Attention percentage=0%2024.08 | 53.51 | 70.31 | 44.21 | 58.91 | 52.31 | — | |
| Mamba2-Llama3Attention percentage=12.5%2024.08 | 53.33 | 72.16 | 50.85 | 63.61 | 61.12 | — | |
| Mamba-Llama3Attention percentage=12.5%2024.08 | 52.9 | 72.46 | 49.2 | 59.19 | 53 | — | |
| RecurrentGemma-9BType=Instruct-tuned2024.08 | 52 | 80.4 | 60.5 | 73.6 | 38.6 | — | |
| CygnetBase Model=Llama-3-8B-Instruct, Training Configuration=Circuit Breaker + RepE Fine-tune2024.06 | — | — | — | — | — | 71.9 | |
| Llama-3-8B-InstructBase Model=Llama-3-8B-Instruct, Training Configuration=Refusal Trained2024.06 | — | — | — | — | — | 68.8 | |
| Mistral-7B-Instruct-v2Base Model=Mistral-7B-Instruct-v2, Training Configuration=Refusal Trained2024.06 | — | — | — | — | — | 65.4 | |
| R2D2Base Model=Mistral-7B-Instruct-v2, Training Configuration=Adv Trained2024.06 | — | — | — | — | — | 61.2 | |
| Representation ReroutingBase Model=Mistral-7B-Instruct-v2, Training Configuration=+ RR2024.06 | — | — | — | — | — | 65.4 | |
| Representation ReroutingBase Model=Llama-3-8B-Instruct, Training Configuration=+ RR2024.06 | — | — | — | — | — | 68.3 |