Information Retrieval on NQ (Accuracy)
53.47AccuracyLM-Cocktail
Evaluation results
| Method | Links | |
|---|---|---|
| LM-Cocktail2023.11 | 53.47 | |
| LM-CocktailVariant=unified2023.11 | 53.4 | |
| Multitask-learning2023.11 | 53.25 | |
| BGE2023.11 | 51.05 | |
| LM-CocktailStrategy=blackbox2023.11 | 50.95 | |
| LM-Cocktailn=52023.11 | 50.75 | |
| Hybrid Mamba-2 + M2RNN-1Parameter scale=410M2026.03 | 23.8 | |
| Hybrid Mamba-2 + M2RNN-3Parameter scale=410M2026.03 | 23.5 | |
| Hybrid Gated DeltaNet + M2RNN-1Parameter scale=410M2026.03 | 23.5 | |
| Hybrid M2RNNParameter scale=410M2026.03 | 22.7 | |
| Hybrid Gated DeltaNetParameter scale=410M2026.03 | 22.2 | |
| Hybrid Mamba-2Parameter scale=410M2026.03 | 21.4 | |
| Hybrid Gated DeltaNet + M2RNN-3Parameter scale=410M2026.03 | 21.4 | |
| Transformer++Parameter scale=410M2026.03 | 15.5 | |
| Gated DeltaNetParameter scale=410M2026.03 | 15.4 | |
| Mamba-2Parameter scale=410M2026.03 | 15 | |
| M2RNNParameter scale=410M2026.03 | 13 | |
| GRUParameter scale=410M2026.03 | 8.8 | |
| RNNParameter scale=410M2026.03 | 5.9 |