Reading Comprehension on DROP (EM)
41.7EMDeepSeek Chat 7B
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek Chat 7B# Shot=1-shot, Total Params=6.9B, Activated Params=6.9B, FLOPs per 4K Tokens=183.5T2024.01 | 41.7 | |
| LLaMA2 SFT 7B# Shot=1-shot, Total Params=6.7B, Activated Params=6.7B, FLOPs per 4K Tokens=187.9T2024.01 | 40 | |
| DeepSeek 7B (Dense)# Shot=1-shot, # Total Params=6.9B, # Activated Params=6.9B, FLOPs per 4K Tokens=183.5T, # Training Tokens=2T2024.01 | 34.9 | |
| LLaMA2 7B# Shot=1-shot, # Total Params=6.7B, # Activated Params=6.7B, FLOPs per 4K Tokens=187.9T, # Training Tokens=2T2024.01 | 34 | |
| DeepSeekMoE Chat 16B# Shot=1-shot, Total Params=16.4B, Activated Params=2.8B, FLOPs per 4K Tokens=74.4T2024.01 | 33.8 | |
| DeepSeekMoE 16B# Shot=1-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.01 | 32.9 | |
| DeepSeekMoE 16B# Shot=1-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.01 | 32.9 | |
| DeepSeekMoE 142B (Half Activated)# Shot=1-shot2024.01 | 28.9 | |
| DeepSeekMoE 145B# Shot=1-shot2024.01 | 27.8 | |
| DeepSeek 67B (Dense)# Shot=1-shot2024.01 | 27.5 | |
| GShard 137B# Shot=1-shot2024.01 | 21.6 |