Question Answering on SQuAD (Latency, Peak HBM Usage, F1)
114.73Latency (s)EPIC
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EPICModel=Mistral-7B-Instruct-v0.32025.12 | 114.73 | 54.13 | 72 | |
| MEPICModel=Mistral-7B-Instruct-v0.32025.12 | 116.03 | 27.67 | 74 | |
| CacheBlendModel=Mistral-7B-Instruct-v0.32025.12 | 119.41 | 54.47 | 73 |