Language Model Inference on Sequence Bucket Ultra-long
77.97Latency (ms)COREY
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| COREYPrecision=FP162026.04 | 77.97 | 840.5 | 90.6 | |
| Static FusionPrecision=FP162026.04 | 87.22 | 751.4 | 102.4 | |
| No FusionPrecision=FP162026.04 | 104.54 | 626.9 | 123 |