Streaming Inference on OHLCV (155–925 samples)
43.4Average Latency (ms)LayerScale
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LayerScale2026.05 | 43.4 | 53.3 | — | |
| SGLang2026.05 | 105.6 | 53.3 | 2.4 | |
| vLLM2026.05 | 139.1 | 53.3 | 3.2 | |
| TensorRT-LLM2026.05 | 228.8 | 53.3 | 5.3 | |
| llama.cppcaching=enabled2026.05 | 254 | 46.7 | 5.9 | |
| GPT-3.5-Turbo2026.05 | 926 | 75 | 21 | |
| GPT-4o-mini2026.05 | 1,124 | 55 | 26 | |
| Claude 3.5 Sonnet2026.05 | 2,628 | 70 | 61 | |
| Claude 3.5 Haiku2026.05 | 3,962 | 65 | 92 |