List your company
ResearchFormerly WizWand

API Latency Measurement on Single 500 B / 1 KB message Round 1 1.0 (inference)

753Mean Latency (ms)Mistral 3B
569.521,808.013,046.54,284.99Dec 17, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Mistral 3BSecurity Protocol=Direct, Prompt Size=500 B request / 1 KB response, Round=12025.12753551
Mistral 3BSecurity Protocol=TEE Proxy, Prompt Size=500 B request / 1 KB response, Round=12025.12756521
Mistral 8BSecurity Protocol=Direct, Prompt Size=500 B request / 1 KB response, Round=12025.12901461
Mistral 8BSecurity Protocol=TEE Proxy, Prompt Size=500 B request / 1 KB response, Round=12025.121,0601231.18
Mistral 3BSecurity Protocol=TLS notary, Prompt Size=500 B request / 1 KB response, Round=12025.121,3301101.77
Mistral 8BSecurity Protocol=TLS notary, Prompt Size=500 B request / 1 KB response, Round=12025.121,5901351.77
Claude-Haiku-4.5Security Protocol=Direct, Prompt Size=500 B request / 1 KB response, Round=12025.121,8001031
Claude-Haiku-4.5Security Protocol=TEE Proxy, Prompt Size=500 B request / 1 KB response, Round=12025.121,9701771.09
OpenAI GPT-4oSecurity Protocol=Direct, Prompt Size=500 B request / 1 KB response, Round=12025.122,180961
OpenAI GPT-4oSecurity Protocol=TEE Proxy, Prompt Size=500 B request / 1 KB response, Round=12025.122,290891.05
Claude-Haiku-4.5Security Protocol=TLS notary, Prompt Size=500 B request / 1 KB response, Round=12025.122,4602041.37
OpenAI GPT-4oSecurity Protocol=TLS notary, Prompt Size=500 B request / 1 KB response, Round=12025.123,3004371.51
Claude-Sonnet-4.5Security Protocol=Direct, Prompt Size=500 B request / 1 KB response, Round=12025.124,5901661
Claude-Sonnet-4.5Security Protocol=TEE Proxy, Prompt Size=500 B request / 1 KB response, Round=12025.124,6802321.02
Claude-Sonnet-4.5Security Protocol=TLS notary, Prompt Size=500 B request / 1 KB response, Round=12025.125,3401551.16