Question Answering on HPC Q&A Run 1
0.724Cosine SimilarityPhi-3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Phi-3Size=14B, Latency (s)=7.30, Resp. Length=125.0, RAG top-k=52026.05 | 0.724 | 0.071 | 0.841 | |
| Mistral NemoSize=12B, Latency (s)=6.12, Resp. Length=123.5, RAG top-k=52026.05 | 0.715 | 0.065 | 0.841 | |
| Phi-2Size=2.7B, Latency (s)=5.71, Resp. Length=138.1, RAG top-k=52026.05 | 0.696 | 0.07 | 0.846 | |
| TinyLlamaSize=1.1B, Latency (s)=4.27, Resp. Length=94.3, RAG top-k=52026.05 | 0.632 | 0.066 | 0.829 | |
| HPC-LLMSize=8B, Latency (s)=5.22, Resp. Length=90.7, RAG top-k=52026.05 | 0.608 | 0.07 | 0.808 |