Response correctness and completeness evaluation on Coding
85F1 ScoreHALT
Evaluation Results
| Method | Links | |
|---|---|---|
| HALTLLM=Gemma-2-9B, n_desired=n_capable2025.06 | 85 | |
| HALTLLM=Llama-3-70B, n_desired=n_capable2025.06 | 83 | |
| HALTLLM=Mistral-7B, n_desired=n_capable2025.06 | 82 | |
| HALTLLM=Llama-3-8B, n_desired=n_capable2025.06 | 77 | |
| RandomTrimLLM=Llama-3-70B, n_desired=n_capable2025.06 | 75 | |
| UnchangedLLM=Llama-3-70B, n_desired=n_capable2025.06 | 71 | |
| RandomTrimLLM=Gemma-2-9B, n_desired=n_capable2025.06 | 65 | |
| UnchangedLLM=Gemma-2-9B, n_desired=n_capable2025.06 | 62 | |
| RandomTrimLLM=Llama-3-8B, n_desired=n_capable2025.06 | 61 | |
| UnchangedLLM=Llama-3-8B, n_desired=n_capable2025.06 | 58 | |
| UnchangedLLM=Mistral-7B, n_desired=n_capable2025.06 | 57 | |
| RandomTrimLLM=Mistral-7B, n_desired=n_capable2025.06 | 52 | |
| Abstain (IDK)LLM=Llama-3-70B, n_desired=n_capable2025.06 | 48 | |
| UnchangedLLM=Llama-3-70B, n_desired=n_all2025.06 | 44 | |
| HALTLLM=Gemma-2-9B, n_desired=n_all2025.06 | 44 | |
| HALTLLM=Llama-3-70B, n_desired=n_all2025.06 | 41 | |
| RandomTrimLLM=Llama-3-70B, n_desired=n_all2025.06 | 38 | |
| HALTLLM=Llama-3-8B, n_desired=n_all2025.06 | 35 | |
| HALTLLM=Mistral-7B, n_desired=n_all2025.06 | 34 | |
| UnchangedLLM=Gemma-2-9B, n_desired=n_all2025.06 | 34 | |
| RandomTrimLLM=Gemma-2-9B, n_desired=n_all2025.06 | 34 | |
| UnchangedLLM=Llama-3-8B, n_desired=n_all2025.06 | 33 | |
| UnchangedLLM=Mistral-7B, n_desired=n_all2025.06 | 33 | |
| RandomTrimLLM=Llama-3-8B, n_desired=n_all2025.06 | 30 | |
| RandomTrimLLM=Mistral-7B, n_desired=n_all2025.06 | 25 | |
| Abstain (IDK)LLM=Llama-3-70B, n_desired=n_all2025.06 | 19 | |
| Abstain (IDK)LLM=Llama-3-8B, n_desired=n_capable2025.06 | 18 | |
| Abstain (IDK)LLM=Gemma-2-9B, n_desired=n_capable2025.06 | 8 | |
| Abstain (IDK)LLM=Llama-3-8B, n_desired=n_all2025.06 | 5 | |
| Abstain (IDK)LLM=Mistral-7B, n_desired=n_capable2025.06 | 4 | |
| Abstain (IDK)LLM=Gemma-2-9B, n_desired=n_all2025.06 | 2 | |
| Abstain (IDK)LLM=Mistral-7B, n_desired=n_all2025.06 | 1 |