List your company
ResearchFormerly WizWand

Response correctness and completeness evaluation on Coding

85F1 ScoreHALT
-2.3620.324365.68Jun 4, 2025
Updated 8mo ago

Evaluation results

MethodLinks
HALTLLM=Gemma-2-9B, n_desired=n_capable2025.0685
HALTLLM=Llama-3-70B, n_desired=n_capable2025.0683
HALTLLM=Mistral-7B, n_desired=n_capable2025.0682
HALTLLM=Llama-3-8B, n_desired=n_capable2025.0677
RandomTrimLLM=Llama-3-70B, n_desired=n_capable2025.0675
UnchangedLLM=Llama-3-70B, n_desired=n_capable2025.0671
RandomTrimLLM=Gemma-2-9B, n_desired=n_capable2025.0665
UnchangedLLM=Gemma-2-9B, n_desired=n_capable2025.0662
RandomTrimLLM=Llama-3-8B, n_desired=n_capable2025.0661
UnchangedLLM=Llama-3-8B, n_desired=n_capable2025.0658
UnchangedLLM=Mistral-7B, n_desired=n_capable2025.0657
RandomTrimLLM=Mistral-7B, n_desired=n_capable2025.0652
Abstain (IDK)LLM=Llama-3-70B, n_desired=n_capable2025.0648
UnchangedLLM=Llama-3-70B, n_desired=n_all2025.0644
HALTLLM=Gemma-2-9B, n_desired=n_all2025.0644
HALTLLM=Llama-3-70B, n_desired=n_all2025.0641
RandomTrimLLM=Llama-3-70B, n_desired=n_all2025.0638
HALTLLM=Llama-3-8B, n_desired=n_all2025.0635
HALTLLM=Mistral-7B, n_desired=n_all2025.0634
UnchangedLLM=Gemma-2-9B, n_desired=n_all2025.0634
RandomTrimLLM=Gemma-2-9B, n_desired=n_all2025.0634
UnchangedLLM=Llama-3-8B, n_desired=n_all2025.0633
UnchangedLLM=Mistral-7B, n_desired=n_all2025.0633
RandomTrimLLM=Llama-3-8B, n_desired=n_all2025.0630
RandomTrimLLM=Mistral-7B, n_desired=n_all2025.0625
Abstain (IDK)LLM=Llama-3-70B, n_desired=n_all2025.0619
Abstain (IDK)LLM=Llama-3-8B, n_desired=n_capable2025.0618
Abstain (IDK)LLM=Gemma-2-9B, n_desired=n_capable2025.068
Abstain (IDK)LLM=Llama-3-8B, n_desired=n_all2025.065
Abstain (IDK)LLM=Mistral-7B, n_desired=n_capable2025.064
Abstain (IDK)LLM=Gemma-2-9B, n_desired=n_all2025.062
Abstain (IDK)LLM=Mistral-7B, n_desired=n_all2025.061