List your company
ResearchFormerly WizWand

Instruction Following on ComplexInstruct Level 6

64.2ISRDVRws
3.98419.61735.2550.883Oct 16, 2024
Updated 8mo ago

Evaluation results

MethodLinks
DVRwsBackbone=Llama-3.1-70B-Instruct-AWQ-INT4, warm-start=true2024.1064.2
DVRcsBackbone=Llama-3.1-70B-Instruct-AWQ-INT4, cold-start=true2024.1063.1
CRITICBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1061.3
ReActBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1052.1
R-SampleBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1050.7
DVRBackbone=Llama3.1-8B, Evaluation Protocol=zero-shot2024.1049.6
DVRBackbone=Llama-3-8B-Instruct, Start Type=Warm-Start2024.1046.9
DVRBackbone=Llama-3-8B-Instruct, Start Type=Cold-Start2024.1043.8
GPT-4-turboEvaluation Protocol=zero-shot2024.1042.6
CRITICBackbone=Llama-3-8B-Instruct2024.1041.5
U-SCBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1036.3
ReflexionBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1036.1
VanillaBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1035.9
BSMBackbone=Llama-3.1-70B-Instruct-AWQ-INT42024.1034.2
R-SampleBackbone=Llama-3-8B-Instruct2024.1031
ReActBackbone=Llama-3-8B-Instruct2024.1030.7
Llama3.1-8BEvaluation Protocol=zero-shot2024.1025.3
DVRBackbone=Mistral-7B, Evaluation Protocol=zero-shot2024.1023.4
ReflexionBackbone=Llama-3-8B-Instruct2024.1020.9
U-SCBackbone=Llama-3-8B-Instruct2024.1020.6
VanillaBackbone=Llama-3-8B-Instruct2024.1020.4
BSMBackbone=Llama-3-8B-Instruct2024.1019.5
Mistral-7BEvaluation Protocol=zero-shot2024.106.3