List your company

Question Answering on ASQA, HotpotQA, NQ, TriviaQA, POPQA, and BIOASQ

83ASQA ScoreOSCAR-llama
68.4472.227679.78Mar 17, 2025

Evaluation results

MethodLinks
OSCAR-llamaSetting=standalone, Decoder=Mistral-7B2025.0383648091668077
RECOMPBackbone=Mistral-7B2025.0383617889588375
OSCAR-8-LayersSetting=standalone, Decoder=Mistral-7B2025.0382648191647977
OSCAR-10-LayersSetting=standalone, Decoder=Mistral-7B2025.0382648191648077
No compressionBackbone=Mistral-7B2025.0382628090618276
ProvenceBackbone=Mistral-7B2025.0382608089588576
OSCAR-llamaBackbone=Mistral-7B2025.0382648191658177
OSCAR-5-LayersBackbone=Mistral-7B2025.0382627990637776
OSCAR-8-LayersBackbone=Mistral-7B2025.0382648091647977
OSCAR-llamaBackbone=Qwen-7B2025.0382627990658176
No compressionBackbone=Mistral-24B2025.0382718092708580
OSCAR-llamaBackbone=Mistral-24B2025.0382658292678479
OSCAR-llamaSetting=e2e, Decoder=Mistral-7B2025.0381637991668077
OSCAR-8-LayersSetting=e2e, Decoder=Mistral-7B2025.0381637990647876
OSCAR-10-LayersSetting=e2e, Decoder=Mistral-7B2025.0381658291667877
PISCOBackbone=Mistral-7B2025.0381607889577974
OSCAR-8-LayersBackbone=Qwen-7B2025.0381617890648076
No compressionBackbone=Qwen-7B2025.0380677891658478
OSCAR-5-LayersBackbone=Llama-1B2025.0371537085557268
No compressionBackbone=Llama-1B2025.0369486681527665