List your company
ResearchFormerly WizWand

Question Answering on QASPER-E 2021 (Full)

60F1 ScoreSkyline
-2.413.83046.2Oct 2, 2025
Updated 8mo ago

Evaluation results

MethodLinks
SkylineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1060
SkylineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1052
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.72025.1038
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.52025.1036
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.72025.1034
SkylineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1027
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.72025.1026
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.32025.1025
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.52025.1022
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.32025.1015
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.52025.1014
NLDSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1010
CIPHERSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1010
NLDSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1010
BaselineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.106
AC (mean)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.106
AC (sum)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.105
BaselineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.104
BaselineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.103
AC (mean)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.103
AC (sum)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.103
CIPHERSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.103
NLDSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.102
AC (replace)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.102
CIPHERSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.101
AC (replace)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.100
AC (mean)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100
AC (replace)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100
AC (sum)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.32025.100