List your company
ResearchFormerly WizWand

Question Answering on MuSiQuest-E 2022 (Full)

46F1 ScoreKVComm
-1.8410.582335.42Oct 2, 2025
Updated 8mo ago

Evaluation results

MethodLinks
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.52025.1046
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.32025.1040
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.72025.1036
SkylineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1033
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.72025.1030
NLDSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1028
SkylineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1025
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.52025.1025
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.72025.1021
CIPHERSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1020
NLDSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1018
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.32025.1011
SkylineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1010
CIPHERSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.109
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.52025.108
NLDSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.107
AC (mean)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.107
BaselineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.106
AC (mean)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.106
AC (sum)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.106
BaselineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.106
BaselineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.106
AC (sum)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.106
CIPHERSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.102
AC (replace)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.101
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.32025.101
AC (replace)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.101
AC (mean)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100
AC (replace)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100
AC (sum)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100