List your company
ResearchFormerly WizWand

Summarization on SAMSum Full 2019

37F1 ScoreCIPHER
-1.488.5118.528.49Oct 2, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CIPHERSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1037
SkylineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1036
SkylineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1035
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.72025.1035
SkylineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1033
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.72025.1032
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.52025.1031
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.52025.1030
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.72025.1029
NLDSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1028
CIPHERSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1028
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.52025.1028
NLDSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1028
NLDSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1028
KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.32025.1028
BaselineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1027
BaselineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1026
AC (mean)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1026
AC (replace)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1026
AC (sum)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.1026
AC (mean)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1026
AC (replace)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1026
AC (sum)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1026
KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.32025.1025
BaselineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.1025
KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.32025.1018
CIPHERSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.1017
AC (sum)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.104
AC (mean)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.101
AC (replace)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.100