Question Answering on HotpotQA-E 2018 (Full)
78F1 ScoreSkyline
Evaluation Results
| Method | Links | |
|---|---|---|
| SkylineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 78 | |
| SkylineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 77 | |
| KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.72025.10 | 71 | |
| KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.52025.10 | 61 | |
| KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.32025.10 | 59 | |
| KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.52025.10 | 59 | |
| KVCommSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated, compression_rate=0.72025.10 | 59 | |
| SkylineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 58 | |
| NLDSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 52 | |
| CIPHERSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 51 | |
| NLDSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 45 | |
| KVCommSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B, compression_rate=0.32025.10 | 44 | |
| KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.72025.10 | 40 | |
| CIPHERSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 28 | |
| AC (mean)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 24 | |
| NLDSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 24 | |
| AC (mean)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 24 | |
| AC (sum)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 23 | |
| AC (sum)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 23 | |
| BaselineSender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 22 | |
| KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.52025.10 | 21 | |
| BaselineSender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 21 | |
| BaselineSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 15 | |
| AC (replace)Sender Model (Ms)=Orion-ehristoforu/falcon3-ultraset, Receiver Model (Mr)=huihui-ai/Falcon3-7B-Instruct-abliterated2025.10 | 12 | |
| AC (replace)Sender Model (Ms)=huihui-ai/Llama-3.2-3B-Instruct-abliterated, Receiver Model (Mr)=suayptalha/DeepSeek-R1-Distill-Llama-3B2025.10 | 6 | |
| CIPHERSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 4 | |
| AC (mean)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 3 | |
| AC (sum)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 3 | |
| KVCommSender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B, compression_rate=0.32025.10 | 2 | |
| AC (replace)Sender Model (Ms)=Orion-zhen/Qwen2.5-7B-Instruct-Uncensored, Receiver Model (Mr)=bespokelabs/Bespoke-Stratos-7B2025.10 | 0 |