List your company
ResearchFormerly WizWand

Question Answering on Natural Questions

70.58EMCOMI
3.167220.668638.1755.6714Apr 5, 2022Nov 24, 2022Jul 15, 2023Mar 4, 2024Oct 23, 2024Jun 13, 2025Feb 2, 2026
Updated 8mo ago

Evaluation results

MethodLinks
COMICompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.0270.58
COMICompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.0269.38
COMICompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.0264.22
COMICompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.0257.48
Activation BeaconCompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.0251.83
Activation BeaconCompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.0248.47
COMICompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.0247.53
Activation BeaconCompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.0241.43
PaLM 540BEvaluation Protocol=Few-shot, Number of Shots=642022.0439.6
PaLM 2-Lprompting=1-shot2023.0537.5
LLaMA2 SFT 7B# Shot=0-shot, Total Params=6.7B, Activated Params=6.7B, FLOPs per 4K Tokens=187.9T2024.0135.2
DeepSeekMoE Chat 16B# Shot=0-shot, Total Params=16.4B, Activated Params=2.8B, FLOPs per 4K Tokens=74.4T2024.0135.1
DeepSeek Chat 7B# Shot=0-shot, Total Params=6.9B, Activated Params=6.9B, FLOPs per 4K Tokens=183.5T2024.0132.7
GLaM 62B/64EEvaluation Protocol=Few-shot, Number of Shots=12022.0432.5
PaLM 2-Mprompting=1-shot2023.0532
Camelidae-8x34B-proShots=0-shot2024.0131.2
DeepSeekMoE 16B ChatShots=0-shot2024.0130.7
Llama2 70B ChatShots=0-shot2024.0130.6
PaLMprompting=1-shot2023.0529.3
Qwen 72B ChatShots=0-shot2024.0129.3
PaLM 540BEvaluation Protocol=1-shot2022.0429.3
GLaM 62B/64EEvaluation Protocol=1-shot2022.0426.3
LLaMA2 7B# Shot=5-shot, # Total Params=6.7B, # Activated Params=6.7B, FLOPs per 4K Tokens=187.9T, # Training Tokens=2T2024.0125.5
DeepSeekMoE 16B# Shot=5-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.0125.5
DeepSeekMoE 16B# Shot=5-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.0125.5
PaLM 2-Sprompting=1-shot2023.0525.3
DeepSeekMoE 145B# Shot=5-shot2024.0125
GLaM 62B/64EEvaluation Protocol=0-shot2022.0424.7
Yi 34B ChatShots=0-shot2024.0123.7
DeepSeekMoE 142B (Half Activated)# Shot=5-shot2024.0123.5
DeepSeek 67B (Dense)# Shot=5-shot2024.0122.6
Mixtral 8x7B Inst.Shots=0-shot2024.0122.5
LongLLMLinguaCompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.0222.49
DeepSeek 7B (Dense)# Shot=5-shot, # Total Params=6.9B, # Activated Params=6.9B, FLOPs per 4K Tokens=183.5T, # Training Tokens=2T2024.0122.2
PaLM 540BEvaluation Protocol=0-shot2022.0421.2
LongLLMLinguaCompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.0219.81
LongLLMLinguaCompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.0219.44
GShard 137B# Shot=5-shot2024.0119
LLMLingua-2-largeCompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.0215.52
LongLLMLinguaCompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.0215.07
LLMLingua-2-largeCompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.0214.39
LongLLMLinguaCompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.0213.37
LLMLingua-2-largeCompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.0212.77
Activation BeaconCompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.0211.68
LLMLingua-2-largeCompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.0210.36
LLMLingua-2-largeCompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.029.6
Activation BeaconCompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.027.95
SnapKVCompression Rate=8x, Backbone=Qwen2-7B-Instruct2026.026.25
SnapKVCompression Rate=16x, Backbone=Qwen2-7B-Instruct2026.026.06
SnapKVCompression Rate=2x, Backbone=Qwen2-7B-Instruct2026.026.01
SnapKVCompression Rate=4x, Backbone=Qwen2-7B-Instruct2026.025.99
SnapKVCompression Rate=32x, Backbone=Qwen2-7B-Instruct2026.025.76