List your company

Long-term Memory and Reasoning on Grocery Shopping CSQA Source (train and dev)

81.44ACCcSirLLM
45.840855.082964.32573.5671May 21, 2024

Evaluation results

MethodLinks
SirLLMLLM Backbone=Yi-34b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0581.4489.60.0963.31
StreamLLMLLM Backbone=Yi-34b, # Entropy=0, # Recent=1020, Attention Sink Size=4, Ndecay=12024.0581.3526.29——
SirLLMLLM Backbone=Yi-6b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0572.4499.271.1173.54
StreamLLMLLM Backbone=Yi-6b, # Entropy=0, # Recent=1020, Attention Sink Size=4, Ndecay=12024.0571.3325.73——
RandomLLMLLM Backbone=Yi-6b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0570.3377.55-151.82
IntervalLLMLLM Backbone=Yi-6b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0563.9821.72-7.2-4.01
SirLLMLLM Backbone=Vicuna-13b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0560.2397.080.1372.63
StreamLLMLLM Backbone=Vicuna-13b, # Entropy=0, # Recent=1020, Attention Sink Size=4, Ndecay=12024.0560.124.45——
SirLLMLLM Backbone=Vicuna-7b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0551.0496.170.267.52
RandomLLMLLM Backbone=Vicuna-7b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0550.9785.040.1356.39
StreamLLMLLM Backbone=Vicuna-7b, # Entropy=0, # Recent=1020, Attention Sink Size=4, Ndecay=12024.0550.8428.65——
IntervalLLMLLM Backbone=Vicuna-7b, # Entropy=1020, # Recent=0, Attention Sink Size=4, Ndecay=12024.0547.2123.72-3.63-4.93