List your company
ResearchFormerly WizWand

completion task on MemoTrap (test)

61.83AccuracyIRCAN-CAD
-2.473214.220930.91547.6091Jun 26, 2024
Updated 8mo ago

Evaluation results

MethodLinks
IRCAN-CADModel Backbone=LLaMA-2-7B2024.0661.8312.79
IRCAN-CADModel Backbone=LLaMA-2-13B2024.0658.649.38
IRCANModel Backbone=LLaMA-2-7B2024.0656.0818.55
IRCAN-CADModel Backbone=LLaMA-3-8B2024.0654.3716.84
IRCANModel Backbone=LLaMA-2-13B2024.0652.2414.29
IRCANModel Backbone=LLaMA-3-8B2024.0647.7620.68
IRCAN-CADModel Backbone=Amber (7B)2024.0645.8425.59
CADModel Backbone=LLaMA-2-7B2024.0644.5632.84
IRCANModel Backbone=Amber (7B)2024.0641.1531.56
CADModel Backbone=LLaMA-3-8B2024.0639.6636.03
CADModel Backbone=LLaMA-2-13B2024.0639.2323.24
CADModel Backbone=Amber (7B)2024.0636.0734.97
ITI (Probe Weight Direction)Model Backbone=LLaMA-2-7B2024.0631.7744.78
ITI (Mass Mean Shift)Model Backbone=LLaMA-2-7B2024.0631.3444.99
IRCAN-CADModel Backbone=Gemma-2B2024.0627.0817.27
OriginalModel Backbone=LLaMA-2-13B2024.0627.0846.7
ITI (Probe Weight Direction)Model Backbone=Gemma-2B2024.0626.0125.16
OriginalModel Backbone=Amber (7B)2024.0624.9548.4
IRCANModel Backbone=Gemma-2B2024.0624.7330.28
CADModel Backbone=Gemma-2B2024.0624.5221.96
OriginalModel Backbone=LLaMA-2-7B2024.0624.5250.96
OriginalModel Backbone=Gemma-2B2024.0623.2435.82
ITI (Probe Weight Direction)Model Backbone=LLaMA-2-13B2024.0623.0351.17
ITI (Mass Mean Shift)Model Backbone=LLaMA-2-13B2024.0622.652.45
ITI (Probe Weight Direction)Model Backbone=Amber (7B)2024.0620.2643.5
OriginalModel Backbone=LLaMA-3-8B2024.0620.2653.3
ITI (Probe Weight Direction)Model Backbone=LLaMA-3-8B2024.0618.3453.52
ITI (Mass Mean Shift)Model Backbone=LLaMA-3-8B2024.0618.1253.94
ITI (Mass Mean Shift)Model Backbone=Gemma-2B2024.0600
ITI (Mass Mean Shift)Model Backbone=Amber (7B)2024.0600