List your company
ResearchFormerly WizWand

Classification on MTOPIntent (test)

96.46AccuracyCayley-Encoder (GIN)
58.583268.416678.2588.0834Mar 24, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Cayley-Encoder (GIN)Base Model=Llama3-8B2026.0396.46
Cayley-Encoder (GIN)Base Model=Gemma2-2B2026.0396.43
FC-Encoder (GCN)Base Model=Llama3-8B2026.0396.19
Cayley-Encoder (GCN)Base Model=Gemma2-2B2026.0395.97
Cayley-Encoder (GCN)Base Model=Llama3-8B2026.0395.9
FC-Encoder (GIN)Base Model=Llama3-8B2026.0395.65
Set-EncoderBase Model=Llama3-8B2026.0395.43
FC-Encoder (GCN)Base Model=Gemma2-2B2026.0395.34
FC-Encoder (GCN)Base Model=Pythia-410m2026.0395.04
Cayley-Encoder (GIN)Base Model=Pythia-410m2026.0394.72
FC-Encoder (GIN)Base Model=Gemma2-2B2026.0394.47
Set-EncoderBase Model=Gemma2-2B2026.0394.37
FC-Encoder (GIN)Base Model=Pythia-410m2026.0394.32
Cayley-Encoder (GCN)Base Model=Pythia-410m2026.0394.19
MLP Last LayerBase Model=Gemma2-2B2026.0392.73
DWATTBase Model=Llama3-8B2026.0392.41
Set-EncoderBase Model=Pythia-410m2026.0392.21
MLP Last LayerBase Model=Llama3-8B2026.0392.09
DWATTBase Model=Pythia-410m2026.0391.66
DWATTBase Model=Gemma2-2B2026.0390.93
MLP Last LayerBase Model=Pythia-410m2026.0383.75
Best Single LayerBase Model=Llama3-8B2026.0378.17
Best Single LayerBase Model=Gemma2-2B2026.0375.36
MLP Best LayerBase Model=Gemma2-2B2026.0373.69
Last LayerBase Model=Llama3-8B2026.0373.39
WeightedBase Model=Gemma2-2B2026.0373.2
WeightedBase Model=Llama3-8B2026.0371.78
Best Single LayerBase Model=Pythia-410m2026.0371.18
MLP Best LayerBase Model=Pythia-410m2026.0370.79
Last LayerBase Model=Gemma2-2B2026.0368.02
Last LayerBase Model=Pythia-410m2026.0366.97
WeightedBase Model=Pythia-410m2026.0361.68
MLP Best LayerBase Model=Llama3-8B2026.0360.04