Classification on MTOP Domain (test)
99.16AccuracyCayley-Encoder (GCN)
Evaluation Results
| Method | Links | |
|---|---|---|
| Cayley-Encoder (GCN)Base Model=Gemma2-2B2026.03 | 99.16 | |
| FC-Encoder (GIN)Base Model=Gemma2-2B2026.03 | 99.07 | |
| Cayley-Encoder (GCN)Base Model=Llama3-8B2026.03 | 99.03 | |
| FC-Encoder (GCN)Base Model=Llama3-8B2026.03 | 98.99 | |
| Cayley-Encoder (GIN)Base Model=Llama3-8B2026.03 | 98.98 | |
| Set-EncoderBase Model=Gemma2-2B2026.03 | 98.92 | |
| Cayley-Encoder (GIN)Base Model=Gemma2-2B2026.03 | 98.88 | |
| FC-Encoder (GCN)Base Model=Gemma2-2B2026.03 | 98.87 | |
| Cayley-Encoder (GIN)Base Model=Pythia-410m2026.03 | 98.77 | |
| Set-EncoderBase Model=Llama3-8B2026.03 | 98.77 | |
| FC-Encoder (GIN)Base Model=Llama3-8B2026.03 | 98.77 | |
| FC-Encoder (GIN)Base Model=Pythia-410m2026.03 | 98.68 | |
| Cayley-Encoder (GCN)Base Model=Pythia-410m2026.03 | 98.67 | |
| FC-Encoder (GCN)Base Model=Pythia-410m2026.03 | 98.65 | |
| MLP Last LayerBase Model=Llama3-8B2026.03 | 98.58 | |
| DWATTBase Model=Gemma2-2B2026.03 | 98.39 | |
| MLP Last LayerBase Model=Gemma2-2B2026.03 | 98.37 | |
| DWATTBase Model=Pythia-410m2026.03 | 98.03 | |
| DWATTBase Model=Llama3-8B2026.03 | 97.97 | |
| Set-EncoderBase Model=Pythia-410m2026.03 | 97.59 | |
| MLP Last LayerBase Model=Pythia-410m2026.03 | 96.97 | |
| Best Single LayerBase Model=Llama3-8B2026.03 | 89.01 | |
| MLP Best LayerBase Model=Pythia-410m2026.03 | 87.25 | |
| Best Single LayerBase Model=Gemma2-2B2026.03 | 87.09 | |
| WeightedBase Model=Gemma2-2B2026.03 | 85.17 | |
| Last LayerBase Model=Llama3-8B2026.03 | 84.42 | |
| MLP Best LayerBase Model=Gemma2-2B2026.03 | 84.24 | |
| WeightedBase Model=Llama3-8B2026.03 | 83.85 | |
| Best Single LayerBase Model=Pythia-410m2026.03 | 83.78 | |
| Last LayerBase Model=Pythia-410m2026.03 | 80.88 | |
| Last LayerBase Model=Gemma2-2B2026.03 | 80.79 | |
| WeightedBase Model=Pythia-410m2026.03 | 79.79 | |
| MLP Best LayerBase Model=Llama3-8B2026.03 | 47.64 |