Multimodal Reasoning on GEO-AT Molecule
0.415METEORCuttlefish + Mistral-3-8B-I
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Cuttlefish + Mistral-3-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Mistral-3-8B-Instruct2026.02 | 0.415 | 0.653 | |
| Cuttlefish + Llama-3.1-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Llama-3.1-8B-Instruct2026.02 | 0.391 | 0.875 | |
| Cuttlefish + Qwen3-8BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen3-8B2026.02 | 0.389 | 0.853 | |
| Cuttlefish + R1-7BArchitecture=Cuttlefish augmentation, Backbone Model=DeepSeek-R1-D-Qwen-7B2026.02 | 0.342 | 0.812 | |
| Cuttlefish + GLM-4-9BArchitecture=Cuttlefish augmentation, Backbone Model=GLM-4-9B2026.02 | 0.327 | 0.83 | |
| Cuttlefish + Qwen2.5-7BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen2.5-7B2026.02 | 0.314 | 0.863 | |
| Cuttlefish + Mistral3-8B-RArchitecture=Cuttlefish augmentation, Backbone Model=Mistral3-8B-Reasoning2026.02 | 0.309 | 0.776 | |
| Llama-3.1-8B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 0.229 | 0.778 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 0.204 | 0.673 | |
| Mistral-3-8B-Instruct-2512Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 0.185 | 0.732 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 0.185 | 0.823 | |
| GLM-4-9B-0414Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 0.174 | 0.644 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 0.16 | 0.721 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 0.158 | 0.709 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 0.147 | 0.753 | |
| Qwen-2.5-7B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 0.137 | 0.691 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 0.103 | 0.715 |