Multimodal Reasoning on GEO-AT DNA
52.9METEORCuttlefish + Llama-3.1-8B-I
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Cuttlefish + Llama-3.1-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Llama-3.1-8B-Instruct2026.02 | 52.9 | 81.6 | |
| Cuttlefish + GLM-4-9BArchitecture=Cuttlefish augmentation, Backbone Model=GLM-4-9B2026.02 | 44.3 | 88.8 | |
| Cuttlefish + Qwen3-8BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen3-8B2026.02 | 39.1 | 86 | |
| Cuttlefish + Mistral-3-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Mistral-3-8B-Instruct2026.02 | 35.8 | 85.2 | |
| Cuttlefish + R1-7BArchitecture=Cuttlefish augmentation, Backbone Model=DeepSeek-R1-D-Qwen-7B2026.02 | 33 | 80.2 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 26.4 | 80.6 | |
| Cuttlefish + Qwen2.5-7BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen2.5-7B2026.02 | 26.2 | 80.6 | |
| Cuttlefish + Mistral3-8B-RArchitecture=Cuttlefish augmentation, Backbone Model=Mistral3-8B-Reasoning2026.02 | 25.6 | 79.8 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 20.6 | 65.9 | |
| GLM-4-9B-0414Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 20.4 | 69 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 17.7 | 79.2 | |
| Llama-3.1-8B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 17.5 | 65.8 | |
| Qwen-2.5-7B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 17.1 | 64.4 | |
| Mistral-3-8B-Instruct-2512Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 15.6 | 66.5 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 14.9 | 75.6 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 14.7 | 64.4 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 13.1 | 66.5 |