Multimodal Reasoning on GEO-AT Protein
41.7METEORCuttlefish + Llama-3.1-8B-I
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Cuttlefish + Llama-3.1-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Llama-3.1-8B-Instruct2026.02 | 41.7 | 89.6 | |
| Cuttlefish + Qwen3-8BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen3-8B2026.02 | 37.7 | 88.8 | |
| Cuttlefish + Mistral-3-8B-IArchitecture=Cuttlefish augmentation, Backbone Model=Mistral-3-8B-Instruct2026.02 | 33.3 | 85.8 | |
| Cuttlefish + R1-7BArchitecture=Cuttlefish augmentation, Backbone Model=DeepSeek-R1-D-Qwen-7B2026.02 | 32.7 | 80.1 | |
| Cuttlefish + Mistral3-8B-RArchitecture=Cuttlefish augmentation, Backbone Model=Mistral3-8B-Reasoning2026.02 | 32 | 75.6 | |
| Cuttlefish + Qwen2.5-7BArchitecture=Cuttlefish augmentation, Backbone Model=Qwen2.5-7B2026.02 | 31.7 | 81.6 | |
| Cuttlefish + GLM-4-9BArchitecture=Cuttlefish augmentation, Backbone Model=GLM-4-9B2026.02 | 26.2 | 83.1 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 21.9 | 61.7 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 19.2 | 75.5 | |
| Llama-3.1-8B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 17.8 | 74.2 | |
| DeepSeek-R1-D-Qwen-7BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 14.1 | 70.7 | |
| Mistral-3-8B-Instruct-2512Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 13.4 | 71.4 | |
| GLM-4-9B-0414Reasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 11 | 67.2 | |
| Mistral-3-8B-Reasoning-2512Reasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 10.8 | 81.1 | |
| Qwen-2.5-7B-InstructReasoning Capability=Non-Reasoning, Modality Handling=Sequence Only2026.02 | 9.1 | 65.1 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Sequence Only2026.02 | 4.4 | 67.9 | |
| Qwen3-8BReasoning Capability=Reasoning, Modality Handling=Modality Enhanced Tokenizer2026.02 | 2.7 | 57.3 |