Question Answering on ARC-Challenge (ARC-C) (test)
53.7VWRSMART
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| SMARTBackbone=Qwen-0.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 53.7 | 46.85 | 22.04 | 52 | |
| SMARTBackbone=Mistral-7B, Plug-in=None, Safety buffer gamma=0.252026.05 | 52.44 | 41.88 | 26.67 | 68.67 | |
| R3FBackbone=Qwen-0.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 49.68 | 45 | 26.4 | 52.13 | |
| CEBackbone=Qwen-0.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 49.64 | 45.06 | 26.4 | 52.33 | |
| CE+augBackbone=Qwen-0.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 48.07 | 43.34 | 38.58 | 54.2 | |
| SMARTBackbone=Mistral-7B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 43.6 | 36.11 | 36.09 | 71.73 | |
| R3FBackbone=Qwen-0.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 39.45 | 35.64 | 28.09 | 53.93 | |
| CE+augBackbone=Qwen-0.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 38.46 | 34.26 | 38.31 | 53.87 | |
| SMARTBackbone=Qwen-0.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 36.83 | 34.11 | 27.73 | 53.13 | |
| CE+augBackbone=Mistral-7B, Plug-in=None, Safety buffer gamma=0.252026.05 | 32.91 | 28.09 | 61.18 | 75.8 | |
| SMARTBackbone=Qwen-1.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 31.61 | 27.49 | 50.18 | 75 | |
| CEBackbone=Mistral-7B, Plug-in=None, Safety buffer gamma=0.252026.05 | 31.41 | 27.13 | 52.18 | 75.33 | |
| R3FBackbone=Mistral-7B, Plug-in=None, Safety buffer gamma=0.252026.05 | 31.32 | 26.99 | 50.73 | 75.2 | |
| R3FBackbone=Mistral-7B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 30.99 | 25.8 | 51.2 | 75.07 | |
| R3FBackbone=Qwen-1.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 30.95 | 27.85 | 47.89 | 74.33 | |
| CEBackbone=Qwen-1.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 30.87 | 27.83 | 48.98 | 74.13 | |
| CE+augBackbone=Qwen-1.5B, Plug-in=None, Safety buffer gamma=0.252026.05 | 30.34 | 26.3 | 63.84 | 76.2 | |
| CE+augBackbone=Mistral-7B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 29.6 | 25.96 | 65.96 | 76.93 | |
| SMARTBackbone=Qwen-1.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 29.09 | 25.4 | 52.71 | 74.93 | |
| R3FBackbone=Qwen-1.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 28.43 | 26.33 | 48.8 | 74.2 | |
| CE+augBackbone=Qwen-1.5B, Plug-in=FragileFlow, Safety buffer gamma=0.252026.05 | 26.46 | 23.77 | 68.62 | 76.93 |