General Downstream Evaluation on All Downstream Tasks 15 tasks
36.4Average Accuracy (All Tasks)MIDAS
Evaluation Results
| Method | Links | |
|---|---|---|
| MIDASParameters=8B, d(i)/i (block size)=9, Schedule=PROP-2, Speedup=1.26x2024.09 | 36.4 | |
| MIDASParameters=2B, d(i)/i (block size)=8, Schedule=PROP-2, Speedup=1.24x2024.09 | 32.9 | |
| BaselineParameters=8B, d(i)/i (block size)=72, Schedule=1x, Speedup=1x2024.09 | 32.8 | |
| MIDASParameters=2B, d(i)/i (block size)=8, Schedule=PROP-1, Speedup=1.39x2024.09 | 29.5 | |
| MIDASParameters=1B, d(i)/i (block size)=4, Schedule=PROP-3, Speedup=1.16x2024.09 | 28.3 | |
| BaselineParameters=2B, d(i)/i (block size)=48, Schedule=1x, Speedup=1x2024.09 | 28 | |
| MIDASParameters=1B, d(i)/i (block size)=3, Schedule=PROP-2, Speedup=1.26x2024.09 | 27.5 | |
| MIDASParameters=1B, d(i)/i (block size)=4, Schedule=PROP-2, Speedup=1.24x2024.09 | 26.8 | |
| MIDASParameters=1B, d(i)/i (block size)=4, Schedule=PROP-1, Speedup=1.39x2024.09 | 26.7 | |
| GRADSTACKParameters=2B, d(i)/i (block size)=8, Schedule=PROP-2, Speedup=1.24x2024.09 | 25.9 | |
| MIDASParameters=1B, d(i)/i (block size)=3, Schedule=PROP-1, Speedup=1.41x2024.09 | 25.6 | |
| BaselineParameters=1B, d(i)/i (block size)=24, Schedule=1x, Speedup=1x2024.09 | 24 | |
| GRADSTACKParameters=1B, d(i)/i (block size)=4, Schedule=PROP-1, Speedup=1.39x2024.09 | 22.6 | |
| GRADSTACKParameters=1B, d(i)/i (block size)=4, Schedule=PROP-2, Speedup=1.24x2024.09 | 20.4 |