Few-shot classification on Meta-Dataset (test)
94.4OmniglotURT
Evaluation results
| Method | Links | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| URT2020.06 | 94.4 | 55.7 | 85.8 | 76.3 | 71.8 | — | 63.5 | 88.2 | 69.4 | 52.2 | 1.6 | — | — | — | — | — | — | — | — | |
| SUR2020.06 | 93.1 | 56.3 | 85.4 | 71.4 | 71.5 | — | 63.1 | 82.8 | 70.4 | 52.4 | 2.5 | — | — | — | — | — | — | — | — | |
| Simple CNAPS2019.12 | 91.7 | — | 82.4 | 74.9 | 67.8 | 77.7 | 46.9 | 90.7 | — | — | — | 58.6 | — | — | — | — | — | — | — | |
| SimpleCNAPS2020.06 | 91.7 | 58.6 | 82.4 | 74.9 | 67.8 | — | 46.9 | 90.7 | 73.5 | 46.2 | 2.4 | — | — | — | — | — | — | — | — | |
| Simple AR-CNAPS2019.12 | 91.1 | — | 81.8 | 74.3 | 72.8 | 75.2 | 45.6 | 90.3 | — | — | — | 56.5 | — | — | — | — | — | — | — | |
| TaskNorm2020.06 | 90.7 | 50.6 | 83.8 | 74.6 | 62.1 | — | 48.7 | 89.6 | 67 | 43.4 | 3.8 | — | — | — | — | — | — | — | — | |
| AR-CNAPS2019.12 | 88.4 | — | 80.5 | 72.2 | 58.3 | 72.5 | 47.4 | 86 | — | — | — | 52.3 | — | — | — | — | — | — | — | |
| CNAPS2020.06 | 88.4 | 52.3 | 80.5 | 72.2 | 58.3 | — | 47.4 | 86 | 60.2 | 42.6 | 5.1 | — | — | — | — | — | — | — | — | |
| CNAPS2019.12 | 88 | — | 76.8 | 71.4 | 62.5 | 71.9 | 46 | 89.2 | — | — | — | 51.3 | — | — | — | — | — | — | — | |
| RelationNet2019.12 | 86.6 | — | 69.7 | 54.1 | 56.6 | 61.8 | 32.6 | 76.1 | — | — | — | 30.9 | — | — | — | — | — | — | — | |
| RelationNetTraining Data Source=all datasets2019.03 | 86.57 | 30.89 | 69.71 | 54.14 | 56.56 | 61.75 | 32.56 | 76.08 | 37.48 | 27.41 | 5.8 | — | — | — | — | — | — | — | — | |
| TSPapplied on=TA2-Net2024.12 | 85.2 | — | 73.5 | 76.6 | 78.3 | 71.5 | 47 | 92.2 | 88.7 | 58.6 | 1.2 | 60.6 | — | 95.3 | 83.2 | 72.8 | 60.6 | 76.9 | 75.7 | |
| fo-MAMLTraining Data Source=all datasets2019.03 | 83.92 | 37.83 | 76.41 | 62.43 | 64.16 | 59.73 | 33.54 | 79.94 | 42.91 | 29.37 | 4.25 | — | — | — | — | — | — | — | — | |
| MAML2020.06 | 83.9 | 37.8 | 76.4 | 62.4 | 64.1 | — | 33.5 | 79.9 | 42.9 | 29.4 | 8 | — | — | — | — | — | — | — | — | |
| TSPapplied on=TSA2024.12 | 83.3 | — | 73.2 | 76 | 78.2 | 70.8 | 46.6 | 91.8 | 87.5 | 59.4 | 2 | 60.1 | — | 94.5 | 83.1 | 71.2 | 60.1 | 76.3 | 75 | |
| ProtoMAML2019.12 | 82.9 | — | 74.2 | 70 | 67.9 | 66.6 | 42 | 88.5 | — | — | — | 47.9 | — | — | — | — | — | — | — | |
| ProtoMAML2020.06 | 82.7 | 46.5 | 75.2 | 69.9 | 68.3 | — | 42 | 88.7 | 52.4 | 41.7 | 5.4 | — | — | — | — | — | — | — | — | |
| fo-Proto-MAMLTraining Data Source=all datasets2019.03 | 82.69 | 46.52 | 75.23 | 69.88 | 68.25 | 66.84 | 41.99 | 88.72 | 52.42 | 41.74 | 1.5 | — | — | — | — | — | — | — | — | |
| TA2-Net2024.12 | 81.1 | — | 72.6 | 75.1 | 76.8 | 68.4 | 45.3 | 91 | 84.1 | 58 | 3.4 | 59.3 | — | 94.9 | 82 | 70.8 | 59.3 | 75 | 73.8 | |
| ProtoNet2019.12 | 79.6 | — | 71.1 | 67 | 65.2 | 64.9 | 40.3 | 86.9 | — | — | — | 44.5 | — | — | — | — | — | — | — | |
| ProtoNet2020.06 | 79.6 | 44.5 | 71.1 | 67 | 65.2 | — | 40.3 | 86.9 | 46.5 | 39.9 | 7.3 | — | — | — | — | — | — | — | — | |
| ProtoNetTraining Data Source=all datasets2019.03 | 79.56 | 44.5 | 71.14 | 67.01 | 65.18 | 64.88 | 40.26 | 86.85 | 46.48 | 39.87 | 2.85 | — | — | — | — | — | — | — | — | |
| MatchingNet2019.12 | 78.3 | — | 69.2 | 56.4 | 61.8 | 60.8 | 33.7 | 81.9 | — | — | — | 36.1 | — | — | — | — | — | — | — | |
| MatchingNetTraining Data Source=all datasets2019.03 | 78.25 | 36.08 | 69.17 | 56.4 | 61.8 | 60.81 | 33.7 | 81.9 | 55.57 | 28.79 | 4.95 | — | — | — | — | — | — | — | — | |
| TSA2024.12 | 78.2 | — | 72.2 | 74.9 | 77.3 | 67.6 | 44.7 | 90.9 | 82.5 | 59 | 3.7 | 59.5 | — | 93.9 | 82.1 | 70.7 | 59.5 | 74.5 | 73.3 | |
| GAP+ Proto-MAML2024.12 | 77.6 | — | 68.5 | 73.5 | 71.4 | 65.4 | 38.6 | 86.8 | 66.9 | 46.8 | 6.1 | 56.7 | — | 94 | 74.5 | 63.2 | 56.7 | 68.9 | 68 | |
| BOHB-E2020.06 | 77.5 | 55.4 | 60.9 | 73.6 | 72.8 | — | 44.5 | 90.6 | 57.5 | 51.9 | 4.4 | — | — | — | — | — | — | — | — | |
| k-NNTraining Data Source=all datasets2019.03 | 74.6 | 38.55 | 64.98 | 66.35 | 63.58 | 44.88 | 37.12 | 83.47 | 40.11 | 29.55 | 5.05 | — | — | — | — | — | — | — | — | |
| k-NN2019.12 | 74.6 | — | 65 | 66.4 | 63.6 | 44.9 | 37.1 | 83.5 | — | — | — | 38.6 | — | — | — | — | — | — | — | |
| MAML2019.12 | 71.9 | — | 52.8 | 47.2 | 56.7 | 50.5 | 21 | 70.9 | — | — | — | 32.4 | — | — | — | — | — | — | — | |
| FinetuneTraining Data Source=all datasets2019.03 | 71.11 | 43.08 | 72.03 | 59.82 | 69.14 | 47.05 | 38.16 | 85.28 | 66.74 | 35.17 | 3.6 | — | — | — | — | — | — | — | — | |
| Finetune2019.12 | 71.1 | — | 72 | 59.8 | 69.1 | 47.1 | 38.2 | 85.3 | — | — | — | 43.1 | — | — | — | — | — | — | — | |
| MOKD2024.12 | 70.9 | — | 59.8 | 73.6 | 76.1 | 61.2 | 47 | 88.5 | 61.6 | 55.3 | 5.1 | 57.3 | — | 88.3 | 72.2 | 63.1 | 57.3 | 68.1 | 67.3 | |
| BOHB2024.12 | 67.6 | — | 54.1 | 70.7 | 68.3 | 50.3 | 41.4 | 87.3 | 51.8 | 48 | 7.5 | 51.9 | — | — | — | — | 51.9 | 59.9 | 59.1 | |
| OursDistillation=false, Backbone=ResNet-12, Evaluation protocol=Variable ways and shots2021.03 | 65.55 | 60.64 | 65.65 | 77.84 | 81.07 | 57.91 | 49.26 | 92.06 | 78.92 | 55.07 | — | — | 68.4 | — | — | — | — | — | — | |
| Ours-DistillDistillation=true, Backbone=ResNet-12, Evaluation protocol=Variable ways and shots2021.03 | 65.53 | 61.36 | 66.58 | 78.23 | 80.42 | 59.02 | 49.5 | 92.66 | 79.92 | 55.68 | — | — | 68.89 | — | — | — | — | — | — | |
| RFS-SimpleDistillation=false, Backbone=ResNet-12, Evaluation protocol=Variable ways and shots2021.03 | 64.92 | 60.14 | 63.12 | 77.69 | 78.59 | 62.48 | 47.12 | 91.6 | 77.51 | 57 | — | — | 68.02 | — | — | — | — | — | — | |
| RFS-DistillDistillation=true, Backbone=ResNet-12, Evaluation protocol=Variable ways and shots2021.03 | 64.31 | 61.48 | 62.32 | 79.47 | 79.28 | 60.83 | 48.53 | 91 | 76.33 | 59.28 | — | — | 68.28 | — | — | — | — | — | — | |
| fo-Proto-MAMLTraining Data Source=ILSVRC-2012 only2019.03 | 63.37 | 49.53 | 55.95 | 68.66 | 66.49 | 51.52 | 39.96 | 87.15 | 48.83 | 43.74 | 1.85 | — | — | — | — | — | — | — | — | |
| fo-Proto-MAMLDistillation=false, Evaluation protocol=Variable ways and shots2021.03 | 63.37 | 49.53 | 55.95 | 68.66 | 66.49 | 51.52 | 39.96 | 87.15 | 48.83 | 43.74 | — | — | 57.52 | — | — | — | — | — | — | |
| ALFA+ Proto-MAML2024.12 | 61.9 | — | 63.4 | 69.8 | 70.8 | 59.2 | 41.5 | 86 | 60.8 | 48.1 | 7 | 52.8 | — | — | — | — | 52.8 | 62.4 | 61.4 | |
| FLUTE2024.12 | 61.6 | — | 48.5 | 47.9 | 63.8 | 57.5 | 31.8 | 80.1 | 46.5 | 41.4 | 8.9 | 46.9 | — | 80.8 | 65.4 | 52.7 | 46.9 | 56.5 | 55.8 | |
| FinetuneTraining Data Source=ILSVRC-2012 only2019.03 | 60.85 | 45.78 | 68.69 | 57.31 | 69.05 | 42.6 | 38.2 | 85.51 | 66.79 | 34.86 | 2.9 | — | — | — | — | — | — | — | — | |
| ProtoNetTraining Data Source=ILSVRC-2012 only2019.03 | 59.98 | 50.5 | 53.1 | 68.79 | 66.56 | 48.96 | 39.71 | 85.27 | 47.12 | 41 | 2.65 | — | — | — | — | — | — | — | — | |
| fo-MAMLTraining Data Source=ILSVRC-2012 only2019.03 | 55.55 | 45.51 | 56.24 | 63.61 | 68.04 | 43.96 | 32.1 | 81.74 | 50.93 | 35.3 | 3.7 | — | — | — | — | — | — | — | — | |
| MatchingNetTraining Data Source=ILSVRC-2012 only2019.03 | 52.27 | 45 | 48.97 | 62.21 | 64.15 | 42.87 | 33.97 | 80.13 | 47.8 | 34.99 | 4.65 | — | — | — | — | — | — | — | — | |
| RelationNetTraining Data Source=ILSVRC-2012 only2019.03 | 45.35 | 34.69 | 40.73 | 49.51 | 52.97 | 43.3 | 30.55 | 68.76 | 33.67 | 29.15 | 6.55 | — | — | — | — | — | — | — | — | |
| k-NNTraining Data Source=ILSVRC-2012 only2019.03 | 37.07 | 41.03 | 46.81 | 50.13 | 66.36 | 32.06 | 36.16 | 83.1 | 44.59 | 30.38 | 5.7 | — | — | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 61.5 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 64.07 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.87 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.98 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 61.72 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.57 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.78 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.51 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.05 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.47 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.84 | — | — | — | — | — | — | |
| EXP3-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 66.84 | — | — | — | — | — | — | |
| Exploit-OnlyBase Model=T5-XL, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 59.79 | — | — | — | — | — | — | |
| Exploit-OnlyBase Model=T5-XL, Auxiliary Data=P32023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.49 | — | — | — | — | — | — | |
| Exploit-OnlyBase Model=T0-3B, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.87 | — | — | — | — | — | — | |
| Exploit-OnlyBase Model=T0-3B, Auxiliary Data=P32023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.87 | — | — | — | — | — | — | |
| Explore-OnlyBase Model=T5-XL, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 59.18 | — | — | — | — | — | — | |
| Explore-OnlyBase Model=T5-XL, Auxiliary Data=P32023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.61 | — | — | — | — | — | — | |
| Explore-OnlyBase Model=T0-3B, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 61.17 | — | — | — | — | — | — | |
| Explore-OnlyBase Model=T0-3B, Auxiliary Data=P32023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.77 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T5-XL, Auxiliary Data=T0Mix, Variant=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 53.22 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T5-XL, Auxiliary Data=P3, Variant=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 55.19 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T0-3B, Auxiliary Data=T0Mix, Variant=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 59.47 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T0-3B, Auxiliary Data=P3, Variant=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.66 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T5-XL, Auxiliary Data=T0Mix, Variant=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 55.98 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T5-XL, Auxiliary Data=P3, Variant=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 56.4 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T0-3B, Auxiliary Data=T0Mix, Variant=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.47 | — | — | — | — | — | — | |
| Loss-ScalingBase Model=T0-3B, Auxiliary Data=P3, Variant=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 60.7 | — | — | — | — | — | — | |
| Target-OnlyBase Model=T5-XL, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 52.82 | — | — | — | — | — | — | |
| Target-OnlyBase Model=T0-3B, Auxiliary Data=T0Mix2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 56.44 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.01 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.52 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.35 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=GA2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 66.29 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 61.67 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.21 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.85 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=GMS2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 66 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=T0Mix, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.08 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T5-XL, Auxiliary Data=P3, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 65.63 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=T0Mix, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 62.93 | — | — | — | — | — | — | |
| UCB1-FLADBase Model=T0-3B, Auxiliary Data=P3, Reward Function=AGG2023.02 | — | — | — | — | — | — | — | — | — | — | — | — | 66.29 | — | — | — | — | — | — |