List your company

Natural Language Understanding on SuperGLUE 1,000 examples (test)

86.7BoolQFT
64.23670.06875.981.732Feb 24, 2024

Evaluation results

MethodLinks
FTModel=Mistral-7b, Optimization=Full-parameter fine-tuning with Adam2024.0286.787.171.286.195.691.286.3
S-MeZOModel=Mistral-7b, Variant=Sparse MeZO2024.0285.384.564.384.994.286.183.2
LoRAModel=Mistral-7b2024.0284.887.468.283.995.69185.2
R-MeZOModel=Mistral-7b, Variant=MeZO with Random Mask2024.028478.763.283.192.484.180.9
MeZO - LoRAModel=Mistral-7b2024.0283.580.160.782.693.886.981.3
MeZOModel=Mistral-7b2024.0281.680.963.282.793.886.781.5
ICLModel=Mistral-7b, Strategy=In-Context Learning2024.0276.77861.471.394.69078.7
Zero-ShotModel=Mistral-7b2024.0269.355.25057.155.58461.9
ICLModel=LLaMA-7b, Protocol=In-Context Learning2024.0267.454.552.758.781.284.466.5
Zero-ShotModel=LLaMA-7b2024.0265.149.550.655.879.759.760.1