Language Understanding on MMLU Expert (test)
0.7566Astronomy AccuracyOracle Expert
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Oracle ExpertSetup=1 Meta + 6 Experts2024.03 | 0.7566 | 0.6966 | 0.8626 | 0.8512 | 0.784 | 0.7796 | 0.7844 | |
| ETRSetup=1 Meta + 6 Experts2024.03 | 0.7236 | 0.6344 | 0.8473 | 0.7603 | 0.7006 | 0.7755 | 0.7352 | |
| LLM-BlenderSetup=1 Meta + 6 Experts, Variant=Top12024.03 | 0.6842 | 0.5448 | 0.7405 | 0.6694 | 0.6759 | 0.5388 | 0.6369 | |
| meta LLMSetup=1 Meta + 6 Experts2024.03 | 0.5987 | 0.4966 | 0.7099 | 0.6364 | 0.6111 | 0.6041 | 0.6073 | |
| LLM-BlenderSetup=1 Meta + 6 Experts, Variant=Fused2024.03 | 0.5987 | 0.4621 | 0.7405 | 0.6446 | 0.5926 | 0.449 | 0.568 | |
| Meta-Prompting-ESetup=1 Meta + 6 Experts2024.03 | 0.5855 | 0.6414 | 0.8321 | 0.7273 | 0.7222 | 0.5959 | 0.6789 | |
| Multi LLM DebateSetup=1 Meta + 6 Experts2024.03 | 0.5526 | 0.5448 | 0.7176 | 0.6198 | 0.6389 | 0.6449 | 0.6229 |