# Multi-Task

Source: [Multi-Task](<https://www.sota2.com/research/task/multi-task>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.



## Benchmarks

Page 1 of 7. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/task/multi-task.md?page=2>): Follow this link to discover more pages.

- [Multi-task Language Understanding on MMLU](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Language Understanding on MMLU (MMLU Acc)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-mmlu-acc.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Language Understanding on MMLU (Accuracy)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-accuracy.md>): Multi-task Language Understanding; MMLU.

- [Accuracy on MMLU (Multi-task Language Understanding)](<https://www.sota2.com/research/sota/accuracy-on-mmlu-multi-task-language-understanding.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Language Understanding on MMLU (MMLU Score)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-mmlu-score.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Language Understanding on MMLU (test)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-test.md>): Multi-task Language Understanding; MMLU (test).

- [Multi-task Linear Regression on Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test)](<https://www.sota2.com/research/sota/multi-task-linear-regression-on-synthetic-multi-task-linear-regression-n-d-50-0-2-test.md>): Multi-task Linear Regression (Local Parameter Estimation); Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test).

- [Multi-task Logistic Regression on Synthetic Multi-task Logistic Regression (n=d=50, epsilon=0.2)](<https://www.sota2.com/research/sota/multi-task-logistic-regression-on-synthetic-multi-task-logistic-regression-n-d-50-epsilon-0-2.md>): Multi-task Logistic Regression; Synthetic Multi-task Logistic Regression (n=d=50, epsilon=0.2).

- [Multi-task Linear Regression (Global Parameter Estimation) on Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test)](<https://www.sota2.com/research/sota/multi-task-linear-regression-global-parameter-estimation-on-synthetic-multi-task-linear-regression-n-d-50-0-2-test.md>): Multi-task Linear Regression (Global Parameter Estimation); Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test).

- [Multi-Task Incremental Learning on MTIL Order II](<https://www.sota2.com/research/sota/multi-task-incremental-learning-on-mtil-order-ii.md>): Multi-Task Incremental Learning; MTIL Order II.

- [Multi-Task Learning on PASCAL Context](<https://www.sota2.com/research/sota/multi-task-learning-on-pascal-context.md>): Multi-Task Learning; PASCAL Context.

- [Multi-task Language Understanding on MMLU (MMLU, Avg.5)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-mmlu-avg-5.md>): Multi-task Language Understanding; MMLU.

- [Multi-Task Instruct-Tuning on SuperNI (test)](<https://www.sota2.com/research/sota/multi-task-instruct-tuning-on-superni-test.md>): Multi-Task Instruct-Tuning; SuperNI (test).

- [Multi-Task Adaptation on PASCAL-Context (test)](<https://www.sota2.com/research/sota/multi-task-adaptation-on-pascal-context-test.md>): Multi-Task Adaptation; PASCAL-Context (test).

- [Multi-task Knowledge and Reasoning on MMLU-Pro](<https://www.sota2.com/research/sota/multi-task-knowledge-and-reasoning-on-mmlu-pro.md>): Multi-task Knowledge and Reasoning; MMLU-Pro.

- [Multi-task Language Understanding on MMLU Pro (Accuracy)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-accuracy.md>): Multi-task Language Understanding; MMLU Pro.

- [Accuracy on MMLU-Pro](<https://www.sota2.com/research/sota/accuracy-on-mmlu-pro.md>): Multi-task Language Understanding; MMLU-Pro.

- [Multi-task Language Understanding on MMLU-Pro (pass@1)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-pass-at-1.md>): Multi-task Language Understanding; MMLU-Pro.

- [Multi-task Question Answering on ARC, OBQA, and Riddle Combined (test)](<https://www.sota2.com/research/sota/multi-task-question-answering-on-arc-obqa-and-riddle-combined-test.md>): Multi-task Question Answering; ARC, OBQA, and Riddle Combined (test).

- [Multi-task Image Classification on 20 Vision Tasks](<https://www.sota2.com/research/sota/multi-task-image-classification-on-20-vision-tasks.md>): Multi-task Image Classification; 20 Vision Tasks.

- [Multi-task Image Classification on 14 vision tasks](<https://www.sota2.com/research/sota/multi-task-image-classification-on-14-vision-tasks.md>): Multi-task Image Classification; 14 vision tasks.

- [Multi-task Image Classification on 8 vision tasks](<https://www.sota2.com/research/sota/multi-task-image-classification-on-8-vision-tasks.md>): Multi-task Image Classification; 8 vision tasks.

- [Multi-task Language Understanding on MMLU-Redux](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-redux.md>): Multi-task Language Understanding; MMLU-Redux.

- [Multi-task Knowledge Understanding on MMLU-Pro](<https://www.sota2.com/research/sota/multi-task-knowledge-understanding-on-mmlu-pro.md>): Multi-task Knowledge Understanding; MMLU-Pro.

- [Multi-task Language Understanding on MMLU (Top-1 agreement)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-top-1-agreement.md>): Multi-task Language Understanding; MMLU.

- [MMLU Question Answering (Accuracy and Performance Gain)](<https://www.sota2.com/research/sota/mmlu-question-answering-accuracy-and-performance-gain.md>): Multi-task Question Answering; MMLU.

- [Multi-task Learning on Cityscapes (test)](<https://www.sota2.com/research/sota/multi-task-learning-on-cityscapes-test.md>): Multi-task Learning; Cityscapes (test).

- [Multi-task Language Understanding on MMLU-PRO (Acc %, ECE)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-acc-percent-ece.md>): Multi-task Language Understanding; MMLU-PRO.

- [Multi-task model merging on 14 vision tasks](<https://www.sota2.com/research/sota/multi-task-model-merging-on-14-vision-tasks.md>): Multi-task model merging; 14 vision tasks.

- [Multi-task Language Understanding on MMLU Biz](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-biz.md>): Multi-task Language Understanding; MMLU Biz.

- [Multi-task Evaluation on PostTrainBench](<https://www.sota2.com/research/sota/multi-task-evaluation-on-posttrainbench.md>): Multi-task Evaluation; PostTrainBench.

- [Multi-task Language Understanding on MMLU Pro (Subject Breakdown)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-subject-breakdown.md>): Multi-task Language Understanding; MMLU Pro.

- [Multi-task Model Merging on 20 Vision Tasks](<https://www.sota2.com/research/sota/multi-task-model-merging-on-20-vision-tasks.md>): Multi-task model merging; 20 vision tasks.

- [Multi-task Evaluation on GRIT ablation set](<https://www.sota2.com/research/sota/multi-task-evaluation-on-grit-ablation-set.md>): General Robust Image Task (GRIT) multi-task evaluation; GRIT ablation set (same).

- [Unified Multi-task Language Understanding and Instruction Following on Open LLM Leaderboard v1 (test)](<https://www.sota2.com/research/sota/unified-multi-task-language-understanding-and-instruction-following-on-open-llm-leaderboard-v1-test.md>): Unified Multi-task Language Understanding and Instruction Following; Open LLM Leaderboard v1 (test).

- [Multi-task Generalization on Other tasks (test)](<https://www.sota2.com/research/sota/multi-task-generalization-on-other-tasks-test.md>): Multi-task Generalization; Other tasks (test).

- [Episodic multi-task classification on DomainNet meta (test)](<https://www.sota2.com/research/sota/episodic-multi-task-classification-on-domainnet-meta-test.md>): Episodic multi-task classification; DomainNet meta (test).

- [Episodic multi-task classification on Office-Home meta (test)](<https://www.sota2.com/research/sota/episodic-multi-task-classification-on-office-home-meta-test.md>): Episodic multi-task classification; Office-Home meta (test).

- [Multi-task Language Understanding on MMLUpro (test)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlupro-test.md>): Multi-task Language Understanding; MMLUpro (test).

- [Multi-task Unlearning Interference on NYU v2](<https://www.sota2.com/research/sota/multi-task-unlearning-interference-on-nyu-v2.md>): Multi-task Unlearning Interference; NYU v2.

- [Multi-task Language Understanding on MMLU (Generic Score)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-generic-score.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Knowledge on MMLU-Pro](<https://www.sota2.com/research/sota/multi-task-knowledge-on-mmlu-pro.md>): Multi-task Knowledge; MMLU-Pro.

- [Multi-task Learning on NYUD v2 (mIoU, RMSE, odsF)](<https://www.sota2.com/research/sota/multi-task-learning-on-nyud-v2-miou-rmse-odsf.md>): Multi-task Learning; NYUD v2.

- [Multi-task Adversarial Attack Evaluation on COCO, Flickr30k, TextVQA, VQAv2, POPE](<https://www.sota2.com/research/sota/multi-task-adversarial-attack-evaluation-on-coco-flickr30k-textvqa-vqav2-pope.md>): Multi-task Adversarial Attack Evaluation; COCO, Flickr30k, TextVQA, VQAv2, POPE.

- [Multi-task Language Understanding on CMMLU](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-cmmlu.md>): Multi-task Language Understanding; CMMLU.

- [Multi-task Language Understanding on MMLU Pro (test)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-test.md>): Multi-task Language Understanding; MMLU Pro (test).

- [Multi-Task Learning on PASCAL-Context (val)](<https://www.sota2.com/research/sota/multi-task-learning-on-pascal-context-val.md>): Multi-Task Learning; PASCAL-Context (val).

- [Multi-Task Incremental Learning on MTIL Suite (11 Datasets)](<https://www.sota2.com/research/sota/multi-task-incremental-learning-on-mtil-suite-11-datasets.md>): Multi-Task Incremental Learning; MTIL Aircraft, Caltech101, CIFAR100, DTD, EuroSAT, Flowers, Food, MNIST, OxfordPet, Cars, SUN397.

- [Multi-task Image Classification on 8 Vision Tasks (Low Confidence setting, test)](<https://www.sota2.com/research/sota/multi-task-image-classification-on-8-vision-tasks-low-confidence-setting-test.md>): Multi-task Image Classification; 8 Vision Tasks Low Confidence setting (test).

- [Multi-task Image Classification on 8 Vision Tasks Norm Mismatch setting (test)](<https://www.sota2.com/research/sota/multi-task-image-classification-on-8-vision-tasks-norm-mismatch-setting-test.md>): Multi-task Image Classification; 8 Vision Tasks Norm Mismatch setting (test).

- [Multi-task Image Restoration on All-in-One IR Benchmark Average](<https://www.sota2.com/research/sota/multi-task-image-restoration-on-all-in-one-ir-benchmark-average.md>): Multi-task Image Restoration; All-in-One IR Benchmark Average.

- [Multi-task Language Understanding on MMLU (5-shot)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-5-shot.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Language Understanding on CEval](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-ceval.md>): Multi-task Language Understanding; CEval.

- [Multi-Task Learning on NYU v2 (test)](<https://www.sota2.com/research/sota/multi-task-learning-on-nyu-v2-test.md>): Multi-Task Learning; NYU v2 (test).

- [Multi-task Language Understanding on MMLU (Inference Latency)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-inference-latency.md>): Multi-task Language Understanding; MMLU.

- [Multi-task Image Classification on 20-task Model Merging Benchmark (14-task + EMNIST, CIFAR10, Food101, FashionMNIST, RenderedSST2, KMNIST)](<https://www.sota2.com/research/sota/multi-task-image-classification-on-20-task-model-merging-benchmark-14-task-plus-emnist-cifar10-food101-fashionmnist-renderedsst2-kmnist.md>): Multi-task Image Classification; 20-task Model Merging Benchmark (14-task + EMNIST, CIFAR10, Food101, FashionMNIST, RenderedSST2, KMNIST).

- [Multi-task Language Understanding and Reasoning on OpenCompass Suite (SIQA, GSM8K, WiC, HumanEval, MMLU, CSQA)](<https://www.sota2.com/research/sota/multi-task-language-understanding-and-reasoning-on-opencompass-suite-siqa-gsm8k-wic-humaneval-mmlu-csqa.md>): Multi-task Language Understanding and Reasoning; OpenCompass SIQA, GSM8K, WiC, HumanEval, MMLU, CSQA.

- [Multi-Task Reinforcement Learning (LTL Instruction Following) on Warehouse Finite Horizon](<https://www.sota2.com/research/sota/multi-task-reinforcement-learning-ltl-instruction-following-on-warehouse-finite-horizon.md>): Multi-Task Reinforcement Learning (LTL Instruction Following); Warehouse Finite Horizon.

- [Multi-task Reasoning on Average (2WikiMultiHop, MMLU, GSM8k) (in-distribution)](<https://www.sota2.com/research/sota/multi-task-reasoning-on-average-2wikimultihop-mmlu-gsm8k-in-distribution.md>): Multi-task Reasoning; Average (2WikiMultiHop, MMLU, GSM8k) (in-distribution).

- [Multi-task agent evaluation on Skill-MAS Evaluation Suite Aggregate](<https://www.sota2.com/research/sota/multi-task-agent-evaluation-on-skill-mas-evaluation-suite-aggregate.md>): Multi-task agent evaluation; Skill-MAS Evaluation Suite Aggregate.

- [Multi-task Image Classification on 8-task vision suite](<https://www.sota2.com/research/sota/multi-task-image-classification-on-8-task-vision-suite.md>): Multi-task Image Classification; 8-task vision suite.

- [Multi-task Image Classification on TA-8 (test)](<https://www.sota2.com/research/sota/multi-task-image-classification-on-ta-8-test.md>): Multi-task Image Classification; TA-8 (test).

- [Multi-task Language Understanding on MMLU (Accuracy, Exit Position)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-accuracy-exit-position.md>): Multi-task Language Understanding; MMLU.

- [Compositional Multi-tasking Summarization, Tone Adjustment, Translation (test)](<https://www.sota2.com/research/sota/compositional-multi-tasking-summarization-tone-adjustment-translation-test.md>): Compositional Multi-tasking; Compositional Multi-tasking Summarization, Tone Adjustment, Translation (test).

- [Multi-task (Building, Road, Flood) on Average](<https://www.sota2.com/research/sota/multi-task-building-road-flood-on-average.md>): Multi-task (Building, Road, Flood); Average.

- [Multi-task Language Understanding on MMLU-M](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-m.md>): Multi-task Language Understanding; MMLU-M.

- [Multi-Task Learning on NYU v2](<https://www.sota2.com/research/sota/multi-task-learning-on-nyu-v2.md>): Multi-Task Learning; NYU v2.

- [Multi-task Language Understanding on MMLU-Pro (Best and Average Accuracy)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-pro-best-and-average-accuracy.md>): Multi-task Language Understanding; MMLU-Pro.

- [Multi-Task Learning on LIBERO](<https://www.sota2.com/research/sota/multi-task-learning-on-libero.md>): Multi-Task learning; LIBERO.

- [Multi-Task Agent Generalization on Five Agent Environments Average (test)](<https://www.sota2.com/research/sota/multi-task-agent-generalization-on-five-agent-environments-average-test.md>): Multi-Task Agent Generalization; Five Agent Environments Average (test).

- [Multi-task Performance Evaluation on GPQA-Diamond, GSM8K, MATH-500, AIME’24, and IFEval Aggregate](<https://www.sota2.com/research/sota/multi-task-performance-evaluation-on-gpqa-diamond-gsm8k-math-500-aime-24-and-ifeval-aggregate.md>): Multi-task Performance Evaluation; GPQA-Diamond, GSM8K, MATH-500, AIME’24, and IFEval Aggregate.

- [Multi-task Language Model Evaluation on TÜLU Evaluation Suite (MMLU, TQA, PopQA, BBH, DROP, CHE, CHE+, GSM8K, MATH, IFEval) 2/3](<https://www.sota2.com/research/sota/multi-task-language-model-evaluation-on-t-lu-evaluation-suite-mmlu-tqa-popqa-bbh-drop-che-che-plus-gsm8k-math-ifeval-2-3.md>): Multi-task Language Model Evaluation; TÜLU Evaluation Suite (MMLU, TQA, PopQA, BBH, DROP, CHE, CHE+, GSM8K, MATH, IFEval) 2/3.

- [Multi-task Language Understanding on MMLU (153-question shuffled subset)](<https://www.sota2.com/research/sota/multi-task-language-understanding-on-mmlu-153-question-shuffled-subset.md>): Multi-task Language Understanding; MMLU 153-question shuffled subset.

- [Multi-task aerial image reasoning on HRVQA-VL (test)](<https://www.sota2.com/research/sota/multi-task-aerial-image-reasoning-on-hrvqa-vl-test.md>): Multi-task aerial image reasoning; HRVQA-VL (test).

- [Vision-Language Multi-task Evaluation on MS COCO Unified Multi-task (test)](<https://www.sota2.com/research/sota/vision-language-multi-task-evaluation-on-ms-coco-unified-multi-task-test.md>): Vision-Language Multi-task Evaluation; MS COCO Unified Multi-task (test).

- [Multi-task Reasoning on Average Out-of-domain](<https://www.sota2.com/research/sota/multi-task-reasoning-on-average-out-of-domain.md>): Multi-task Reasoning; Average Out-of-domain.

- [Long-context multi-task evaluation on LongBench-E](<https://www.sota2.com/research/sota/long-context-multi-task-evaluation-on-longbench-e.md>): Long-context multi-task evaluation; LongBench-E.

- [Multi-task Image Classification on Vision Benchmark 20-task](<https://www.sota2.com/research/sota/multi-task-image-classification-on-vision-benchmark-20-task.md>): Multi-task Image Classification; Vision Benchmark 20-task.

- [Multi-task Image Classification on 8-task vision benchmark](<https://www.sota2.com/research/sota/multi-task-image-classification-on-8-task-vision-benchmark.md>): Multi-task Image Classification; 8-task vision benchmark.

- [Inference Latency on MMLU-Pro](<https://www.sota2.com/research/sota/inference-latency-on-mmlu-pro.md>): Multi-task Language Understanding; MMLU-Pro.