Benchmarks

Task NameDataset NameSOTA ResultTrendResults

Multi-hop Question Answering

Average MuSiQue, 2wiki, HotpotQA

43.3EM

Image Deraining

Average Test100, Test1200, Rain100H, Rain100L (test)

35.28PSNR

Entity Matching

Average (Febrl4, F.-Zagat, Bikes, eBooks, Movies)

91.2F1 Score

Arithmetic Reasoning

Average Arithmetic Reasoning Tasks

84.3Accuracy

Code Generation

Average dataset-weighted (test)

34.1Sig. Score

General Language Understanding

Average

72.93Average Accuracy

Multi-task (Building, Road, Flood)

Average

69.18mIoU

Mathematical Reasoning

Average AIME24, AIME25, AMC23, MATH500

55.4Pass@1

Overall Performance

Average across all benchmarks

70.3Average Score

Image Deraining

Average across Deraining Datasets (test)

34.24PSNR

Anomaly Detection

Average

76.6AUROC

Video Transmission

Average All Datasets

109.7BD-CBR (LPIPS)

Reasoning

Average (MATH500, AMC23, AIME24, AIME25, GPQA-d)

87.2Accuracy

Video Understanding

Average VideoMME, LongVideoBench, MLVU

55.9Score

Question Answering

Average (NQ, TabMWP, HotpotQA, TriviaQA, TriState)

38.42EM

Mathematical Reasoning

Average (MATH-OAI, Minerva Math, OlympiadBench, AIME24, AMC23)

32.69Average Score @16

Question Answering

Average (TriviaQA, HotpotQA, 2Wiki, Musique, Bamboogle)

49.12EM Accuracy

Mathematical Reasoning

Average Six Benchmarks

68.66Accuracy

Classification

Average

87.41Base Score

Image Classification

Average

90.2Accuracy

Calibration

Average StrategyQA, HotpotQA, NQ, Bamboogle

0.264ECE

Multi-hop Question Answering

Average (2WikiMQA, Bamboogle, Frames, MuSiQue)

54.8Accuracy

Instruction Following Evaluation

Average (Vicuna, Self-instruct, Dolly, BPO) (test)

22Delta Win Rate (ΔWR)

Aggregate NLP Tasks

Average (Emotion, Irony, Stance, MRPC, RTE)

2.93Delta 1

AI-Generated Image Detection

Average Post-processed

97.1AUC