Natural Language Processing tasks on MTI BENCH
4,320M.S. ScoreGPT-4
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4Inference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 4,320 | 4,250 | 4,290 | |
| GPT-4Inference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 3,330 | 3,100 | 3,220 | |
| GPT-4Inference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 2,580 | 3,570 | 3,080 | |
| GPT-3.5-TURBOInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 2,150 | 2,620 | 2,390 | |
| GPT-3.5-TURBOInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 1,890 | 2,370 | 2,130 | |
| GPT-3.5-TURBOInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 1,810 | 1,910 | 1,860 | |
| LLAMA-2-CHAT-70BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 1,600 | 2,000 | 1,800 | |
| TULU-30BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 820 | 540 | 680 | |
| LLAMA-2-CHAT-70BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 800 | 940 | 870 | |
| LLAMA-2-CHAT-70BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 740 | 830 | 790 | |
| VICUNA-13BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 730 | 930 | 830 | |
| VICUNA-13BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 650 | 1,160 | 910 | |
| TULU-65BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 560 | 710 | 640 | |
| LLAMA-2-CHAT-7BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 550 | 790 | 670 | |
| VICUNA-7BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 390 | 480 | 440 | |
| TULU-13BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 290 | 260 | 280 | |
| LLAMA-2-CHAT-7BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 280 | 440 | 360 | |
| TULU-65BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 240 | 300 | 270 | |
| LLAMA-2-CHAT-13BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 240 | 420 | 330 | |
| VICUNA-13BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 240 | 190 | 220 | |
| VICUNA-7BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 220 | 230 | 230 | |
| TULU-13BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 210 | 380 | 300 | |
| TULU-30BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 200 | 100 | 150 | |
| TULU-30BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 150 | 440 | 300 | |
| TULU-65BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 140 | 460 | 300 | |
| VICUNA-7BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 130 | 150 | 140 | |
| LLAMA-2-CHAT-7BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 100 | 0 | 50 | |
| LLAMA-2-CHAT-13BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 100 | 300 | 200 | |
| TULU-7BInference Strategy=Single-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 90 | 90 | 90 | |
| TULU-7BInference Strategy=Multi-Task Inference, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 40 | 150 | 100 | |
| TULU-7BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 0 | 0 | 0 | |
| TULU-13BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 0 | 0 | 0 | |
| LLAMA-2-CHAT-13BInference Strategy=Batch Prompting, Evaluation Prompting=one-shot, Reasoning Method=Chain-of-Thought2024.02 | 0 | 0 | 0 |