Skill Composition on SKILL-MIX held-out (test)
97Full Marks Ratio (k=1)Mistral-7B-Instruct-v0.2
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Mistral-7B-Instruct-v0.2Fine-tuning Data=D_SKILL-MIX (1), Grading Model=GPT-42024.09 | 97 | 97 | 64 | 81 | 20 | 68 | 5 | 57 | 4 | 60 | |
| Mistral-7B-Instruct-v0.2Fine-tuning Data=D_SKILL-MIX (1, 2), Grading Model=GPT-42024.09 | 97 | 97 | 85 | 93 | 37 | 74 | 17 | 74 | 10 | 70 | |
| Mistral-7B-Instruct-v0.2Fine-tuning Data=D_SKILL-MIX (1, 2, 3), Grading Model=GPT-42024.09 | 97 | 97 | 86 | 93 | 49 | 82 | 26 | 76 | 13 | 74 | |
| LLaMA-2-13B-ChatFine-tuning dataset=D_SKILL-MIX (1, 2, 3)2024.09 | 96 | 96 | 78 | 88 | 37 | 75 | 9 | 69 | 2 | 60 | |
| LLaMA-2-13B-ChatFine-tuning dataset=D_SKILL-MIX (1, 2)2024.09 | 95 | 95 | 75 | 87 | 25 | 68 | 5 | 60 | 2 | 56 | |
| LLaMA-2-13B-ChatFine-tuning dataset=D_SKILL-MIX (1)2024.09 | 87 | 87 | 43 | 70 | 5 | 54 | 1 | 49 | 0 | 44 | |
| Mistral-7B-Instruct-v0.2Fine-tuning Data=None, Grading Model=GPT-42024.09 | 85 | 85 | 48 | 73 | 8 | 56 | 1 | 42 | 1 | 39 | |
| LLaMA-2-13B-ChatFine-tuning dataset=None2024.09 | 46 | 46 | 28 | 50 | 4 | 42 | 1 | 39 | 0 | 43 |