Instruction Following and Tool Use on VectraYX-Bench B1-B5 suite
34.3B1 KW ScoreVectraYX-Nano v2
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VectraYX-Nano v2Params=42M, Seed Configuration=orig. seed2026.05 | 34.3 | 19 | 2.9 | 0 | 0 | 70 | |
| VectraYX-Pro 3BParams=3.2B, Backbone=Qwen2.5-3B, Fine-tuning Strategy=LoRA-642026.05 | 34.1 | 69.5 | 68.6 | 8.6 | 60 | 80 | |
| VectraYX-Pro 7BParams=7B, Backbone=Qwen2.5-7B, Fine-tuning Strategy=QLoRA-322026.05 | 33.5 | 81.5 | 68.6 | 11.4 | 88 | 80 | |
| SmolLM2-135MParams=135M, Strategy=LoRA-322026.05 | 33.4 | 22.5 | 14.3 | 0 | 16 | 80 | |
| VectraYX-Base 260MParams=260M, Seed Configuration=1 seed2026.05 | 32.5 | 22 | 11.4 | 0 | 0 | 80 | |
| VectraYX-Nano v2Params=42M, Seed Configuration=N=42026.05 | 22.8 | 19.6 | 2.9 | 0 | 0 | 77.5 | |
| Base 260MParams=260M, Fine-tuning Strategy=LoRA, Training Corpus=mini, 1:21, Seed Configuration=N=42026.05 | 1.9 | 20.3 | 2.9 | 0 | 44.5 | 60 | |
| Nano 42MParams=42M, Fine-tuning Strategy=LoRA, Training Corpus=mini, 1:21, Seed Configuration=N=42026.05 | 1.1 | 20.1 | 2.1 | 0 | 14.5 | 57.5 | |
| SmolLM2-135MParams=135M, Strategy=base, zero-shot2026.05 | 0.1 | 19.5 | 5.7 | 0 | 0 | 80 |