Requirements Engineering on EDA requirements engineering
70.07F1 Score (Ref)Qwen2.5-7B-Instruct + LoRA (w/ all)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct + LoRA (w/ all)Model=Qwen2.5-7B-Instruct, LoRA Adaptation=true, Context Strategy=w/ all2026.03 | 70.07 | 68.65 | 71.55 | 42.29 | 76.75 | 55.1 | 3.75 | 41.98 | |
| Qwen2.5-7B-Instruct + LoRA (w/ all)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/ all contexts2026.03 | 69.56 | 67.12 | 72.15 | 43.54 | 75.75 | 57.47 | — | 42.53 | |
| PA-RAG (w/ rel. + irr.)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=PA-RAG (relevant + irrelevant)2026.03 | 68.79 | 68.48 | 69.11 | 24.26 | 75.29 | 32.22 | 41.31 | 38.14 | |
| Qwen2.5-7B-Instruct + LoRA (w/ all)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/ all contexts2026.03 | 68.3 | 66.36 | 70.36 | 28.21 | 73.56 | 38.35 | 36.56 | 36.1 | |
| Qwen2.5-7B-Instruct + LoRA (w/o context)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/o context2026.03 | 68.26 | 68.51 | 68.02 | 43.52 | 76.69 | 56.75 | — | 43.25 | |
| Qwen2.5-7B-Instruct + LoRA (PA-RAG)Model=Qwen2.5-7B-Instruct, LoRA Adaptation=true, Context Strategy=PA-RAG (w/ rel. + irr)2026.03 | 67.6 | 68.51 | 66.72 | 40.75 | 78.6 | 51.84 | 2.95 | 45.76 | |
| Qwen2.5-72B-InstructModel=Qwen2.5-72B-Instruct, Adaptation=None2026.03 | 67.59 | 53.61 | 91.42 | 25.25 | 50.61 | 49.89 | 27.16 | 31.39 | |
| PA-RAG (w/ rel. + irr.)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=PA-RAG (relevant + irrelevant)2026.03 | 67.47 | 66.83 | 68.13 | 41.59 | 76.34 | 54.47 | — | 45.53 | |
| Qwen2.5-7B-Instruct + LoRA (RAFT)Model=Qwen2.5-7B-Instruct, LoRA Adaptation=true, Context Strategy=RAFT (w/ noisy + irr)2026.03 | 67.31 | 68.48 | 66.18 | 45 | 79.62 | 56.52 | 3.26 | 40.56 | |
| RAFT (w/ noisy + irr.)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=RAFT (noisy + irrelevant)2026.03 | 66.59 | 66.79 | 66.4 | 23.43 | 73.71 | 31.79 | 41.13 | 39.92 | |
| Qwen2.5-7B-Instruct + LoRA (w/ relevant)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/ relevant2026.03 | 66.12 | 66.07 | 66.18 | 26.61 | 74.85 | 35.56 | 37.4 | 39.4 | |
| Qwen2.5-7B-Instruct + LoRA (w/o context)Model=Qwen2.5-7B-Instruct, LoRA Adaptation=true, Context Strategy=w/o context2026.03 | 65.78 | 66.14 | 65.42 | 40.45 | 77.5 | 52.2 | 6.15 | 42.26 | |
| Qwen2.5-72B-InstructModel=Qwen2.5-72B-Instruct, Adaptation=None2026.03 | 65.75 | 50.21 | 91.75 | 33.54 | 51.48 | 65.15 | — | 34.85 | |
| Qwen2.5-7B-Instruct + LoRA (w/ relevant)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/ relevant2026.03 | 64.86 | 62.9 | 66.94 | 41.33 | 73.24 | 56.43 | — | 43.57 | |
| RAFT (w/ noisy + irr.)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=RAFT (noisy + irrelevant)2026.03 | 64.8 | 64.63 | 64.98 | 39.85 | 72.92 | 54.64 | — | 45.36 | |
| Qwen2.5-7B-Instruct + LoRA (w/o context)Model=Qwen2.5-7B-Instruct, Adaptation=LoRA, Fine-tuning Strategy=w/o context2026.03 | 63.95 | 64 | 63.9 | 28.06 | 74.14 | 37.85 | 36.43 | 35.83 | |
| Qwen2.5-72B-InstructModel=Qwen2.5-72B-Instruct, LoRA Adaptation=false, Context Strategy=N/A2026.03 | 62.9 | 48.34 | 90.01 | 29.74 | 49.4 | 60.2 | 7.67 | 32.86 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct, Adaptation=None2026.03 | 62.59 | 54.09 | 74.25 | 21.98 | 51.48 | 42.7 | 30.77 | 37.01 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct, Adaptation=None2026.03 | 61.28 | 51.64 | 75.33 | 32.86 | 54.07 | 60.77 | — | 39.23 | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct, LoRA Adaptation=false, Context Strategy=N/A2026.03 | 59.14 | 50.63 | 71.09 | 29.61 | 57.71 | 51.31 | 9.46 | 39.88 | |
| Qwen2.5-7B-Instruct + LoRA (w/ relevant)Model=Qwen2.5-7B-Instruct, LoRA Adaptation=true, Context Strategy=w/ relevant2026.03 | 31.68 | 31.49 | 31.87 | 18.03 | 43.58 | 41.36 | 38.79 | 20.82 |