Risk Factor Reasoning on JD.com Risk Investigation 1.0 (evaluation set (1,000 samples))
63FARVanilla JoyAI-10B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Vanilla JoyAI-10BModel=JoyAI-10B, Inference Mode=Zero-shot2025.10 | 63 | 0.86 | 41 | |
| Vanilla Qwen3-8BModel=Qwen3-8B, Inference Mode=Zero-shot2025.10 | 65 | 0.79 | 45 | |
| Vanilla DeepSeek-R1Model=DeepSeek-R1, Inference Mode=Zero-shot2025.10 | 75 | 0.73 | 58 | |
| Vanilla JoyAI-750BModel=JoyAI-750B, Inference Mode=Zero-shot2025.10 | 78 | 1.03 | 56 | |
| Sherlock Qwen3-8BModel=Qwen3-8B, Inference Mode=STRO-fine-tuned with domain KB and R&R module2025.10 | 89 | 4.26 | 64 | |
| Sherlock JoyAI-10BModel=JoyAI-10B, Inference Mode=STRO-fine-tuned with domain KB and R&R module2025.10 | 92 | 4.34 | 64 |