# Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving

Source: [Drive-KD: Multi-Teacher Distillation for VLMs in Autonomous Driving](<https://www.sota2.com/research/paper/697c3af8a8b01e76cc8da3f3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-06T05:00:11.965Z

Authors: Weitong Lian, Zecong Tang, Haoran Li, Tianjian Gao, Yifei Wang, Zixu Wang, Lingyi Meng, Tengju Ru, Zhejun Cui, Yichen Zhu, Hangshuo Cao, Qi Kang, Tianxing Chen, Kaixuan Wang, Yu Zhang

Published: 2026-01-29T05:41:24.000Z

## Abstract

Autonomous driving is an important and safety-critical task, and recent advances in LLMs/VLMs have opened new possibilities for reasoning and planning in this domain. However, large models demand substantial GPU memory and exhibit high inference latency, while conventional supervised fine-tuning (SFT) often struggles to bridge the capability gaps of small models. To address these limitations, we propose Drive-KD, a framework that decomposes autonomous driving into a "perception-reasoning-planning" triad and transfers these capabilities via knowledge distillation. We identify layer-specific attention as the distillation signal to construct capability-specific single-teacher models that outperform baselines. Moreover, we unify these single-teacher settings into a multi-teacher distillation framework and introduce asymmetric gradient projection to mitigate cross-capability gradient conflicts. Extensive evaluations validate the generalization of our method across diverse model families and scales. Experiments show that our distilled InternVL3-1B model, with ~42 times less GPU memory and ~11.4 times higher throughput, achieves better overall performance than the pretrained 78B model from the same family on DriveBench, and surpasses GPT-5.1 on the planning dimension, providing insights toward efficient autonomous driving VLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multimodal Autonomous Driving Capability on DriveBench](<https://www.sota2.com/research/sota/multimodal-autonomous-driving-capability-on-drivebench.md>): Multimodal Autonomous Driving Capability; DriveBench; Perception Accuracy: 45.63.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.21288>)

[PDF](<https://arxiv.org/pdf/2601.21288.pdf>)