BLPR: Robust License Plate Recognition under Viewpoint and Illumination Variations via Confidence-Driven VLM Fallback
About
Robust license plate recognition in unconstrained environments remains a significant challenge, particularly in underrepresented regions with limited data availability and unique visual characteristics, such as Bolivia. Recognition accuracy in real-world conditions is often degraded by illumination changes and viewpoint distortion. To address these challenges, we introduce BLPR, a deep learning-based License Plate Detection and Recognition (LPDR) framework designed for Bolivian license plates. BLPR adaptively applies geometric rectification, illumination correction, and VLM-assisted fallback based on image-condition and confidence cues. The proposed system uses a YOLO-based detector pretrained on synthetic data generated in Blender to simulate extreme perspectives and lighting conditions, and is fine-tuned on street-level data collected in La Paz, Bolivia. Detected plates are processed by a YOLO-based character recognizer, while a lightweight vision-language model (Gemma3 4B) is selectively triggered in ambiguous cases as a confidence-driven fallback mechanism. We also introduce the first publicly available Bolivian LPDR dataset for academic research, supporting evaluation under diverse viewpoint and illumination conditions. The system achieves a character-level recognition accuracy of 89.6% on real-world data, demonstrating its effectiveness for deployment in challenging urban environments.