# Revisiting the Learning Objectives of Vision-Language Reward Models Source: [Revisiting the Learning Objectives of Vision-Language Reward Models]() Collection index: [llms.txt]() Data updated: 2025-12-26T05:00:05.118Z Authors: Simon Roy, Samuel Barbeau, Giovanni Beltrame, Christian Desrosiers, Nicolas Thome Published: 2025-12-20T19:50:36.000Z ## Abstract Learning generalizable reward functions is a core challenge in embodied intelligence. Recent work leverages contrastive vision language models (VLMs) to obtain dense, domain-agnostic rewards without human supervision. These methods adapt VLMs into reward models through increasingly complex learning objectives, yet meaningful comparison remains difficult due to differences in training data, architectures, and evaluation settings. In this work, we isolate the impact of the learning objective by evaluating recent VLM-based reward models under a unified framework with identical backbones, finetuning data, and evaluation environments. Using Meta-World tasks, we assess modeling accuracy by measuring consistency with ground truth reward and correlation with expert progress. Remarkably, we show that a simple triplet loss outperforms state-of-the-art methods, suggesting that much of the improvements in recent approaches could be attributed to differences in data and architectures. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Open Door on Meta-World](): Open Door; Meta-World; VOC Score: 62.64. - [Button Press on Meta-World](): Button press; Meta-World; VOC Score: 95.47. - [Open Drawer on Meta-World](): open drawer; Meta-World; VOC Score: 90.17. - [Reward Modeling on Meta-World Button press](): Reward Modeling; Meta-World Button press; Prediction Accuracy: 76.44. - [Reward Modeling on Meta-World Open drawer](): Reward Modeling; Meta-World Open drawer; Prediction Accuracy: 69.01. - [Reward Modeling on Meta-World Open door](): Reward Modeling; Meta-World Open door; Prediction Accuracy: 65.02. ## Original sources and implementation [Original paper]() [PDF]()