# Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents Source: [Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents]() Collection index: [llms.txt]() Data updated: 2026-06-19T05:00:05.274Z Authors: Caleb Chang, Davin Win Kyi, Natasha Jaques, Karen Leung Published: 2026-06-16T23:11:14.000Z ## Abstract Humans often acquire new skills by observing others, since observed behaviors implicitly reveal how to act in an environment. However, observations drawn from a heterogeneous population introduce conflicting behavioral signals, making it difficult to determine which behaviors are worth imitating. We address this challenge with General Reward Inference and Disentanglement (GRID), a social learning method that extracts universally useful behaviors from a heterogeneous population of demonstrators pursuing different goals. GRID decomposes per-agent reward functions into a general reward, capturing behaviors shared across all agents, and specific rewards, capturing individual preferences and objectives. Training exclusively on the general reward provides a new paradigm of generalist pretraining. It yields a generalist agent that internalizes universal environmental competencies, such as safety and basic task proficiency, without the mode-averaging bias that afflicts standard learning from demonstration techniques. This generalist serves as a superior prior for fine-tuning to downstream tasks, including preferences unseen during training. Experiments across a synthetic basis function decomposition, multi-agent Craftax, and a continuous autonomous driving simulator (Highway-Env) confirm that GRID successfully disentangles reward structure in a semantically meaningful way, outperforms standard learning from demonstration baselines, and enables more efficient and stable specialization. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Autonomous Highway Navigation on Highway-Env Generalist](): Autonomous Highway Navigation; Highway-Env Generalist; Reward: 34.7. - [Autonomous Driving on Highway-Env (Specialist Evaluation)](): Autonomous Driving; Highway-Env (Specialist Evaluation); Reward: 34.9. - [Preference-conditioned Autonomous Highway Navigation on Highway-Env Lane 3 Specialist](): Preference-conditioned Autonomous Highway Navigation; Highway-Env Lane 3 Specialist; Reward: 34.9. ## Original sources and implementation [Original paper]() [PDF]()