Mean-field Chaos Diffusion Models

About

In this paper, we introduce a new class of score-based generative models (SGMs) designed to handle high-cardinality data distributions by leveraging concepts from mean-field theory. We present mean-field chaos diffusion models (MF-CDMs), which address the curse of dimensionality inherent in high-cardinality data by utilizing the propagation of chaos property of interacting particles. By treating high-cardinality data as a large stochastic system of interacting particles, we develop a novel score-matching method for infinite-dimensional chaotic particle systems and propose an approximation scheme that employs a subdivision strategy for efficient training. Our theoretical and empirical results demonstrate the scalability and effectiveness of MF-CDMs for managing large high-cardinality data structures, such as 3D point clouds.

Sungwoo Park, Dongjun Kim, Ahmed Alaa · 2024

Related benchmarks

Task	Dataset	Result
Offline Reinforcement Learning	Gaussian Squeeze N=1000 (Expert)	Normalized Return87.1	10
Multi-Agent Reinforcement Learning	Ising Model Medium N=10,000	Normalized Return85.4	10
Multi-Agent Reinforcement Learning	Ising Model N=10,000 (Mixed)	Normalized Return77.5	10
Multi-Agent Reinforcement Learning	Battle N=10,000 (Expert)	Normalized Return80.2	10
Multi-Agent Reinforcement Learning	Battle Medium N=10,000	Normalized Return71.2	10
Multi-Agent Reinforcement Learning	Battle Med-Replay N=10,000	Normalized Return66.3	10
Multi-Agent Reinforcement Learning	Battle N=10,000 (Mixed)	Normalized Return61.4	10
Multi-Agent Reinforcement Learning	Gaussian Squeeze Expert N=10,000	Normalized Return88.5	10
Multi-Agent Reinforcement Learning	Gaussian Squeeze Medium N=10,000	Normalized Return81.2	10
Multi-Agent Reinforcement Learning	Gaussian Squeeze Med-Replay N=10,000	Normalized Return (%)76.3	10