A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model

Jianwen Xie, Yaxuan Zhu, Jun Li, and Ping Li

Cognitive Computing Lab, Baidu Research, USA

Abstract

This paper studies the cooperative learning of two generative flow models, in which the two models are iteratively updated based on the jointly synthesized examples. The first flow model is a normalizing flow that transforms an initial simple density into a target density by applying a sequence of invertible transformations. The second flow model is a Langevin flow that runs finite steps of gradient-based MCMC toward an energy-based model. We start from proposing a generative framework that trains an energy-based model with a normalizing flow as an amortized sampler to initialize the MCMC chains of the energy-based model. In each learning iteration, we generate synthesized examples by using a normalizing flow initialization followed by a short-run Langevin flow revision toward the current energy-based model. Then we treat the synthesized examples as fair samples from the energy-based model and update the model parameters with the maximum likelihood learning gradient, while the normalizing flow directly learns from the synthesized examples by maximizing the tractable likelihood. Under the short-run non-mixing MCMC scenario, the estimation of the energy-based model is shown to follow the perturbation of maximum likelihood, and the short-run Langevin flow and the normalizing flow form a two-flow generator that we call CoopFlow. We provide an understating of the CoopFlow algorithm by information geometry and show that it is a valid generator as it converges to a moment matching estimator. We demonstrate that the trained CoopFlow is capable of synthesizing realistic images, reconstructing images, and interpolating between images.

Results

Paper

The ICLR conference paper can be downloaded here.

The ICLR tex file can be downloaded here.

The poster can be downloaded here.

The slide can be downloaded here.

Code and Data

Related Work

(1) Energy-based model/ Langevin Flow

• Jianwen Xie*, Yang Lu*, Song-Chun Zhu, Ying Nian Wu. "A Theory of Generative ConvNet." International Conference on Machine Learning. 2016. (*equal contribution)
• Erik Nijkamp, Mitch Hill, Song-Chun Zhu, Ying Nian Wu. "Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model. " Neural Information Processing Systems. 2020.

(2) Generative Cooperative Learning

• Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu. "Cooperative Training of Descriptor and Generator Networks." IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018.
• Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu. "Cooperative Learning of Energy-Based Model and Latent Variable Model via MCMC Teaching." The Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
• Jianwen Xie*, Zilong Zheng*, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu. "Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning. " IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021.

(3) Normalizing Flow

• Diederik P. Kingma, Prafulla Dhariwal. "Glow: Generative Flow with Invertible 1x1 Convolutions " Neural Information Processing Systems. 2018.
• Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, Pieter Abbeel. "Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design. " International Conference on Machine Learning. 2019.

Top