• AI Research Scientist

  • E-mail: jianwen at ucla dot edu
  • Curriculum Vitae (updated 2024.12)

About

  • I am a research scientist in artificial intelligence, mainly interested in Generative AI, Representation learning, Computer vision, Computational neuroscience, and AI for science. I received my Ph.D. in Statistics at University of California, Los Angeles (UCLA), under the supervision of Prof. Ying Nian Wu and Prof. Song-Chun Zhu.

News

  • [02/2025] I will serve as a Senior Area Chair for NeurIPS 2025
  • [11/2024] I am honored to receive IEEE TNNLS Outstanding Associate Editor Award.
  • [11/2024] I will serve as an Area Chair for ICML 2025

Education

Academic Service

Associate Editor:

  • IEEE Transactions on Neural Networks and Learning Systems (TNNLS) (2024 - )
  • IEEE Transactions on Image Processing (TIP) (2024 - )
  • Pattern Recognition (PR) (2024 - )
  • Computer Vision and Image Understanding (CVIU) (2025 - )
  • Journal of Artificial Intelligence Research (JAIR) (2025 - )
  • Senior Area Chair:

  • Annual Conference on Neural Information Processing Systems (NeurIPS) - 2024, 2025
  • Area Chair:

  • Annual Conference on Neural Information Processing Systems (NeurIPS) - 2022, 2023
  • International Conference on Machine Learning (ICML) - 2022, 2023, 2024, 2025
  • International Conference on Learning Representations (ICLR) - 2022, 2023, 2024, 2025
  • IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) - 2021, 2023, 2024, 2025
  • European Conference on Computer Vision (ECCV) - 2024
  • IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) - 2025
  • ACM Multimedia - 2024, 2025
  • Senior Program Committee:

  • Annual AAAI Conference on Artificial Intelligence (AAAI) - 2023, 2024, 2025
  • International Joint Conference on Artificial Intelligence (IJCAI) - 2023, 2024, 2025
  • Current Research Themes

    • (Theme 1) Deep Energy-Based Model
    • Energy-Based Generative ConvNet for Images and Theoretical Foundation (ICML 2016 by Xie et al. )
    • Energy-Based Generative Spatial-Temporal ConvNet for Videos (CVPR 2017, TPAMI 2020 by Xie et al. )
    • Energy-Based Generative VoxelNet for 3D Volumetric Shapes (CVPR 2018, TPAMI 2020 by Xie et al. )
    • Energy-Based Generative PointNet for Unordered 3D Point Clouds (CVPR 2021 by Xie et al. )
    • Energy-Based Patchwise Generative ConvNet for Deep Internal Learning (CVPR 2021 by Zheng et al. )
    • Energy-Based Inverse Optimal Control for Trajectory Prediction (TNNLS 2022 by Xu et al. )
    • Energy-Based Generative ConvNet for High-Fidelity Image Synthesis (ICLR 2021 by Zhao et al. )

    Examples of energy-based synthesis of textures, objects, faces, scenes, videos, 3D voxels, unordered point clouds, trajectories, and unpaired image-to-image tanslation.


    • (Theme 2) Generative Cooperative Learning
    • Cooperative Networks (CoopNets) = EBM + Generator (AAAI 2018, TPAMI 2018 by Xie et al.)
    • Conditional Cooperative Networks (CCoopNets) = Conditional EBM + Conditional Generator (TPAMI 2021 by Xie et al.)
    • Cycle-Consistent Cooperative Networks (CycleCoopNets) + Alternating MCMC Teaching (AAAI 2021 by Xie et al.)
    • Variational MCMC Teaching = EBM + VAE (AAAI 2021 by Xie et al.)
    • Generative Cooperative Saliency Prediction (SalCoopNets) (AAAI 2022 by Zhang et al.)
    • Cooperative Flow = Langevin Flow + Normalizing Flow (CoopFlow) (ICLR 2022 by Xie et al.)
    • Initializing Adversarial Learning by Cooperative Learning (CoopInit) (AAAI 2023 by Zhao et al.)
    • Cooperative Learning for Supervised Image Hashing (CoopHash) (arXiv 2022 by Doan et al.)
    • Cooperative Diffusion Recovery Likelihood (ICLR 2024 by Zhu et al.)

    Examples of cooperative learning on image synthesis, supervised conditional learning, unpaired image-to-image translation, saliency prediction, and sequence-to-sequence translation.


    • (Theme 3) Deep Latent Variable Model
    • Dynamic Generator + Alternating Back-propagation Through Time (AAAI 2019 by Xie et al.)
    • Motion-Based Generator + Alternating Back-propagation Through Time (AAAI 2020 by Xie et al.)
    • Feature-to-Feature Translator + Conditional Alternating Back-propagation (ICCV 2019 by Zhu et al.)
    • Noise-Aware Encoder-Decoder + Conditional Alternating Back-propagation (ECCV 2020 by Zhang et al.)
    • Short-Run MCMC Inference with Optimal Transport Correction (CVPR 2021 by An et al.)
    • Generative Vision Transformer with Energy-Based Prior (NeurIPS 2021, TPAMI 2023 by Zhang et al.)
    • Generator with Latent Space Flow-based Prior + Langevin Dynamics Inference (AAAI 2023 by Xie et al.)
    • Generative Radiance Field with Energy-Based Prior (AISTATS 2023 by Zhu et al.)

    • (Theme 4) Vector-Matrix Representational Model
    • Grid Cell Model (ICLR 2019 by Gao et al.)
    • Grid Cell Model with Head Direction (NeurIPS 2021 by Gao et al.)
    • V1 Cell Model (AAAI 2022 by Gao et al.)
    • While vector can be used to represent entities (nouns), matrix can be used to represent actions, changes, and relations (verbs) that form groups. Group representation is a central topic in modern mathematics and physics.


    • (Theme 5) FRAME (Filters, Random Fields and Maximum Entropy) Models
    • Sparse Inhomogeneous FRAME Model (CVPR 2014, IJCV 2014 by Xie et al.)
    • Gnerative Boosting for Sparse FRAME Model (ACHA 2015 by Xie et al.)
    • Hierarchical Sparse FRAME Model (CVPR 2017 by Xie et al.)
    • Deep FRAME Model (AMSA 2018 by Wu et al.)
    • Examples of image synthesis from the FRAME models via MCMC.

    Tutorial

    Phd Dissertation

    Selected Papers

      Latent Energy-Based Odyssey: Black-Box Optimization via Expanded Exploration in the Energy-Based Latent Space [PDF]
    • --Peiyu Yu, Dinghuai Zhang, Hengzhi He, Xiaojian Ma, Ruiyao Miao, Yifan Lu, Yasi Zhang, Deqian Kong, Ruiqi Gao, Jianwen Xie, Guang Cheng, Ying Nian Wu
    • --arXiv 2024

    • Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translation [PDF]
    • --Weinan Song, Yaxuan Zhu, Lei He, Ying Nian Wu, Jianwen Xie
    • --arXiv 2024

    • CoopHash: Cooperative Learning of Multipurpose Descriptor and Contrastive Pair Generator via Variational MCMC Teaching for Supervised Image Hashing [PDF]
    • --Khoa Doan, Jianwen Xie, Yaxuan Zhu, Yang Zhao, Ping Li
    • --arXiv 2024

    • Latent Space Energy-based Neural ODEs [PDF]
    • --Sheng Cheng *, Deqian Kong *, Jianwen Xie, Kookjin Lee, Ying Nian Wu, Yezhou Yang
    • --Transactions on Machine Learning Research (TMLR) 2025

    • Molecule Design by Latent Prompt Transformer [PDF]
    • --Deqian Kong, Yuhao Huang, Jianwen Xie, Edouardo Honig, Ming Xu, Shuanghong Xue, Pei Lin, Sanping Zhou. Sheng Zhong, Nanning Zheng, Ying Nian Wu
    • --Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS) 2024
    • (an expanded version of the NeurIPS-23 workshop paper [PDF] on AI for science)

    • Latent Plan Transformer: Planning as Latent Variable Inference [PDF]
    • --Deqian Kong, Dehong Xu, Minglu Zhao, Bo Pang, Jianwen Xie, Andrew Lizarraga, Yuhao Huang, Sirui Xie, Ying Nian Wu
    • --Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS) 2024

    • Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [PDF]
    • --Yaxuan Zhu, Jianwen Xie, Ying Nian Wu, Ruiqi Gao
    • --The Twelfth International Conference on Learning Representations (ICLR) 2024

    • An Energy-Based Prior for Generative Saliency [PDF]
    • --Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2023
    • (an expanded version of the NeurIPS-21 paper for generative saliency)

    • Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representation [PDF] [code]
    • --Yaxuan Zhu, Jianwen Xie, Ping Li
    • --The Twenty-Sixth International Conference on Artificial Intelligence and Statistics (AISTATS) 2023

    • A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference [PDF] [project page]
    • --Jianwen Xie, Yaxuan Zhu, Yifei Xu, Dingcheng Li, Ping Li
    • --The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI) 2023

    • CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning [PDF]
    • --Yang Zhao, Jianwen Xie, Ping Li
    • --The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI) 2023

    • Energy-Based Continuous Inverse Optimal Control [PDF] [project page]
    • --Yifei Xu, Jianwen Xie, Tianyang Zhao, Chris Baker, Yibiao Zhao, Ying Nian Wu
    • --IEEE Transactions on Neural Networks and Learning Systems (TNNLS) 2022
    • (an expanded version of the NeurIPS 2020 workshop paper on machine learning for autonomous driving)

    • Blind Image Super-Resolution with Elaborate Degradation Modeling on Noise and Kernel [PDF]
    • --Zongsheng Yue, Qian Zhao, Jianwen Xie, Lei Zhang, Deyu Meng, Kwan-Yee K. Wong
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022

    • A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model [PDF] [project page]
    • --Jianwen Xie, Yaxuan Zhu, Jun Li, Ping Li
    • --Tenth International Conference on Learning Representations (ICLR) 2022

    • Learning V1 Simple Cells with Vector Representations of Local Contents and Matrix Representations of Local Motions [PDF] [project page]
    • --Ruiqi Gao, Jianwen Xie, Siyuan Huang, Yufan Ren, Song-Chun Zhu, Ying Nian Wu
    • --The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI) 2022

    • Energy-Based Generative Cooperative Saliency Prediction [PDF]
    • --Jing zhang, Jianwen Xie, Zilong Zheng, Nick Barnes
    • --The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI) 2022

    • On Path Integration of Grid Cells: Group Representation and Isotropic Scaling [PDF]
    • --Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, Ying Nian Wu
    • --Thirty-fifth Annual Conference on Neural Information Processing Systems (NeurIPS) 2021

    • Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction [PDF]
    • --Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li
    • --Thirty-fifth Annual Conference on Neural Information Processing Systems (NeurIPS) 2021

    • Monocular 3D Pose Estimation via Pose Grammar and Data Augmentation [PDF]
    • --Yuanlu Xu, Wenguan Wang, Tengyu Liu, Xiaobai Liu, Jianwen Xie, Song-Chun Zhu
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021

    • Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning [PDF] [project page]
    • --Jianwen Xie *, Zilong Zheng *, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021

    • Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning [PDF] [project page]
    • --Zilong Zheng, Jianwen Xie, Ping Li
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021 (Oral)

    • Learning Deep Latent Variable Models by Short-Run MCMC Inference with Optimal Transport Correction [PDF]
    • --Dongsheng An, Jianwen Xie, Ping Li
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021

    • Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification [PDF] [project page]
    • --Jianwen Xie *, Yifei Xu *, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021

    • Semi-Supervised Video Deraining with Dynamical Rain Generator [PDF]
    • --Zongsheng Yue, Jianwen Xie, Qian Zhao, Deyu Meng
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021

    • Learning Energy-Based Generative Models via Coarse-to-Fine Expanding and Sampling [PDF] [code]
    • --Yang Zhao, Jianwen Xie, Ping Li
    • --Ninth International Conference on Learning Representations (ICLR) 2021

    • Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation [PDF] [project page]
    • --Jianwen Xie *, Zilong Zheng *, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI) 2021

    • Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler [PDF] [project page]
    • --Jianwen Xie, Zilong Zheng, Ping Li
    • --The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI) 2021

    • Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis [PDF] [project page]
    • --Jianwen Xie *, Zilong Zheng *, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2020
    • (an expanded version of the CVPR-18 paper for energy-based 3D descriptor network)

    • Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection [PDF]
    • --Jing Zhang, Jianwen Xie, Nick Barnes
    • --European Conference on Computer Vision (ECCV) 2020

    • Motion-Based Generator Model: Unsupervised Disentanglement of Appearance, Trackable and Intrackable Motions in Dynamic Patterns [PDF][project page]
    • --Jianwen Xie *, Ruiqi Gao *, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI) 2020 (Oral)

    • Representation Learning: A Statistical Perspective [PDF][Tex]
    • --Jianwen Xie, Ruiqi Gao, Erik Nijkamp, Song-Chun Zhu, Ying Nian Wu
    • --Annual Review of Statistics and Its Application (ARSIA) 2020
    • (an review article on the subject of Representation Learning to Annual Reviews, whose Impact Factor rankings are among the highest in scholarly publishing)

    • Learning Energy-based Spatial-Temporal Generative ConvNet for Dynamic Patterns [PDF] [project page]
    • --Jianwen Xie, Song-Chun Zhu, Ying Nian Wu
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2019
    • (an expanded version of the CVPR-17 paper for spatial-temporal generative ConvNet)

    • Revisiting Video Saliency Prediction in the Deep Learning Era [PDF]
    • --Wenguan Wang, Jianbing Shen, Jianwen Xie, Ming-Ming Cheng, Haibin Ling, Ali Borji
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2019

    • Semantic-Guided Multi-Attention Localization for Zero-Shot Learning [PDF]
    • --Yizhe Zhu, Jianwen Xie, Zhiqiang Tang, Xi Peng, Ahmed Elgammal
    • --Thirty-third Conference on Neural Information Processing Systems (NeurIPS) 2019

    • Learning Feature-to-Feature Translator by Alternating Back-Propagation for Generative Zero-Shot Learning [PDF]
    • --Yizhe Zhu, Jianwen Xie, Bingchen Liu, Ahmed Elgammal
    • --International Conference on Computer Vision (ICCV) 2019

    • Learning Grid Cells as Vector Representation of Self-Position Coupled with Matrix Representation of Self-Motion [PDF][project page]
    • --Ruiqi Gao *, Jianwen Xie *, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --Seventh International Conference on Learning Representations (ICLR) 2019

    • Learning Dynamic Generator Model by Alternating Back-Propagation Through Time [PDF][project page]
    • --Jianwen Xie *, Ruiqi Gao *, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) 2019 (Spotlight)

    • Segregated Temporal Assembly Recurrent Networks for Weakly Supervised Multiple Action Detection
    • --Yunlu Xu, Chengwei Zhang, Zhanzhan Cheng, Jianwen Xie, Yi Niu, Shiliang Pu, Fei Wu
    • --The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) 2019 (Oral)

    • Cooperative Training of Descriptor and Generator Networks [PDF][project page]
    • --Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu
    • --IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2018
    • (an expanded version of the AAAI-18 paper for cooperative learning)

    • Learning Descriptor Networks for 3D Shape Synthesis and Analysis [PDF][project page]
    • --Jianwen Xie *, Zilong Zheng *, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018 (Oral)

    • A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects [PDF]
    • --Yuanlu Xu *, Lei Qin *, Xiaobai Liu, Jianwen Xie, Song-Chun Zhu (* equal contributions)
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018

    • Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer [PDF]
    • --Hao-Shu Fang, Guansong Lu, Xiaolin Fang, Jianwen Xie, Yu-Wing Tai, Cewu Lu
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018 (Spotlight)

    • Cooperative Learning of Energy-Based Model and Latent Variable Model via MCMC Teaching [PDF][project page]
    • --Jianwen Xie, Yang Lu, Ruiqi Gao, Ying Nian Wu
    • --The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI) 2018 (Oral)

    • Sparse and Deep Generalizations of the FRAME Model [PDF][Tex]
    • --Ying Nian Wu, Jianwen Xie, Yang Lu, Song-Chun Zhu
    • --Annals of Mathematical Sciences and Applications (AMSA) 2018
    • (an invited paper to the AMSA special issue in honor of Professor David Mumford on his 80th birthday)

    • Super-Trajectory for Video Segmentation [PDF]
    • --Wenguan Wang, Jianbing Shen, Jianwen Xie, Fatih Porikli
    • --International Conference on Computer Vision (ICCV) 2017

    • Synthesizing Dynamic Pattern by Spatial-Temporal Generative ConvNet [PDF][project page]
    • --Jianwen Xie, Song-Chun Zhu, Ying Nian Wu
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

    • Generative Hierarchical Learning of Sparse FRAME Models [PDF][project page]
    • --Jianwen Xie, Yifei Xu, Erik Nijkamp, Ying Nian Wu, Song-Chun Zhu
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

    • Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms [PDF]
    • --Jianwen Xie, Pamela K. Douglas, Ying Nian Wu, Arthur L. Brody, Ariana E. Anderson
    • --Journal of Neuroscience Methods 2017

    • A Theory of Generative ConvNet [PDF]
    • --Jianwen Xie *, Yang Lu *, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • (an expanded version of the ICML-16 paper written for statisticians.)

    • A Theory of Generative ConvNet [PDF][project page]
    • --Jianwen Xie *, Yang Lu *, Song-Chun Zhu, Ying Nian Wu (* equal contributions)
    • --International Conference on Machine Learning (ICML) 2016 (Oral)

    • Inducing Wavelets into Random Fields via Generative Boosting [PDF][project page]
    • --Jianwen Xie, Yang Lu, Song-Chun Zhu, Ying Nian Wu
    • --Journal of Applied and Computational Harmonic Analysis (ACHA) 2015
    • (ACHA is a premier journal in harmonic analysis and applied mathematics)

    • Learning Sparse FRAME Models for Natural Image Patterns [PDF][project page]
    • --Jianwen Xie, Wenze Hu, Song-Chun Zhu, Ying Nian Wu
    • --International Journal of Computer Vision (IJCV) 2014

    • Learning Inhomogeneous FRAME Models for Object Patterns [PDF][project page]
    • --Jianwen Xie, Wenze Hu, Song-Chun Zhu, Ying Nian Wu
    • --IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2014

    • Non-negative Matrix Factorization of Multimodal MRI, fMRI and Phenotypic Data reveals Differential Changes in Default Mode Subnetworks in ADHD [PDF]
    • --Ariana Anderson, Pamela Douglas, Wesley Kerr, Virginia Haynes, Alan Yuille, Jianwen Xie, Ying Nian Wu, Jesse Brown, Mark Cohen
    • --NeuroImage 2014.