Jun-Yan Zhu(朱俊彦)

Computer Science and Artificial Intelligence Laboratory

Massachusetts Institute of Technology

Email: junyanz at mit dot edu

I am a postdoctoral researcher at MIT, working with William T. Freeman, Josh Tenenbaum, and Antonio Torralba. I obtained my Ph.D. from UC Berkeley after spending five wonderful years at CMU and UC Berkeley with Alexei A. Efros. I received my B.E from Tsinghua University. I study computer vision, computer graphics, and machine learning with the goal of building intelligent machines, capable of recreating our visual world.

IJCV Special Issue on Generative Adversarial Networks for Computer Vision (Due: March 31, 2019)

Guest editors: Jun-Yan Zhu, Hongsheng Li, Eli Shechtman, Ming-Yu Liu, Jan Kautz, Antonio Torralba

[Code] PyTorch implementation for CycleGAN and pix2pix (with PyTorch 0.4+).

[Teaching] Co-taught the Deep learning course at Udacity.

[Service] SIGGRAPH Asia 2018, Technical Papers Committee member.

[Service] International Journal of Computer Vision, Guest editor.

[Tutorial] CVPR 2018 Tutorial on Generative Adversarial Networks.

[Tutorial] ICCV 2017 Tutorial on Generative Adversarial Networks.

[Workshop] ICML 2017 Workshop on Visualization for Deep Learning.

[Course] SIGGRAPH Asia 2014 invited Course on Data-Driven Visual Computing.

[CatPapers] Cool vision, learning, and graphics papers on Cats.

[2019] GAN Dissection: Visualizing and Understanding Generative Adversarial Networks David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba ICLR 2019

[2018] Dataset Distillation Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, Alexei A. Efros arXiv 2018

[2018] Propagation Networks for Model-Based Control Under Partial Observation Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B. Tenenbaum, Antonio Torralba, Russ Tedrake arXiv 2018

[2018] Visual Object Networks: Image Generation with Disentangled 3D Representation Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman NeurIPS 2018

[2018] 3D-Aware Scene Manipulation via Inverse Graphics Shunyu Yao*, Tzu-Ming Harry Hsu*, Jun-Yan Zhu, Jiajun Wu, Antonio Torralba, William T. Freeman, Joshua B. Tenenbaum NeurIPS 2018

[2018] Video-to-Video Synthesis Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro NeurIPS 2018

[2018] CyCADA: Cycle-Consistent Adversarial Domain Adaptation Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Alexei A. Efros, and Trevor Darrell ICML 2018

[2018] High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro CVPR 2018

[2018] Spatially Transformed Adversarial Examples Chaowei Xiao*, Jun-Yan Zhu*, Bo Li, Mingyan Liu, and Dawn Song ICLR 2018

[2018] Generating Adversarial Examples with Adversarial Networks Chaowei Xiao, Bo Li, Jun-Yan Zhu, Mingyan Liu, and Dawn Song IJCAI 2018

[2017] Toward Multimodal Image-to-Image Translation Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, and Eli Shechtman NIPS 2017

[2017] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu*, Taesung Park*, Phillip Isola, and Alexei A. Efros ICCV 2017

[2017] Image-to-Image Translation with Conditional Adversarial Nets Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros CVPR 2017

[2017] Real-Time User-Guided Image Colorization with Learned Deep Priors Richard Zhang*, Jun-Yan Zhu*, Phillip Isola, Xinyang Geng, Angela S. Lin, Tianhe Yu, and Alexei A. Efros SIGGRAPH 2017

[2017] Light Field Video Capture Using a Learning-Based Hybrid Imaging System Ting-Chun Wang, Jun-Yan Zhu, Nima Khademi Kalantari, Alexei A. Efros, and Ravi Ramamoorthi SIGGRAPH 2017

[2016] Generative Visual Manipulation on the Natural Image Manifold Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros ECCV 2016

[2016] A 4D Light-Field Dataset and CNN Architectures for Material Recognition Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A. Efros, and Ravi Ramamoorthi ECCV 2016

[2015] Learning a Discriminative Model for the Perception of Realism in Composite Images Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros ICCV 2015

[2015] Unsupervised Object Class Discovery via Saliency-Guided Multiple Class Learning Jun-Yan Zhu, Jiajun Wu, Yan Xu, Eric Chang and Zhuowen Tu TPAMI 2015 | CVPR 2012

[2014] Mirror Mirror: Crowdsourcing Better Portraits Jun-Yan Zhu, Aseem Agarwala, Alexei A. Efros, Eli Shechtman, and Jue Wang SIGGRAPH Asia 2014

[2014] AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections Jun-Yan Zhu, Yong Jae Lee and Alexei A. Efros SIGGRAPH 2014

[2014] MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation Jiajun Wu*, Yibiao Zhao*, Jun-Yan Zhu, Siwei Luo and Zhuowen Tu CVPR 2014

[2014] Reverse Image Segmentation: A High-Level Solution to a Low-Level Task Jiajun Wu, Jun-Yan Zhu, and Zhuowen Tu BMVC 2014

[2014] Multiple Clustered Instance Learning for Histopathology Cancer Image Classification, Segmentation and Clustering Yan Xu*, Jun-Yan Zhu*, Eric I-Chao Chang and Zhuowen Tu CVPR 2012 | Medical Image Analysis 2014

[2013] Motion-Aware Gradient Domain Video Composition Tao Chen, Jun-Yan Zhu, Ariel Shamir and Shi-Min Hu TIP 2013

VON: Code for synthesizing textured 3D objects.

gandissect: Pytorch-based tools for visualizing and understanding the neurons of a GAN.

vid2vid: High-resolution (e.g., 2048x1024) photorealistic video-to-video translation.

CYCADA: Pytorch implementation of cycle-consistent adversarial domain adaptation.

pix2pixHD: 2048x1024 image synthesis with conditional GANs.

BicycleGAN: multimodal image-to-image translation.

Interactive Deep Colorization: real-time interface for user-guided colorization.

PyTorch Colorization: PyTorch code for training interactive colorization models.

Light Field Video: light field video applications (e.g. video refocusing, changing aperture and view).

CycleGAN: Torch implementation for learning an image-to-image translation without input-output pairs.

pix2pix: Torch implementation for learning a mapping from input images to output images.

pytorch CycleGAN & pix2pix: PyTorch implementation for both unpaired and paired image-to-image translation.

iGAN: a deep learning software that easily generates images with a few brushstrokes.

RealismCNN: code for predicting and improving visual realism in composite images.

MCILBoost: a boosting-based Multiple Instance Learning (MIL) software.

MirrorMirror: an expression training App that helps users mimic their own expressions.

SelectGoodFace: a program for selecting attractive/serious portraits from a personal photo collection.

FaceDemo: a simple 3D face alignment and warping demo.

Learning to Generate Images

SIGGRAPH Dissertation Award Talk (2018)

Unpaired Image-to-Image Translation

CVPR Tutorial on GANs (2018)

Learning to Synthesize and Manipulate Natural Photos

MIT, HKUST CSE Departmental Seminar, ICCV Tutorial on GANs, O'Reilly AI, AI with the best, Y Conf, DEVIEW, ODSC West (2017)

On Image-to-Image Translation

Stanford, MIT, Facebook, CUHK, SNU (2017)

Interactive Deep Colorization

SIGGRAPH, NVIDIA Innovation Theater, Global AI Hackathon (2017)

Visual Manipulation and Synthesis on the Natural Image Manifold

Facebook, MSR, Berkeley BAIR, THU, ICML workshop "Visualization for Deep Learning" (2016)

Mirror Mirror: Crowdsourcing Better Portraits

SIGGRAPH Asia (2014)

What Makes Big Visual Data Hard?

SIGGRAPH Asia invited course "Data-Driven Visual Computing" (2014)

AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections

Co-instructor, Deep Learning at Udacity (with Sebastian Thrun, Ian Goodfellow, Andrew Trask, and Udacity Deep Learning Team).

Guest Lecturer, Advances in Computer Vision (6.819/6.869) at MIT.

Teaching Assistant, Image Manipulation and Computational Photography (CS 194-26) at UC Berkeley.

ACM SIGGRAPH Outstanding Doctoral Dissertation Award (2018)

David J. Sakrison Memorial Prize for Outstanding Doctoral Research, by Berkeley EECS (2018)

NVIDIA Pioneer Research Award (2018)

Facebook Fellowship (2015)

Outstanding Undergraduate Thesis in Tsinghua University (2012)

Excellent Undergraduate Student in Tsinghua University (2012)

National Scholarship, by the Ministry of Education of China (2009 and 2010)

Singapore Technologies Engineering China Scholarship (2010, 2011, and 2012)

Photo of my cat Aquarius and my dog Arya.