HKU HKU Dept of Statistics & Actuarial Science, HKU

Seminar by Dr. Shangzhe WU from Computer Science Department, Stanford University

DateWednesday, 22 February 2023
Time3:30 p.m. – 4:30 p.m. 2:30 p.m. – 3:30 p.m.
Venuein Room 301, Run Run Shaw Building
TitleLearning dynamic 3D objects in the wild

We live in a dynamic physical world, surrounded by all kinds of 3D objects. Designing perception systems that can see the world in 3D from only 2D observations is not only key to many AR and robotics applications, but also a cornerstone for general visual understanding. Prevalent learning-based methods often treat images simply as compositions of 2D patterns, ignoring the fact that they arise from a 3D world. The major obstacle is the lack of large-scale 3D annotations for training, which are prohibitively expensive to collect. Natural intelligences, on the other hand, develop comprehensive 3D understanding of the world primarily by observing 2D projections, without relying on extensive 3D supervision. This begs the question: "can machines learn to perceive the 3D world without explicit 3D supervision?" In this talk, I will present some of our recent efforts in approaching this question, and show that physically-grounded, disentangled 3D object representations can be learned simply from raw photos and videos on the Internet, through an inverse rendering framework.

About the speaker

Shangzhe Wu obtained PhD from University of Oxford, advised by Andrea Vedaldi and Christian Rupprecht at the Visual Geometry Group (VGG). His current research focuses on unsupervised 3D perception and inverse rendering. He also spent time interning at Google Research with Noah Snavely's team. His work on unsupervised learning of symmetric 3D objects received the Best Paper Award at CVPR 2020. Homepage: