RealWonder: Real-Time Physical Action-Conditioned Video Generation

◆Wei Liu, ◆Ziyu Chen, Zizhang Li, Yue Wang, ◇Hong-Xing Yu, ◇Jiajun Wu

Under Review

RealWonder turns a single image into an interactive scene using a controllable video model. By incorporating physics, it enables physically grounded 3D interactions.

The Neverwhere Visual Parkour Benchmark Suite

Ziyu Chen, Henghui Bao, Haoran Chang, Alan Yu, Ran Choi, Kai McClennen, Gio Huh, Kevin Yang, Ri-Zhao Qiu, Yajvan Ravan, John J. Leonard, Xiaolong Wang, Phillip Isola, ◇Ge Yang, ◇Yue Wang

Under Review

A toolchain and benchmark suite for hyper-realistic visual locomotion, building high-fidelity digital twins of real-world environments for closed-loop evaluation.

OmniRe: Omni Urban Scene Reconstruction

Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, Li Song, Yue Wang

ICLR 2025 (Spotlight)

OmniRe is a holistic framework for dynamic scene reconstruction that provides comprehensive coverage, including static backgrounds, driving vehicles, and non-rigidly moving dynamic actors.

360-Degree Panorama Generation from Few Unregistered NFoV Images

◆Jionghao Wang, ◆Ziyu Chen, Jun Ling, Rong Xie, Li Song

ACM Multimedia 2023

We proposed a diffusion-based pipeline that generates complete 360 panoramas using one or more unregistered NFoV images captured from arbitrary angles. This pipeline not only delivers superior quality panoramas but also offers a broader range of outcomes due to its text-conditioned generation capability.

L-Tracing: Fast Light Visibility Estimation on Neural Surfaces by Sphere Tracing

Ziyu Chen, Chenjing Ding, Jianfei Guo, Dongliang Wang, Yikang Li, Xuan Xiao, Wei Wu, Li Song

ECCV 2022

Our L-Tracing based reflectance factorization framework produces photo-realistic novel view images with nearly 10x speedup, compared with the same framework applying volumetric integration for light visibility estimation.

A 3DGS codebase for dynamic urban scene reconstruction and simulation, featuring different Gaussian representations for rigid actors (vehicles) and non-rigid actors (pedestrians, cyclists, etc.). Supported datasets include Waymo, PandaSet, ArgoVerse2, KITTI, NuScenes, NuPlan.