Yuxiong Wang's Homepage

About Me

Make things as simple as possible, but not simpler

– Albert Einstein

Welcome to my homepage. I am an Assistant Professor of Computer Science at the University of Illinois Urbana-Champaign (UIUC).

My research lies in computer vision, machine learning, and robotics, with a specific focus on open-world perception, meta-learning, multi-modal learning, generative modeling, and agent learning.

Before joining Illinois CS, I was a postdoctoral fellow in the Robotics Institute at Carnegie Mellon University, advised by Prof. Martial Hebert. I was a visitor to the Center for Data Science at New York University, working with Prof. Jean Ponce. I obtained my Ph.D. under the supervision of Prof. Martial Hebert in the Robotics Institute. I have also been closely working with Prof. Deva Ramanan and Prof. Ruslan Salakhutdinov. I receive the Best Paper Honorable Mention Award for streaming perception in ECCV 2020, and Best Paper Award Finalist in CVPR 2019, 2022. I am recognized as a Notable Area Chair in ICLR 2023, and an Expert Reviewer in TMLR. I am selected to participate in the National Academy of Engineering’s (NAE) Frontiers of Engineering symposium.

Please feel free to contact me if interested!

  • I am always seeking self-motivated Ph.D., M.S., and undergraduate students. 
  • Our group also has openings for visiting students.

Current Research Topics

  • Meta-learning and learning to learn
  • Open-world, multi-modal, few-shot, and self-supervised learning
  • 3D vision
  • Generative modeling, predictive learning
  • Human motion and human-object interaction modeling
  • Reinforcement learning, robot/agent learning
  • In-the-wild applications in robotics, autonomous driving, agriculture, materials science, chemistry, healthcare, etc.

Dissertation


Talks

One of our primary research efforts focuses on bridging generative and discriminative learning, facilitating autonomous agents to perceive, interact, and act in the open world. For representative works, please refer to my two recent talks: the first talk, presented at the C3.ai Generative AI Workshop, elaborates on how we ground generative modeling in 3D and 4D; the second talk, given at the London Machine Learning Meetup, explores various techniques that leverage LLMs and generative visual models for perception and decision-making.


Students

PhD Students


Recent Publications

Emerging Pixel Grounding in Large Multimodal Models Without Grounding Supervision

Emerging Pixel Grounding in Large Multimodal Models Without Grounding Supervision

Shengcao Cao, Liang-Yan Gui, Yu-Xiong Wang

arXiv, 2024

[Website] [PDF] [Code]

Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers

Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers

Kai Yan, Alex Schwing, Yu-Xiong Wang

NeurIPS, 2024 (Spotlight)

[Website] [PDF] [Code]

InstructG2I: Synthesizing Images from Multimodal Attributed Graphs

InstructG2I: Synthesizing Images from Multimodal Attributed Graphs

Bowen Jin, Ziqi Pang, Bingjun Guo, Yu-Xiong Wang, Jiaxuan You, Jiawei Han

NeurIPS, 2024

[Website] [PDF] [Code]

ProgressEditor: Simple Progression is All You Need for High-Quality 3D Scene Editing

ProgressEditor: Simple Progression is All You Need for High-Quality 3D Scene Editing

Jun-Kun Chen, Yu-Xiong Wang

NeurIPS, 2024

[Website]

SceneCraft: Layout-Guided 3D Scene Generation

SceneCraft: Layout-Guided 3D Scene Generation

Xiuyu Yang, Yunze Man, Jun-Kun Chen, Yu-Xiong Wang

NeurIPS, 2024

[Website] [PDF] [Code]

InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction

InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction

Sirui Xu, Ziyin Wang, Yu-Xiong Wang, Liang-Yan Gui

NeurIPS, 2024.

[Website] [PDF]

Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Reasoning

Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Reasoning

Yunze Man, Shuhong Zheng, Zhipeng Bao, Martial Hebert, Liang-Yan Gui, Yu-Xiong Wang

NeurIPS, 2024.

[Website] [PDF] [Code]

More Publications


Teaching