Shuang Ma

I am a senior researcher in the Autonomous Systems Research team at Microsoft. My research interests center around multimodal learning, computer vision, reinforcement learning, and robotics. Recently, I have been primarily focused on developing large-scale foundational models for perception and control. My particular interest lies in designing general-purpose model for sequential decision-making, utilizing unified approaches to learn from diverse and/or unlabeled sequential data, and boarder logistic and applications with large foundation models.
I obtained my Ph.D. degree in Computer Science from SUNY Buffalo, where I was a member of Ubiquitous Multimedia Lab (UBMM). I was lucky to have my advisor Prof. Chang Wen Chen and my mentors Daniel Mcduff, Yale Song, and Ashish Kapoor.
Email: yunyikristy AT gmail DOT com
Research Interests
-
Foundation models
-
Multimodal Learning
-
Computer Vision
-
Robotics and Reinforcement Learning
News
-
I am organizing workshop "PerDream: PERception, Decision making and REAsoning through Multimodal foundational modeling" and co-organizing "Workshop on Robot Learning and SLAM" at ICCV 2023. Our websites are coming soon!
-
I am serving as Area Chair for NeurIPS 2023.
-
We announced SMART and released our code! [Blog][MSR Research Focus][Code]
-
Our paper 'SMART: Self-supervised Multi-task pretrAining with contRol Transformers' is accepted by ICLR 2023 as notable top 25% (Spotlight).
-
Our paper 'LaTTe: Language Trajectory TransformEr' is accepted by ICRA 2023. [Blog][Code]
-
Our team EgoMotion-COMPASS got the 2nd place on two tasks of Ego4D challenge (ECCV 2022).
-
Our paper 'COMPASS: Contrastive multimodal pretraining for autonomous systems' is accepted by IROS 2022.
-
Our paper 'Reshaping robot trajectories using natural language commands: A study of multi-modal data alignment using transformers' is accepted by IROS 2022
-
Our paper 'CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning' is accepted by CLeaR 2022. [Blog][Code]
Interns
-
Jianchen Lei, Zhejiang University 2023
-
Yao Wei, Zhejiang University 2023
-
Ruijie Zheng, University of Maryland 2023
-
Yanchao Sun, University of Maryland 2022
-
Arthur Fender Coelho Bucker, Technical University of Munich (TUM) 2022
-
Weijian Xu, UC San Diego 2021
-
Cherie Ho, Carnegie Mellon University 2021
-
Zhaoyang Zeng, Sun Yat-sen University 2020
-
Mingzhi Yu, University of Pittsburgh 2020