Xuanlei Zhao

me.jpeg

CS PhD student at NUS

xuanlei [at] comp.nus.edu.sg

Hello! I am a first-year PhD student in Computer Science at National University of Singapore advised by Yang You. Prior to that, I earned my M.S. in Computer Science from the same institution supervised by Yang You. My academic journey began with the B.Eng. in Computer Science & Electronic Information from Huazhong University of Science and Technology supervised by Xinggang Wang and Xiaojun Hei.

My research focuses on efficient and scalable machine learning system with a primary emphasis on the following areas:

  • Distributed training optimization: Explore novel parallelism and communication strategy to enhance efficiency. [PPoPP24]
  • Memory optimization: Leverage compiler and heterogeneous to achieve efficient memory usage. [ICLR24, MLSys24]
  • Acceleration of AI applications: Speed up and reduce memory cost of diverse AI applications. [PPoPP24]

Open-source Projects:

  • OpenDiT: An Easy, Fast and Memory-Efficent System for DiT Training and Inference
  • FastFold: Optimizing AlphaFold Training and Inference on GPU Clusters
  • ColossalAI: Making large AI models cheaper, faster and more accessible

Moreover, I play a pivotal role in an open-source training system Colossal-AI (with over 35k stars on GitHub). As one of the core contributors (ranked 5th), I lead a MoE and a compiler project, and actively participating in the development of various parallelism strategies.

Feel free to reach out via email if you’re interested in collaborating with me, especially for the acceleration of AI applications!

I am actively looking for summer research internship in 2025. Please feel free to reach out if there are any opportunities available.


selected publications

  1. AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
    Xuanlei Zhao, Shenggan Cheng, Guangyang Lu, Haotian Zhou, Bin Jia, and Yang You
    ICLR 2024
  2. FastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours
    Shenggan Cheng, Xuanlei Zhao, Guangyang Lu, Jiarui Fang, Zhongming Yu, Tian Zheng, Ruidong Wu, Xiwen Zhang, Jian Peng, and Yang You
    PPoPP 2024
  3. HeteGen: Efficient Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
    Xuanlei Zhao, Bin Jia, Haotian Zhou, Ziming Liu, Shenggan Cheng, and Yang You
    MLSys 2024

experience

2022.6 - 2023.12
HPC-AI Tech (ColossalAI)
Machine Learning System Research Intern

education

2024.1 - Now
National University of Singapore
Ph.D. in Computer Science
2022.8 - 2023.12
National University of Singapore
M.S. in Computer Science
2018.9 - 2022.6
Huazhong University of Science and Technology
B.Eng. in Computer Science & Electronic Information

news

2023.11
Got one second-authored paper accepted to PPoPP 2024. Congrats to Shenggan!
2023.10
I’m delighted to share that I’ve been accepted into the Doctor of Philosophy program in Computer Science at the National University of Singapore, commencing in Spring 2024.