Xuanlei Zhao

Xuanlei Zhao

PhD Candidate at National University of Singapore

About

Hi! I am a second-year PhD student in Computer Science at National University of Singapore advised by Yang You, where I also completed my master’s studies. I obtained my bachelor’s degree in CS & EE from Huazhong University of Science and Technology.

Research

My current research mainly focuses on efficient and scalable machine learning through algorithm and infrastructure co-optimization, recently with a primary emphasis on efficient video generation.

I am always happy to chat about interesting research ideas, and looking for academic collaborations and interns. Please drop me an email if you are interested in collaborating with me.

Selected Publications (all)

Efficient Video Generation

  • Real-Time Video Generation with Pyramid Attention Broadcast
    International Conference on Learning Representations (ICLR), 2025
    Xuanlei Zhao*, Xiaolong Jin*, Kai Wang*, Yang You
    | paper | code | blog |

  • Training Variable Sequences with Data-Centric Parallel
    arXiv
    Geng Zhang*, Xuanlei Zhao*, Kai Wang†, Yang You†
    | code | blog |

  • DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
    arXiv
    Xuanlei Zhao, Shenggan Cheng, Chang Chen, Zangwei Zheng, Ziming Liu, Zheming Yang, Yang You
    | paper | code |

Efficient Memory Cost

  • AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
    International Conference on Learning Representations (ICLR), 2024
    Xuanlei Zhao, Shenggan Cheng, Guangyang Lu, Jiarui Fang, Haotian Zhou, Bin Jia, Ziming Liu, Yang You
    | paper | code |

  • HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
    Annual Conference on Machine Learning and Systems (MLSys), 2024
    Xuanlei Zhao*, Bin Jia*, Haotian Zhou*, Ziming Liu, Shenggan Cheng, Yang You
    | paper |

Efficient AI for Science

  • FastFold: Optimizing AlphaFold Training and Inference on GPU Clusters
    Annual Symposium on Principles and Practice of Parallel Programming (PPoPP), 2024
    Shenggan Cheng, Xuanlei Zhao, Guangyang Lu, Jiarui Fang, Tian Zheng, Ruidong Wu, Xiwen Zhang, Jian Peng, Yang You
    | paper | code |

Open-Source Projects

  • VideoSys: An Easy and Efficient System for Video Generation

  • Colossal-AI: Making large AI models cheaper, faster and more accessible

Experience

Education

  • National University of Singapore
    Ph.D. in Computer Science
    2024.01 - now
  • National University of Singapore
    M.S. in Computer Science
    2022.08 - 2023.12
  • Huazhong University of Science and Technology
    B.S. in Computer Science & Electronic Information
    2018.09 - 2022.06