Xuanlei Zhao

Xuanlei Zhao

PhD Candidate at National University of Singapore

About

Hi! I am a first-year PhD student in Computer Science at National University of Singapore advised by Yang You, where I also completed my master’s studies. I obtained my bachelor’s degree in CS & EE from Huazhong University of Science and Technology supervised by Xinggang Wang. Previously, I collaborated at Pika with Chenlin Meng and interned at Colossal-AI with Jiarui Fang.

I am actively looking for summer research internship in 2025. Please feel free to reach out if there are any opportunities available.

Research

My current research mainly focuses on efficient and scalable machine learning systems through parallelism, algorithm, scheduling and compiler and optimization, recently with a primary emphasis on video models acceleration. I am always happy to chat about interesting research ideas, and looking for academic collaborations. Please drop me an email if you are interested in collaborating with me.

Open-Source Projects

  • VideoSys: An Easy and Efficient System for Video Generation
    Project lead.
  • Colossal-AI: Making large AI models cheaper, faster and more accessible
    Core contributor (ranked 5th by 2024) with 38k star.

Selected Publications (all)

System for Video Models

  • Real-Time Video Generation with Pyramid Attention Broadcast
    | Arxiv | First author | paper | code | blog |

  • DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers
    | Arxiv | First author | paper | code |

System with Low Memory Cost

  • AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference
    | ICLR 2024 | First author | paper | code |

  • HeteGen: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices
    | MLSys 2024 | First author | paper |

System for Machine Learning & Science

  • FastFold: Optimizing AlphaFold Training and Inference on GPU Clusters
    | PPoPP 2024 | Second author | paper | code |

Experience

  • Pika
    Host: Chenlin Meng
    Efficient video system.

  • Colossal-AI
    Research Engineer Intern
    Host: Jiarui Fang and Shenggui Li
    Core contributor (ranked 5th by 2024) with 38k star, and responsible for features including ZeRO, TP, PP, compiler, and MoE.
    2022.07 - 2023.12

Education

  • National University of Singapore
    Ph.D. in Computer Science
    2024.01 - now
    M.S. in Computer Science
    2022.08 - 2023.12
  • Huazhong University of Science and Technology
    B.S. in Computer Science & Electronic Information
    2018.09 - 2022.06