Haoran Qiu

Haoran Qiu

Redmond, Washington, United States
2K followers 500+ connections

About

At Microsoft Azure Research, I work on systems for AI efficiency, including agentic AI…

Activity

2K followers

See all activities

Experience

  • Microsoft

    Redmond, Washington, United States

  • -

    Mountain View, California, United States

  • -

    Yorktown Heights, New York, United States

  • -

    Sunnyvale, California, United States

  • -

    Redmond, Washington, United States

  • -

    Sunnyvale, California, United States

  • -

    Hong Kong SAR

  • -

    Hong Kong SAR

  • -

    Madison, Wisconsin

  • -

    Hong Kong SAR

  • -

    Hong Kong SAR

  • -

    Hong Kong SAR

  • -

    Shanghai, China

Education

Licenses & Certifications

Volunteer Experience

Publications

  • SIMPPO: A Scalable and Adaptive Online Learning Framework for Serverless Resource Management

    SoCC 2022

  • FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices

    OSDI 2020

  • A Geography-Based P2P Overlay Network for Fast and Robust Blockchain Systems

    IEEE TSC 2022

  • A Mean-Field Game Approach to Cloud Resource Management with Function Approximation

    NeurIPS 2022

  • Delay Sensitivity-driven Congestion Mitigation for HPC Systems

    ICS 2021

  • Evaluating Hardware Memory Disaggregation Under Delay and Contention

    CompSys 2022

  • Is Serverless Function-as-a-Service a Good Fit for Latency-critical Services?

    WoSC 2021

  • OWL: Understanding and Detecting Concurrency Attacks

    DSN 2018

  • PLOVER: Fast, Multi-core Scalable Virtual Machine Fault-tolerance

    NSDI 2018

  • Reinforcement Learning for Resource Management in Multi-tenant Serverless Platforms

    EuroMLSys 2022

Join now to see all publications

Courses

  • Advanced Algorithms

    CS 577

  • Advanced Distributed Systems

    CS 525

  • Advanced Operating Systems

    CS 523

  • Artificial Intelligence

    COMP 3270

  • Compiler Techniques

    CS 3235

  • Computer Networks

    CS 640

  • Computer Organization

    COMP 2120

  • Data Science and Analytics

    CS 498

  • Data Structure and Algorithms

    COMP 2119

  • Database Systems

    COMP 3278

  • Dependable AI Systems

    ECE 598

  • Discrete Mathematics

    COMP 2121

  • Distributed Systems

    CS 425

  • Fault-tolerant System Design

    CS 536

  • Functional Programming

    CS 3258

  • High-speed and Programmable Networks

    ECE 598

  • ML for Software Engineering

    CS 598

  • Machine Learning

    CS 446

  • Object-oriented Programming

    COMP 2396

  • Operating Systems

    COMP 3230

  • Principles of Programming Languages

    COMP 3259

  • Reinforcement Learning

    CS 498

  • Software Engineering

    COMP 3297

  • System Architecture and Distributed Computing

    COMP 3412

  • Trustworthy Machine Learning

    CS 442

Projects

  • Power-aware Deep Learning Model Serving

    -

    With the increasing popularity of large deep-learning model serving workloads like LLMs, there is a pressing need to reduce the energy consumption of a model-serving cluster while maintaining satisfied throughput or model-serving latency requirements. Model multiplexing approaches such as model parallelism, model placement, replication, and batching aim to optimize the model-serving performance. However, they fall short of leveraging the GPU frequency scaling opportunity for power saving. In…

    With the increasing popularity of large deep-learning model serving workloads like LLMs, there is a pressing need to reduce the energy consumption of a model-serving cluster while maintaining satisfied throughput or model-serving latency requirements. Model multiplexing approaches such as model parallelism, model placement, replication, and batching aim to optimize the model-serving performance. However, they fall short of leveraging the GPU frequency scaling opportunity for power saving. In this project, we demonstrate (1) the benefits of GPU frequency scaling in power saving for model serving; and (2) the necessity for co-design and optimization of fine-grained model multiplexing and GPU frequency scaling. We explore the co-design space and present a novel power-aware model-serving system, µ-Serve. µ-Serve is a model-serving framework that optimizes the power consumption and model serving latency/throughput of serving multiple ML models efficiently in a homogeneous GPU cluster. Evaluation results on production workloads show that µ-Serve achieves 1.2–2.6x power saving by dynamic GPU frequency scaling (up to 61% reduction) without SLO attainment violations.

    Published at ATC 2024: https://haoran-qiu.com/publication/atc-2024/

  • AWARE: Automate Workload Autoscaling with RL in Production Cloud Systems

    -

    AWARE (ATC 2023, NeurIPS 2023) is an extensible framework for deploying and managing RL-based agents in production systems. AWARE leverages meta-learning and bootstrapping to (a) automatically and quickly adapt to different workloads and (b) provide safe and robust RL exploration. AWARE provides a common OpenAI Gym-like RL interface to agent developers for easy integration with different systems tasks. We demonstrated the use of AWARE in the case of multidimensional workload…

    AWARE (ATC 2023, NeurIPS 2023) is an extensible framework for deploying and managing RL-based agents in production systems. AWARE leverages meta-learning and bootstrapping to (a) automatically and quickly adapt to different workloads and (b) provide safe and robust RL exploration. AWARE provides a common OpenAI Gym-like RL interface to agent developers for easy integration with different systems tasks. We demonstrated the use of AWARE in the case of multidimensional workload autoscaling.

    Published at ATC 2023: https://www.usenix.org/conference/atc23/presentation/qiu-haoran

  • Scalable Serverless Computing Resource Management Framework that Support Multi-tenancy

    -

    SIMPPO (SoCC 2022, NeurIPS 2022): SIMPPO is a scalable multi-agent RL framework that provides systems support for RL agents to coexist in a dynamic multi-tenant cloud environment. With SIMPPO, each RL agent is able to converge efficiently during training and achieve online policy-serving performance comparable to the agent trained in an isolated environment. We demonstrated the use of SIMPPO in the task of serverless resource management.

    https://dl.acm.org/doi/abs/10.1145/3542929.3563475

  • CoCo: Coordinated Container Scheduling with Last-Level Cache and Memory Bandwidth Partitioning

    -

    Last-level cache (LLC) and memory bandwidth partitioning are commonly used in existing work to meet QoS requirements of all co-scheduled latency-critical applications consolidated on a physical server. With the increasing popularity of cloud microservices and the Function-as-a-Service paradigm, the number of containers consolidated together increases significantly. However, due to the limitation of hardware features, existing work fails to support such a number of applications.

    To bridge…

    Last-level cache (LLC) and memory bandwidth partitioning are commonly used in existing work to meet QoS requirements of all co-scheduled latency-critical applications consolidated on a physical server. With the increasing popularity of cloud microservices and the Function-as-a-Service paradigm, the number of containers consolidated together increases significantly. However, due to the limitation of hardware features, existing work fails to support such a number of applications.

    To bridge this gap, this project proposes CoCo, coordinated container scheduling with LLC, and Memory bandwidth partitioning. Our quantitative evaluation shows that CoCo outperforms no-partitioning and baseline approaches by up to 920% and 9.4% respectively. Our main contributions are:

    - We present a comprehensive characterization on the impact of LLC and memory bandwidth partitioning and the sensitivity to shared resource contention for popular services widely used in cloud microservices.
    - We propose CoCo, coordinated container scheduling with LLC, and memory bandwidth partitioning. CoCo dynamically profiles consolidated workloads and provides a time-sharing algorithm (multi-level queue-based weighted round-robin algorithm) to maximize the overall performance and resource utilization.
    - We implemented a prototype of CoCo as a user-level runtime system on Linux, which requires no modification to the underlying OS or the consolidated applications.

    See project
  • FIRM: Intelligent Fine-grained Resource Orchestration for Cloud Microservices

    -

    FIRM (OSDI 2020): FIRM is an intelligent fine-grained resource management framework for sharing resources across microservices to provide predictable performance and drive up overall utilization. FIRM leverages online telemetry data and ML/RL methods to adaptively (a) detect/localize microservices that cause SLO violations, (b) identify low-level resources in contention, and (c) take actions to mitigate SLO violations via dynamic…

    FIRM (OSDI 2020): FIRM is an intelligent fine-grained resource management framework for sharing resources across microservices to provide predictable performance and drive up overall utilization. FIRM leverages online telemetry data and ML/RL methods to adaptively (a) detect/localize microservices that cause SLO violations, (b) identify low-level resources in contention, and (c) take actions to mitigate SLO violations via dynamic reprovisioning.

    https://www.usenix.org/conference/osdi20/presentation/qiu

Honors & Awards

  • ML and Systems Rising Stars 2023

    MLCommons

  • Mavis Future Faculty Fellowship

    Grainger College of Engineering, University of Illinois at Urbana-Champaign

  • Yunni & Maxine Pao Memorial Fellowship

    Grainger College of Engineering, University of Illinois, Urbana-Champaign

  • Conference Presentation Award

    Graduate College, University of Illinois, Urbana-Champaign

  • Dean's Honor List

    Faculty of Engineering

  • First Runner-up of Final Year Project Competition

    Department of Computer Science, HKU

  • Dean's Honor List

    Faculty of Engineering

  • Exchange & Visiting International Student Academic Excellence Award

    University of Wisconsin-Madison

  • Lee Shau Kee Scholarships for Student Enrichment

    The University of Hong Kong

  • Honorable Mention, Mathematical Contest of Modelling

    COMAP

  • Dean's Honor List

    Faculty of Engineering

  • HKU Foundation Entrance Scholarship

    The University of Hong Kong

Languages

  • English

    Professional working proficiency

  • Chinese

    Native or bilingual proficiency

  • Cantonese

    Elementary proficiency

View Haoran’s full profile

  • See who you know in common
  • Get introduced
  • Contact Haoran directly
Join to view full profile

Other similar profiles

Explore top content on LinkedIn

Find curated posts and insights for relevant topics all in one place.

View top content

Add new skills with these courses