Technical Papers Archive

Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service

Authors: Baolin Li (Northeastern University); Siddharth Samsi and Vijay Gadepally (Massachusetts Institute of Technology (MIT), Lincoln Laboratory); and Devesh Tiwari (Northeastern University)

Abstract: This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern technology products, but it is also a significant contributor to carbon footprint. We introduce, Clover, a carbon-friendly ML inference service runtime system that balances performance, accuracy, and carbon emissions through mixed-quality models and GPU resource partitioning. Our experimental results demonstrate that Clover is effective in substantially reducing carbon emissions while maintaining high accuracy and meeting service level agreement (SLA) targets.




Back to Technical Papers Archive Listing