跳到主要导航 跳到搜索 跳到主要内容

Taming Serverless Cold Start of Cloud Model Inference With Edge Computing

  • Kongyange Zhao
  • , Zhi Zhou*
  • , Lei Jiao
  • , Shen Cai
  • , Fei Xu
  • , Xu Chen
  • *此作品的通讯作者
  • Sun Yat-Sen University
  • University of Oregon

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Serverless computing is envisioned as the de-facto standard for next-generation cloud computing. However, the cold start dilemma has impeded its adoption by delay-sensitive and burst applications. In this paper, we propose to tame serverless cold start in a cloud inference system with edge computing. Specifically, the proposed solution smooths the serverless cloud workload with user-owned edge computing, reducing the number of cold starts. Leveraging the configurability of requests and serverless functions, the proposed solution further reduces the transmission latency and serverless cost by adapting request configuration (e.g., image resolution) and function configuration (e.g., memory). To alleviate the potential inference accuracy degradation incurred by configuration adaption, we aim to strike a nice balance between inference latency, cost, and accuracy. However, achieving this goal is non-trivial since the underlying optimization is non-convex and involves future uncertain information. To simultaneously address dual challenges, the presented cold-start-aware online algorithms apply the regularization technique to decompose the problem into separate convex subproblems. Then, it applies lazy switching to smooth the number of provisioned functions and thus reduces the cold start. Through rigorous theoretical analysis, realistic prototype evaluations on AWS Lambda, and trace-driven simulations, we comprehensively validate the theoretical and empirical performance of our proposed solution.

源语言英语
页(从-至)8111-8128
页数18
期刊IEEE Transactions on Mobile Computing
卷23
期8
DOI
出版状态已出版 - 2024

学术指纹

探究 'Taming Serverless Cold Start of Cloud Model Inference With Edge Computing' 的科研主题。它们共同构成独一无二的学术指纹。

引用此