A Reinforcement Learning Approach to Reduce Serverless Function Cold Start Frequency

Serverless computing is an event-driven cloud computing architecture for processing requests on-demand, using light weight function containers and a micro-services model. A variety of applications like Internet of Things (IoT) services, edge computing, and stream processing have been introduced to the serverless paradigm. These applications are characterized by their stringent response time requirements, therefore expecting a quick and fault tolerant feedback from the application. The serverless, or Function-as-a-Service (FaaS), paradigm suffers from function `cold start' challenges, where the serverless platform takes time to set up the dependencies, prepare the runtime environment and code for execution before serving the incoming workload. Most of the current works address the problem of cold start by (1) reducing the start-up or preparation time of function containers, or (2) reducing the frequency of function cold starts on the platform. Recent industrial research has identified that factors such as runtime environment, CPU and memory settings, invocation concurrency, and networking requirements, affect the cold start of a function. Therefore, we propose a Reinforcement Learning (Q-Learning) agent setting, to analyze the identified factors such as function CPU utilization, to ascertain the function-invocation patterns and reduce the function cold start frequency by preparing the function instances in advance. The proposed Q-Learning agent interacts with the Kubeless serverless platform by discretizing the environment states, actions and rewards with the use of per-instance CPU utilization, available function instances and success or failure rate of response, respectively. The workload is replicated using the Apache JMeter non-GUI toolkit and our agent is evaluated against the baseline default auto-scale feature of Kubeless. The agent demonstrates the capability of learning the invocation pattern, make informed decisions by preparing the optimal number of function instances over the period of learning, under controlled environment settings.