This is part 3 of a ten-part series, "Top 10 Strategies to Manage Cost and Continuously Optimize AWS," which is an excerpt from an eBook by the same name, produced in cooperation from AWS by HyperGrid Cloud Management Platform. A link to download the entire eBook with all 10 strategies is located at the bottom of this post.
LOOK FOR LIFE BEYOND EC2—EMBRACE SERVERLESS
SITUATION - AWS Serverless Computing (Lambda) provides a DevOps-friendly, provision-free, highly scalable, and highly available environment automatically, without having to build the supporting infrastructure in EC2. As a result, AWS Lambda is increasingly becoming a deployment architecture of choice for web services and data processing workflows.
COMPLICATION - Uncertainty around pricing often leads to discomfort in choosing Lambda for large-scale deployments. EC2 may have its complications with overlapping instance types, but once you’ve chosen, there are no pricing surprises. In contrast, Lambda pricing is multi-dimensional and based on number of transactions, memory allocated to the function, and runtime—each of which can change many times according to business requirements.
IMPLICATION - A poorly architected Lambda deployment may end up costing much more than its EC2 equivalent. For example, running 30 transactions per second—with each transaction having a run-time of one second consuming 512MB of memory—will cost $1304.90 on Lambda. A c5.2xlarge instance could most likely run the same workload with similar performance and cost less than a fifth of Lambda ($248.88 monthly).
POSITION - A good rule-of-thumb for deciding on Lambda is as follows: If you have a function that is not invoked often, Lambda will almost always be cheaper than EC2. If you have a function that is invoked often but is not resource-intensive, then it will most likely be cheaper to run this in Lambda. However, this should be validated during load testing. If you have a resource-intensive function that runs very frequently, then it is possible that an EC2-based architecture—regardless of its elegance—may be the right choice. Alternatively, it could simply be that the function has been over-provisioned with memory; perhaps the function could be optimized in terms of run-time or memory utilization.
Lambda pricing is a combination of a number of invocations and GB-seconds used by the function. The number of invocations is a function of business success, not an architectural attribute. This leaves GB-seconds as the only lever to optimize pricing: allocate more memory and the runtime will reduce—or vice versa. Optimizing Lambda costs, then, is all about adjusting the lever between memory allocation and execution time to find the optimal balance.
ACTION - Since Lambda functions read completely headless, understanding their behavior and optimizing cost tends to be more complicated than any other AWS service. A number of native AWS tools can be used to understand the behavior of Lambda functions: CloudWatch natively supports a number of Lambda metrics. The Metric Filters function of CloudWatch Logs can be used to convert logging data into additional CloudWatch Metrics. This may require additional instrumentation of your Lambda functions, but the overall benefits are worth the effort. Once these are in place, you should run your function, at scale, with different memory allocations, and use CloudWatch to measure execution time and calculate cost at scale. When you have this data, you can determine the best cost tradeoff between memory and run-time.
At HyperGrid, we felt there was a better way. We have added a Lambda monitoring and optimization service to our HyperCloud Analytics platform to support Lambda analysis and optimization. Capabilities include: Visualize spending on Lambda and detailed metrics on each Lambda function, including key parameters like resource allocations, run time, timeouts, error rates, and so forth. Easily identify resource wastage—functions with a high error rate, over-allocation of memory, under-allocated functions (typically those with a high timeout rate), and so forth. Make recommendations on right-sizing Lambda functions by adjusting the balance between run-time and memory allocation. Compare the cost of continuing with Lambda vs. running an equivalent EC2 environment.
BENEFITS - The ability to analyze and optimize Lambda costs will not only result in a more DevOps friendly and scalable architecture, it will also lower your operational costs by reducing the number of different types of resources that need to be managed.
"Top 10 Strategies to Manage Cost and Continuously Optimize AWS"
Read all ten tips now.