Cloudability’s Rightsizing Engine evaluates the underlying resource utilization for each EC2 instance and recommends instance types that are well matched to each utilization profile. The end goal is to keep your costs down while being mindful of operational risks.
To get the most accurate recommendation there are four utilization metrics that need to be assessed: CPU, Disk IOPS, Network Bandwidth and Memory Utilization. For a number of very good reasons we’ve taken the approach of pulling this data directly from CloudWatch. One of these reasons is that three of the metrics are saved to CloudWatch by default for each instance without you doing anything. The fourth one however, Memory Utilization, requires a little initial effort on your end to publish to CloudWatch. This is done using what AWS calls Custom Metrics which you can read about here.
The good news is that AWS has come up with a standard format for memory data publication and we’ve taken a streamlined approach to ingest that data. We require only one custom metric be published:
namespace: System/Linux or Windows/Default
dimensions: InstanceId (it’s important to only add this one dimension)
It’s generally recommended to publish this information every 5 minutes. Once you have this data flowing into CloudWatch we’ll pick it up without requiring you do anything more.
If you’d like to validate within CloudWatch itself you can do so in the AWS console and you should find graphs similar to this:
Install the standard CloudWatch monitoring scripts for Linux available on the Amazon site. This is the most popular and straightforward option. Here is the crontab configuration we use at Cloudability:
*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl --mem-util --from-cron
This will publish exactly what Cloudability requires and nothing more. Other options include creating your own agent which integrates with the AWS SDK. Here is a great example using Golang that some of our customers have had success with.
The standard approach from Amazon has been to use a PowerShell script. You could also again use your own agent connected with the AWS SDK. Reach out to our team if you’d like more advice on this.
Having this memory data within CloudWatch is going to provide benefits well beyond Cloudability and we’d highly recommend going down this path. For example you could use the memory data to trigger autoscaling events or trigger alarms. Once you find what method works for you it’d be a good idea to roll that up into your configuration.