Using ec2 spot instances for ECS Cluster?

Gruntwork Customers
1

Any Gruntwork customers using ec2 spot instances for their ECS cluster? Any tips/guides on what has been a successful approach? Thanks.

1 Like
2

I was browsing the launch config docs and came across this simple spot instance example. It seems like all you need to do is set the spot_instance parameter and the ASG will try to pick instances at that price or lower. I don’t have any experience with running ECS on a spot fleet (though there are plenty of blog posts, such as this one, that show it’s quite doable), so maybe all we need to do is expose that parameter in the ecs-cluster module?

3

@jim
It did turn out to be almost as easy as exposing the spot_price parameter in the ecs-cluster module, but there were a few gotchas:

  • It appears that in the Terraform aws_launch_configuration resource, spot_price and placement_tenancy are mutually exclusive. I was unable to find a clever way to conditionally exclude passing placement_tenancy when a non-empty value was set for spot_price, so I just removed the tenancy variable altogether from my fork of the ecs-cluster module. I don’t think I’ll actually need to request anything but the default AWS option for the foreseeable future, so it was easiest for me to just remove it.
  • On my first attempt I set the default value of spot_price to 0 which didn’t work - changing the default value to an empty string produced the desired behavior of being able to conditionally make a spot instance request by setting simply setting a numeric value for spot_price. So, if you decided to add it to the module, try making spot_price = "" as the default.

So, with a fairly small lift we’re now paying about 1/3 the price for the EC2 instances in our QA and Stage ECS clusters! I’d be happy to put up a PR for this, but I’d probably need some guidance on how to deal with the placement_tenancy issue.

4

I put up a PR for discussion if you’re interested in pursuing this:
https://github.com/gruntwork-io/module-ecs/pull/55

5

Ah, cool finding! I responded in the PR. Let’s continue discussing there to see if we can find a way to make it work.

6

Thanks to the help of a couple customers (Jamie for getting the ball rolling, David for wrapping it up), the ecs-cluster module now supports spot pricing via a new cluster_instance_spot_price parameter: https://github.com/gruntwork-io/module-ecs/releases/tag/v0.6.4.

1 Like