amazon web services - How to autoscale Servers in ECS? -

i started using ecs. able deploy container image in ecr , create task definition container cpu/memory limits. use case each container long running app (no webserver, no port mapping needed). containers spawned on demand 1 @ time , deleted on demand 1 @ time.

i able create cluster n server instances. i'd able server instances automatically scale up/down. example if there isn't enough cpu/memory in cluster, i'd new instance created.

and if there instance no containers running in it, i'd specific instance scaled down / deleted. avoid auto scale down termination of server instance has running tasks in it.

what steps needed able achieve this?

considering have ecs cluster created, aws provides instructions on scaling cluster instances cloudwatch alarms.

assuming want scale cluster based on memory reservation, @ high level, need following:

1. create launch configuration auto scaling group.
2. create auto scaling group, size of cluster can scaled , down.
3. create cloudwatch alarm scale cluster if memory reservation on 70%
4. create cloudwatch alarm scale cluster down if memory reservation under 30%

because it's more of specialty wrote example cloudformation template should started of this:

parameters:   mininstances:     type: number   maxinstances:     type: number   instancetype:     type: string     allowedvalues:       - t2.nano       - t2.micro       - t2.small       - t2.medium       - t2.large   vpcsubnetids:     type: string  mappings:   ecsinstanceamis:     us-east-2:       ami: ami-1c002379     us-east-1:       ami: ami-9eb4b1e5     us-west-2:       ami: ami-1d668865     us-west-1:       ami: ami-4a2c192a     eu-west-2:       ami: ami-cb1101af     eu-west-1:       ami: ami-8fcc32f6     eu-central-1:       ami: ami-0460cb6b     ap-northeast-1:       ami: ami-b743bed1     ap-southeast-2:       ami: ami-c1a6bda2     ap-southeast-1:       ami: ami-9d1f7efe     ca-central-1:       ami: ami-b677c9d2  resources:   cluster:     type: aws::ecs::cluster   role:     type: aws::iam::role     properties:       managedpolicyarns:         - arn:aws:iam::aws:policy/service-role/amazonec2containerserviceforec2role       assumerolepolicydocument:         version: 2012-10-17         statement:           -             effect: allow             action:               - sts:assumerole             principal:               service:                 - ec2.amazonaws.com       instanceprofile:     type: aws::iam::instanceprofile     properties:       path: /       roles:         - !ref role       launchconfiguration:     type: aws::autoscaling::launchconfiguration     properties:       imageid: !findinmap [ecsinstanceamis, !ref "aws::region", ami]       instancetype: !ref instancetype       iaminstanceprofile: !ref instanceprofile       userdata:         fn::base64: !sub |           #!/bin/bash           echo ecs_cluster=${cluster} >> /etc/ecs/ecs.config     autoscalinggroup:     type: aws::autoscaling::autoscalinggroup     properties:       minsize: !ref mininstances       maxsize: !ref maxinstances       launchconfigurationname: !ref launchconfiguration       healthcheckgraceperiod: 300       healthchecktype: ec2       vpczoneidentifier: !split [",", !ref vpcsubnetids] scaleuppolicy:   type: aws::autoscaling::scalingpolicy   properties:     adjustmenttype: changeincapacity     autoscalinggroupname: !ref autoscalinggroup     cooldown: '1'     scalingadjustment: '1' memoryreservationalarmhigh:   type: aws::cloudwatch::alarm   properties:     evaluationperiods: '2'     statistic: average     threshold: '70'     alarmdescription: alarm if cluster memory reservation high     period: '60'     alarmactions:     - ref: scaleuppolicy     namespace: aws/ecs     dimensions:     - name: clustername       value: !ref cluster     comparisonoperator: greaterthanthreshold     metricname: memoryreservation scaledownpolicy:   type: aws::autoscaling::scalingpolicy   properties:     adjustmenttype: changeincapacity     autoscalinggroupname: !ref autoscalinggroup     cooldown: '1'     scalingadjustment: '-1' memoryreservationalarmhigh:   type: aws::cloudwatch::alarm   properties:     evaluationperiods: '2'     statistic: average     threshold: '30'     alarmdescription: alarm if cluster memory reservation low     period: '60'     alarmactions:     - ref: scaleuppolicy     namespace: aws/ecs     dimensions:     - name: clustername       value: !ref cluster     comparisonoperator: lessthanthreshold     metricname: memoryreservation 

this creates ecs cluster, launch configuration, autoscaling group, alarms based on ecs memory reservation.

now can interesting discussions.

why can't scale based on cpu utilization and memory reservation?

the short answer totally can but you're pay lot it. ec2 has known property when create instance, pay minimum of 1 hour, because partial instance hours charged full hours. why that's relevant is, imagine have multiple alarms. have bunch of services running idle, , fill cluster. either cpu alarm scales down cluster, or memory alarm scales cluster. 1 of these scale cluster point it's alarm no longer triggered. after cooldown, period, other alarm undo it's last action, after next cooldown, action redone. instances created destroyed repeatedly on every other cooldown.

after giving bunch of thought this, strategy came use application autoscaling ecs services based on cpu utilization, , memory reservation based on cluster. if 1 service running hot, task added share load. fill cluster memory reservation capacity. when memory gets full, cluster scales up. when service cooling down, services start shutting down tasks. memory reservation on cluster drops, cluster scaled down.

the thresholds cloudwatch alarms might need experimented with, based on task definitions. reason if put scale threshold high, may not scale memory gets consumed, , when autoscaling goes place task, find there isn't enough memory available on instance in cluster, , therefore unable place task.