Elastic Compute Cloud (EC2)

EC2 provides temporary virtual computers for running analyses.

Understanding costs

EC2 is pay-as-you-go. For the lab's instances, compute is normally billed for each second that an instance is in the running state. A running instance costs the same whether it is busy, idle, or waiting at a shell prompt.

The selected instance type is usually the largest part of the cost. Instances with more CPU, memory, GPUs, or specialized hardware have higher rates. The total cost of an analysis also depends on how long the instance runs and on related resources such as EBS storage.

Estimating a job

Before launching an unusually large instance or a job expected to run for several days, make a rough estimate:

estimated cost = instance hourly rate × running hours + prorated EBS storage

Check the current rate for the intended instance type and operating system on the EC2 On-Demand pricing page. Select the Stockholm Region (eu-north-1).

Keeping costs under control

• Choose the smallest approved launch template that can complete the job reliably. Running out of memory or disk and repeating the analysis can cost more than selecting an appropriate size initially.

• Copy only the data needed for the current analysis from S3.

• Monitor the job and terminate failed, completed, or abandoned instances promptly.

• Do not assume a stopped instance is free. Its EBS volumes remain and continue to incur storage charges.

• Apply accurate Owner and Project tags so usage can be attributed and unexpected resources can be investigated.

Launching EC2 instances

Launching an EC2 instance from an approved template

Most users should launch an EC2 instance from one of the provided launch templates: EC2 > Instances > Launch Templates. These templates have been set up by administrators to ensure that reasonable resources are requested and created EBS volumes are deleted on termination. They also set the necessary network settings and IAM profiles for connecting to the instance through SSM Session Manager rather than SSH.

Choose the smallest template that can reasonably run your job. Upload final outputs to S3 and terminate the instance when finished.

With any instances you create, it is expected that you assign an Owner tag with your username, and a Project tag with a short code that will help identify the purpose of the instance. This way, if instances are left running or stopped, we can more easily triage what can safely be terminated.

You can launch templates from EC2 > Instances > Launch Templates > Launch instance from template in the AWS Management Console or with the AWS CLI, for example:

aws ec2 run-instances \
    --launch-template LaunchTemplateName=rprlab-hoodini-standard \
    --tag-specifications \
        "ResourceType=instance,Tags=[
            {Key=Owner,Value=leightonpayne},
            {Key=Project,Value=hoodini}
        ]"

If it launched successfully, the command will return some JSON. Look for the key InstanceID and take note of the value, e.g., i-03e1eb484512d1b89:

{
  "ReservationId": "r-04d942d3be0a42dab",
  "OwnerId": "376129434881",
  "Groups": [],
  "Instances": [
    {
      // Truncated
      "InstanceId": "i-03e1eb484512d1b89"
      // Truncated
    }
  ]
}

To connect in the AWS Management Console, open EC2 > Instances, select the instance, and choose Connect > Session Manager > Connect. Alternatively, use the instance ID with the AWS CLI:

aws ssm start-session --target i-03e1eb484512d1b89

When you are done with the instance, terminate it from EC2 > Instances by choosing Instance state > Terminate instance, or use the AWS CLI:

aws ec2 terminate-instances --instance-ids i-03e1eb484512d1b89

The command will return some JSON. Make sure the CurrentState is shutting-down:

{
  "TerminatingInstances": [
    {
      "InstanceId": "i-03e1eb484512d1b89",
      "CurrentState": {
        "Code": 32,
        "Name": "shutting-down"
      },
      "PreviousState": {
        "Code": 16,
        "Name": "running"
      }
    }
  ]
}

Launching a custom EC2 instance with an approved AMI

If you need a nonstandard instance configuration, e.g., larger storage, more CPU, GPU, or memory, you can launch one of the lab AMIs with a custom instance: EC2 > Images > AMIs. Users who bypass the launch template are responsible for confirming that temporary EBS volumes are deleted on termination and excessive resources are not requested or left running. Creating, modifying, and retiring AMIs is an administrator responsibility.

If you find the resources provided by our approved launch templates are consistently not fit for your purposes, talk to an an Owner or Administrator so we can configure a more appropriate template. It is preferred that we work together to establish reusable AMIs and launch templates to forecast costs and build an organized and maximally useful ecosystem for all users.

You can launch instances from AMIs from the AWS Management Console or the AWS CLI:

aws ec2 run-instances \
    --image-id ami-0438575438cea9d9f \
    --instance-type t3.large \
    --iam-instance-profile Name=rprlab-ec2-research-instance-role \
    --security-group-ids sg-077c3d89ca8f32618
    --tag-specifications \
        "ResourceType=instance,Tags=[
            {Key=Owner,Value=leightonpayne},
            {Key=Project,Value=hoodini}
        ]"

Connect and terminate the instance as if launching an EC2 instance from an approved template.

Launching a custom EC2 instance from scratch

If you need a totally bespoke software and hardware setup, contact an Owner or Administrator. It is preferred that we work together to establish reusable AMIs and launch templates to forecast costs and build an organized and maximally useful ecosystem for all users.

Learn about EBS volumes