Extra CI flexibility with Github Runner on AWS Spot Instances

Amir Kirsh
Amir Kirsh reading time: 8 minutes
March 9, 2022

This article describes how to set up a GitHub Actions self-hosted runner on an AWS spot instance. But first, let’s talk about what a self-hosted GitHub runner is, and why we should use it.

What’s a GitHub Actions runner?

A GitHub Actions runner is an application that runs a job from a GitHub Actions workflow on a machine.
You can use either a ready-made GitHub-hosted runner (Windows, Ubuntu, or MacOSX, each with specific hardware) running on GitHub servers or a self-hosted runner in your own environment.

GitHub Runners can be used for:

  • Compilation
  • Running tests
  • Static Code Analysis (SCA)
  • Building a container image

And much more!

We’ve talked previously in this post about how to use GitHub actions with a C++ project and in this post about how to tie it with Incredibuild. Here, we’ll focus on connecting GitHub actions to trigger operations on an AWS spot instance.

Why use a self-hosted runner?

GitHub-hosted runners are a great option if you want to run a workflow quickly and easily. However, there are cases where self-hosted are the best choice for you (as opposed to GitHub-hosted runners):

  • You are using a private repository and don’t want to spend money for every minute a job is running.
  • You are running a complicated job that requires a machine with better hardware than GitHub-hosted runner machines hardware

You’ll find more about self-hosted runners at GitHub documentation.

What’s an AWS spot instance

AWS offers a variety of instance types to deploy on your cloud, like on-demand, reserved, and spot instances.

All three instance types have the same functionality while running, and pricing is the only difference. An on-demand instance is an instance with no long-term commitments where you have to pay a certain per-second rate. A reserved instance is a rented instance for a specific period at a lower per-second rate than on-demand instances.

Unlike these two, A spot instance uses spare AWS capacity and has the lowest price among these instance types (up to 90% lower than the on-demand price). However, there’s a catch. Since you are using unused resources, your instance can be interrupted (with a 2-minute notification).

Try to avoid using spot instances for stateful applications, databases, or workloads containing sensitive information.

For stateless applications that run short-time tasks and can be interrupted such as application testing, CI, data analysis, image rendering, spot instances are the ideal choice.

So what are we going to do?

In this example, the workflow will validate the syntax of Python files in your repository using Flake8 and Pylint (but of course, any other workflow will also do).

You’ll set up an AWS spot instance to run a GitHub Actions self-hosted runner and configure it to redeploy if AWS interrupts it to increase the flexibility of Continuous Integration (CI) jobs at a minimal cost.

This self-hosted runner will execute the workflow jobs.

All the files we will use to deploy the spot instance can be found in this GitHub Repository under the aws-files directory. In this repository, you will also find an example of a GitHub Actions Workflow.

Walkthrough prerequisites:

Creating and configuring the Spot Instance

To create the spot instance with the correct configuration, you’ll only need 3 files.

  1. user-data.sh – a Bash script file contains the commands that will run on an instance at launch. In our case, the script will run on every instance deployed, even on a new one that replaces an interrupted spot machine. It contains a script that installs and configures a self-hosted runner and starts its service. (For more information about the user-data.sh file you can read in AWS documentation).

    github-user="Your GitHub Username"
    github-repo="Your GitHub Repository name"
    PAT="Your Super Secret PAT"
    # Download jq for extracting the Token
    yum install jq -y
    # Create and move to the working directory
    mkdir /actions-runner && cd /actions-runner
    # Download the latest runner package
    curl -o actions-runner-linux-x64-2.286.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.286.1/actions-runner-linux-x64-2.286.1.tar.gz
    # Extract the installer
    tar xzf ./actions-runner-linux-x64-2.286.1.tar.gz
    # Change the owner of the directory to ec2-user
    chown ec2-user -R /actions-runner
    # Get the runner's token
    token=$(curl -s -XPOST -H "authorization: token $PAT" https://api.github.com/repos/$github-user/$github-repo/actions/runners/registration-token | jq -r .token)
    # Create the runner and start the configuration experience
    sudo -u ec2-user ./config.sh --url https://github.com/<github-username>/$github-user --token $token --name "spot-runner-$(hostname)" --unattended
    # Create the runner's service
    ./svc.sh install
    # Start the service
    ./svc.sh start


  2. spot-instance-launch-template.json – a JSON file describes the spot instance’s launch template configuration (Image, Type, SecurityGroups, etc…). For our purposes, it will also contain the user-data.sh file contents encoded in base64.

      "ImageId": "ami-001089eb624938d9f",
      "InstanceType": "t2.micro",
      "KeyName": "instance key pair name",
      "SecurityGroups": ["security group name"],
      "UserData": "user-data.sh file content encoded in base64",
      "InstanceMarketOptions": {
          "MarketType": "spot",
          "SpotOptions": {
          "MaxPrice": "0.03",
          "SpotInstanceType": "one-time",
          "InstanceInterruptionBehavior": "terminate"

  3. spot-instance-auto-scaling-group.json – a JSON file describing the spot instance’s auto scaling group configuration such as what launch-template to use, the minimal and maximal number of instances, etc. In our scenario, the “auto scaling” is actually used to create a replacement for an interrupted spot machine and not for dynamically flexible growth.This is important because by using an auto scaling group, if a Spot Instance is interrupted and AWS terminates the instance, another spot instance with the same configuration will deploy automatically.
        "AutoScalingGroupName": "spot-instance-asg",
        "LaunchTemplate": {
        "LaunchTemplateName": "spot-instance-launch-template"
        "MinSize": 1,
        "MaxSize": 1,
        "AvailabilityZones": ["your Availability Zones"]

Yalla, let’s deploy

  1. Clone the repository and change directory to the essential files directory

    git clone https://github.com/omermishania/github-runner-on-aws-spot.git && cd github-runner-on-aws-spot/aws-files/


  2. Create an AWS security group
    Create a security group with these rules:
    Inbound – SSH (port 22)
    Outbound – HTTPS (port 443)
  3. Create user-data.sh file
    # Configure the file:Edit the file with your favorite text editor and change the values of the variables to your values when:

    • Github-user = your GitHub username
    • github-repo = your GitHub repository name
    • PAT = your GitHub PAT (personal access token)

      vim user-data.sh

    As an example:


    When you finish, save and exit the file.

    # Encode the content of user-data.sh file to base64


    cat user-data.sh | base64 -w 0

    # Copy the encoded content, and save it in a safe place.


  4. Creating spot-instance-launch-template.json file
    # Configure the fileEdit the file and change:

    • ‘ImageId’ value to an EC2 image ID of your choice.
      You can find the EC2 image ID next to the image name (in the red box):
    • ‘InstanceType’ value to your desired instance type
    • ‘SecurityGroups’ value to a list contains the security group name you created minutes ago
    • ‘UserData’ value to the base64 encoded content you copied
    • ‘MaxPrice’ value to the maximum price of the spot instance
    vim spot-instance-launch-template.json

    As an example:

      "ImageId": "ami-001089eb624938d9f",
      "InstanceType": "t2.micro",
      "SecurityGroups": ["gh-runner-spot-sg"],
      "InstanceMarketOptions": {
          "MarketType": "spot",
          "SpotOptions": {
          "MaxPrice": "0.03",
          "SpotInstanceType": "one-time",
          "InstanceInterruptionBehavior": "terminate"

    Run the command:When you finish,
    save and exit the file.

    Make sure you run this command only after you pasted the base64 encoded user-data.sh

    aws ec2 create-launch-template --launch-template-name spot-instance-launch-template --launch-template-data file://spot-instance-launch-template.json
  5. vim spot-instance-auto-scaling-group.json
  6. Creating spot-instance-auto-scaling-group.json file
    # Configure the fileEdit the file and change ‘AvailabilityZones’ value to a list containing your desired availability zone(s). You can also change the minimum and maximum instance count if you want to deploy more than one runner.

    vim spot-instance-auto-scaling-group.json

    When you finish, save and exit the file.

    # Run the command

    aws autoscaling create-auto-scaling-group --cli-input-json file://spot-instance-auto-scaling-group.json

    Congratulations, you have just created a self-hosted runner on your spot instance! 

Configure the GitHub Actions workflow to use a self-hosted runner

Our brand new self-hosted runner is up and running!
However, the GitHub Actions workflow is still not going to use it. To make that happen, we need to edit the workflow file and change the ‘runs-on’ value to ‘self-hosted’.

As an example, I used the following workflow to check for valid Python syntax. Your workflow can serve any purpose you would like, you only need to change the value of the runs-on field to ‘self-hosted’ (marked in blue):

name: Python Linting
       runs-on: self-hosted
       - uses: actions/checkout@v2
       - name: Install dependencies
       run: |
       python3 -m pip install --upgrade pip
       pip3 install flake8
       pip3 install pylint
       if [ -f requirements.txt ]; then pip3 install --target=/usr/bin -r requirements.txt; fi
       - name: Lint with flake8
       run: |
      /usr/local/bin/flake8 src --count --select=E9,F63,F7,F82 --show-source --statistics
      /usr/local/bin/flake8 src --count --max-complexity=10 --max-line-length=88 --statistics
      - name: Lint with Pylint
      run: |
      /usr/local/bin/pylint src

And That’s it!

Let’s recap what we have done so far:

  • Created and configured an auto-recovering Spot Instance.
  • Configured a GitHub actions runner on that instance.
  • Configure the GitHub Actions Workflow to use it.

The workflow now runs on your newly created GitHub Runner on a spot instance on your cloud, and in case a spot instance is interrupted and terminated, a new one with a new GitHub runner will be created.


By using a GitHub self-hosted runner on a spot instance on your cloud environment, you have the flexibility to use any custom configurations and tools that will serve exactly your CI purpose and even save unnecessary costs.

In our configuration, the spot instance needs to be alive in order to listen to GitHub runner commands. So we actually pay for a spot instance being live consistently(despite the cheap spot tariffs). One could think of another possible configuration where we use a GitHub hosted runner that only raises the spot instance and lets it do the work. This way we would pay for the GitHub hosted runner only for the time it was invoked and running, and also for the spot instance only for a limited amount of time. However, this requires a more complicated setup beyond the scope of this post.

In order to achieve auto flexibility of spot instances for CI builds without a need to manage them manually, you can use Incredibuild for cloud. Tune in next week to learn how! 

Static Code Analysis Webinar Banner

Amir Kirsh
Amir Kirsh reading time: 8 minutes minutes March 9, 2022
March 9, 2022

Table of Contents

Related Posts

8 minutes 8 Reasons Why You Need Build Observability

Read More  

8 minutes These 4 advantages of caching are a game-changer for development projects

Read More  

8 minutes What Level of Build Observability Is Right for You?

Read More