Terraform and Github Actions without AWS Credentials

24 Feb 2022

Reading time ~4 minutes

When we switched to Github Actions at work, we were storing the AWS credentials in Github secrets. The problem is that when we have to rotate the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, we had to go thru all the repositories and that became a tedious task.

The solution was to switch to use OpenID Connect which was general available as of Nov 23, 2021. There are lots of documentations online on setting this up but I was only able to find them in CloudFormation, CDK, manually. As we’re a Terraform shop, we decided go with that route. Hope this is what you’re looking for as well.

AWS Services with Terraform
Github Actions YAML
Logging and Auditing
References

AWS Services with Terraform

Some of the environment variables we will need are the following:

variable "environment" {
  default = "qa"
}

variable "region" {
  default = "us-west-2"
}

variable "github_url" {
  default = "https://token.actions.githubusercontent.com"
}

We store our Github organization name in aws-parameter-store, and the thumbprint_list will be taken from the github certificate.

data "tls_certificate" "github" {
  url = var.github_url
}

data "aws_ssm_parameter" "ssm_devops_github_organization" {
  name = "/devops/github/ORGANIZATION"
}

IAM Identity Provider

The following values are provided by Github.

resource "aws_iam_openid_connect_provider" "github_actions" {
  url             = var.github_url
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.github.certificates.0.sha1_fingerprint]

  tags = {
    name        = "github"
    environment = var.environment
    infra       = "terraform"
  }
}

IAM Role

We use the terraform-aws-modules/iam-assumable-role-with-oidc module to setup the role with oidc support. The repo: lists in oidc_subjects_with_wildcards will be permitted to assume this role. role_policy_arns are the list of policy this role will assume.

Notice! AWS only allow 10 policies to be attached to a role. Depending on how your organization sets up Github-Actions, you might have to plan this out accordingly.

You can request the managed policies per role to max of 20 via Service Quotas.

module "iam_assumable_role_github_actions" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
  version = "~> 4.3"

  create_role      = true
  role_name        = "${var.environment}-github-actions"
  role_description = "Role for Github Actions"

  provider_url                   = var.github_url
  oidc_fully_qualified_audiences = ["sts.amazonaws.com"]
  oidc_subjects_with_wildcards   = [
    "repo:${data.aws_ssm_parameter.ssm_devops_github_organization.value}/devops-bastion:*",
    "repo:${data.aws_ssm_parameter.ssm_devops_github_organization.value}/devops-tools:*",
    "repo:${data.aws_ssm_parameter.ssm_devops_github_organization.value}/trainer-app:*",
    "repo:${data.aws_ssm_parameter.ssm_devops_github_organization.value}/recommendation-api:*",
  ]

  role_policy_arns = [
    "arn:aws:iam::aws:policy/AmazonRoute53FullAccess",
    "arn:aws:iam::aws:policy/AmazonS3FullAccess",
    "arn:aws:iam::aws:policy/CloudWatchLogsFullAccess",
    "arn:aws:iam::aws:policy/ReadOnlyAccess",
    "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess",
    "arn:aws:iam::aws:policy/AmazonECS_FullAccess",
    "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole",
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy",
    module.iam_policy_github_actions.arn,
  ]

  tags = {
    name        = "github"
    environment = var.environment
    infra       = "terraform"
  }
}

IAM Policy

Because AWS only allow 10 policies to be attached to a role, we could create a custom policy and add all the permissions here. In this example i granted dynamodb:ListTables and dynamodb:Scan.

module "iam_policy_github_actions" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-policy"
  version = "~> 4.3"

  name        = "${var.environment}-github-actions"
  path        = "/"
  description = "Managed by Terraform"

  policy = data.aws_iam_policy_document.github_actions.json

  tags = {
    name        = "github"
    environment = var.environment
    infra       = "terraform"
  }
}

data "aws_iam_policy_document" "github_actions" {
  statement {
    actions = [
      "dynamodb:ListTables",
      "dynamodb:Scan"
    ]

    resources = ["*"]
  }
}

Github Actions YAML

We need to have the permissions: block in the yaml file in order for this to work. For more info, see Automatic token authentication.

Permissions:

id-token: write: Required for authentication with OpenID.
contents: write: We set this to write because we use actions/create-update-tag to create a git tag on the branch.
actions: read: If you plan on using actions/action-slack, you will need to set it to read.
pull-requests: write: If you plan on using actions/github-script, you will need to set it to write.

permissions:
  id-token: write
  contents: write
  actions: read
  pull-requests: write

Using the role-to-assume: option from actions/configure-aws-credentials-action-for-github-actions with the role we created earlier via Terraform.

- name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID_QA }}:role/qa-github-actions
          aws-region: ${{ env.AWS_DEFAULT_REGION }}

The final result of the yaml file will look something like this.

---
name: aws-authentication

on:
  push:
    branch:
      - main
    paths-ignore:
      - '.github/workflows/**'

env:
  AWS_ACCOUNT_ID_QA: 123456789012
  AWS_DEFAULT_REGION: us-west-2
  AWS_DEFAULT_OUTPUT: json

permissions:
  id-token: write
  contents: write
  actions: read

jobs:
  aws-authentication:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Configure AWS credentials [QA]
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::${{ env.AWS_ACCOUNT_ID_QA }}:role/qa-github-actions
          aws-region: ${{ env.AWS_DEFAULT_REGION }}

      - name: Run shell commands
        run: |
          aws sts get-caller-identity
          env | grep AWS | sort

Logging and Auditing

Activity logs are being sent to Cloudtrail.

$ aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRoleWithWebIdentity | jq -r '.Events[0].CloudTrailEvent' | jq -r
{
  "userIdentity": {
    "type": "WebIdentityUser",
    ...
    "userName": "repo:organizationName/repositoryName:ref:refs/heads/...",
    "identityProvider": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
  },
  ...
  "eventSource": "sts.amazonaws.com",
  "eventName": "AssumeRoleWithWebIdentity",
  "awsRegion": "us-west-2",
  ...
  "requestParameters": {
    "durationSeconds": 3600,
    "roleArn": "arn:aws:iam::123456789012:role/qa-github-actions",
    "roleSessionName": "GitHubActions"
  ...
}

References