Terraform and PagerDuty

04 Feb 2018

Reading time ~3 minutes

I am a fan of Terraform and have used it at the last and the current company. And being in ops, PagerDuty is practically becoming part of your life. While I was bored after my oral surgery, I decided to move our PagerDuty settings into Terraform. Since I don’t want to write out every team/user/schedule/service/escalation policy, I would have to modulize the resources. There aren’t a lot of docs online for Terraform+PagerDuty other than this page. But his examples are not in modules, so I gave a crack at it!

You can clone the repo from my gitlab account.

Modules

User
Schedule
Service
Service Integration

Environment

backend.tf
teams.tf
users.tf
team-techops.tf

Modules User

This module sets up the user, and contact infomations like: email, role, job title, teams, mobile.

########################
## user
########################

resource "pagerduty_user" "pd_user" {
  name      = "${var.name}"
  email     = "${var.email}"
  role      = "${var.role}"
  job_title = "${var.job_title}"
  teams     = ["${var.teams}"]
}

resource "pagerduty_user_contact_method" "pd_contact_phone" {
  user_id      = "${pagerduty_user.pd_user.id}"
  type         = "phone_contact_method"
  country_code = "+1"
  address      = "${var.mobile}"
  label        = "Mobile"
}

resource "pagerduty_user_contact_method" "pd_contact_sms" {
  user_id      = "${pagerduty_user.pd_user.id}"
  type         = "sms_contact_method"
  country_code = "+1"
  address      = "${var.mobile}"
  label        = "Mobile"
}

Modules Schedule

This module sets up the schedule for a given user. It defines when on-call starts, etc.

########################
## schedule
########################

resource "pagerduty_schedule" "pd_schedule" {
  name      = "${var.name}"
  time_zone = "${var.time_zone}"

  layer {
    name                         = "${var.name}"
    start                        = "${var.oncall_start}"
    rotation_virtual_start       = "${var.rotation_virtual_start}"
    rotation_turn_length_seconds = "${var.rotation_length}"
    users                        = ["${var.users}"]
  }
}

Modules Service

This module sets up the service.

########################
## service
########################

resource "pagerduty_service" "pd_service" {
  name                    = "${var.name}"
  auto_resolve_timeout    = "${var.auto_resolve_timeout}"
  acknowledgement_timeout = "${var.acknowledgement_timeout}"
  escalation_policy       = "${var.escalation_policy}"
  alert_creation          = "${var.alert_creation}"
}

Modules Service Integration

This is third party integrations into the service, such as: New Relic, Slack, CloudWatch, Zendesk, etc.

########################
## service integration
########################

data "pagerduty_vendor" "pd_vendor" {
  name = "${var.name}"
}

resource "pagerduty_service_integration" "pd_service_integration" {
  name              = "${data.pagerduty_vendor.pd_vendor.name}"
  vendor            = "${data.pagerduty_vendor.pd_vendor.id}"
  service           = "${var.service}"
  integration_email = "${var.email}"
}

backend.tf

We store our tfstate file on S3. Just run terraform init after creating this file.

terraform {
  backend "s3" {
    bucket  = "YOUR_BUCKET_NAME"
    key     = "tfstate/pagerduty"
    region  = "us-east-1"
    encrypt = "true"
  }
}

teams.tf

Teams are defined here. I’ve also included a variable for each team by names to be used.

########################
## variable
########################

variable "team_techops_sre" {
  default = "TechOps SRE"
}

variable "team_customer_success" {
  default = "Customer Success"
}

########################
## team
########################

resource "pagerduty_team" "pd_team_techops" {
  name = "TF Test - Team ${var.team_techops_sre}"
}

resource "pagerduty_team" "pd_team_cs" {
  name = "TF Test - Team ${var.team_customer_success}"
}

users.tf

Defining each user here by calling the user module.

########################
## user
########################

module "user_calvin_wong" {
  source    = "../../modules/pagerduty/user"
  name      = "TF Test - Calvin"
  email     = "YOUR_EMAIL_HERE"
  mobile    = "YOUR_MOBILE_HERE"
  job_title = "SRE"
  teams     = "${pagerduty_team.pd_team_techops.id}"
}

team-techops.tf

This is the main file for each team. We call the modules to create the following:

Schedule(s)
Escalation Policies
Service(s)
service Integration(s) (module not as flexible, creating as resources here instead)

########################
## schedule
########################

module "schedule_techops_sre_01" {
  source = "../../modules/pagerduty/schedule"
  name   = "TF Test - ${var.team_techops_sre} OnCall Level 1"

  users = [
    "${module.user_calvin_wong.id}",
    "PPM4NW9",
  ]
}

module "schedule_techops_sre_02" {
  source = "../../modules/pagerduty/schedule"
  name   = "TF Test - ${var.team_techops_sre} OnCall Level 2"

  users = [
    "PPM4NW9",
    "${module.user_calvin_wong.id}",
  ]
}

########################
## escalation policy
########################

resource "pagerduty_escalation_policy" "escalation_policy_techops_sre_01" {
  name      = "TF Test - ${var.team_techops_sre} Escalation Policy"
  num_loops = 2
  teams     = ["${pagerduty_team.pd_team_techops.id}"]

  rule {
    escalation_delay_in_minutes = 15

    target {
      type = "schedule_reference"
      id   = "${module.schedule_techops_sre_01.id}"
    }
  }

  rule {
    escalation_delay_in_minutes = 15

    target {
      type = "schedule_reference"
      id   = "${module.schedule_techops_sre_02.id}"
    }
  }
}

########################
## service
########################

module "service_techops_sre_01" {
  source            = "../../modules/pagerduty/service"
  name              = "TF Test - Prod ${var.team_techops_sre} - High Urgency"
  escalation_policy = "${pagerduty_escalation_policy.escalation_policy_techops_sre_01.id}"
}

########################
## service integration
########################

module "service_integration_techops_sre_01" {
  source  = "../../modules/pagerduty/service_integration"
  name    = "New Relic"
  service = "${module.service_techops_sre_01.id}"
}

module "service_integration_techops_sre_02" {
  source  = "../../modules/pagerduty/service_integration"
  name    = "Zendesk"
  service = "${module.service_techops_sre_01.id}"
  email   = "tf-test-calvin-wong-zendesk@YOUR_COMPANY_NAME.pagerduty.com"
}

module "service_integration_techops_sre_03" {
  source  = "../../modules/pagerduty/service_integration"
  name    = "Email"
  service = "${module.service_techops_sre_01.id}"
  email   = "tf-test-calvin-wong-email@YOUR_COMPANY_NAME.pagerduty.com"
}

module "service_integration_techops_sre_04" {
  source  = "../../modules/pagerduty/service_integration"
  name    = "CloudWatch"
  service = "${module.service_techops_sre_01.id}"
}

module "service_integration_techops_sre_05" {
  source  = "../../modules/pagerduty/service_integration"
  name    = "Slack"
  service = "${module.service_techops_sre_01.id}"
}