AWS Client VPN Endpoint with Terraform

17 Aug 2020

Reading time ~7 minutes

With the recent Bay Area rolling blackouts by PG&E, we decided to migrate our current VPN from our office to AWS. This should solve two issue that we are facing; one, when our internet goes out at the office; two, as I mentioned, when PG&E decides to cut power randomly! :P

List of requirements that we want from migrating to AWS Client VPN Endpoing:

Reliability
Ability to access our internal VPCs
Resolve private and public DNS entries
Static NAT IP for whitelisting
User certificate management

And of course we want to do this via Terraform as much as possible.

Pricing
Certificate Creation
Setup Client VPN Endpoint
- Module
- Environment
Managing User Certificate
Removing Certificate
Auditing
- Connections
- Certificates Matching Our Domain Name
AWS Client VPN Download

Pricing

So this is NOT free, there is a charge per connection for the full hour. More pricing information here.

Certificate Creation

Clone Easy RSA Git Repo.

$ git clone https://github.com/OpenVPN/easy-rsa.git

Initialize Public Key Infrastructure (PKI).

$ cd easy-rsa/easyrsa3
$ ./easyrsa init-pki
init-pki complete; you may now create a CA or requests.
Your newly created PKI dir is: /var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki

Build Certificate Authority.

$ ./easyrsa build-ca nopass
Using SSL: openssl LibreSSL 2.8.3
Generating RSA private key, 2048 bit long modulus
.................................................................+++
...............................+++
e is 65537 (0x10001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Common Name (eg: your user, host, or server name) [Easy-RSA CA]:aws-vpn.company.com

CA creation complete and you may now import and sign cert requests.
Your new CA certificate file for publishing is at:
/var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki/ca.crt

Build Server Certificate.

$ ./easyrsa build-server-full server.aws-vpn.company.com nopass
Using SSL: openssl LibreSSL 2.8.3
Generating a 2048 bit RSA private key
...............................+++
.........+++
writing new private key to '/var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki/easy-rsa-43480.72nxYw/tmp.WW2DKy'
-----
Using configuration from /var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki/easy-rsa-43480.72nxYw/tmp.MCadXI
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'server.aws-vpn.company.com'
Certificate is to be certified until Nov 16 21:10:53 2022 GMT (825 days)

Write out database with 1 new entries
Data Base Updated

Build Client Certificate.

$ ./easyrsa build-client-full client.aws-vpn.company.com nopass
Using SSL: openssl LibreSSL 2.8.3
Generating a 2048 bit RSA private key
...........................................................................................................................................................................................................+++
.................+++
writing new private key to '/var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki/easy-rsa-41705.PurLiw/tmp.4iQdhO'
-----
Using configuration from /var/folders/xf/st0zj2rn1pl1jtmkjzlnl82c0000gn/T/tmp.hBPssWon/easy-rsa/easyrsa3/pki/easy-rsa-41705.PurLiw/tmp.iNaWh1
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'client.aws-vpn.company.com'
Certificate is to be certified until Nov 16 21:08:51 2022 GMT (825 days)

Write out database with 1 new entries
Data Base Updated

Verify the certs and keys.

$ find . | egrep ".key|.crt"
./pki/ca.crt
./pki/issued/server.aws-vpn.company.com.crt
./pki/issued/client.aws-vpn.company.com.crt
./pki/private/server.aws-vpn.company.com.key
./pki/private/client.aws-vpn.company.com.key
./pki/private/ca.key

Upload to AWS Certificate Manager (ACM).

$ aws acm import-certificate --region us-east-1 --certificate fileb://pki/issued/server.aws-vpn.company.com.crt --private-key fileb://pki/private/server.aws-vpn.company.com.key --certificate-chain fileb://pki/ca.crt
$ aws acm import-certificate --region us-east-1 --certificate fileb://pki/issued/client.aws-vpn.company.com.crt --private-key fileb://pki/private/client.aws-vpn.company.com.key --certificate-chain fileb://pki/ca.crt

Backup CA cert/key in parameter store. We pull the CA crt/key from parameter store when we generate certificates for users.

Notice! These will be used when creating certificates for individual users.

$ aws ssm put-parameter --region us-east-1 --name /vpn/crt/ca --value "$(cat pki/ca.crt | base64)" --type SecureString
$ aws ssm put-parameter --region us-east-1 --name /vpn/key/ca --value "$(cat pki/private/ca.key | base64)" --type SecureString

Setup Client VPN Endpoint

We will do this via terraform.

The following setting will associate the Client VPN Endpoint to the central VPC’s central-backend subnet with destination of 0.0.0.0/0’s target nat that is associated with central-public (Elastic IP Address).

Module

main.tf

################################
###  Security Group
################################

resource aws_security_group "client-vpn-access" {
  name   = "terraform-shared-client-vpn-access"
  vpc_id = var.vpc_id

  ingress {
    from_port   = 0
    protocol    = "-1"
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    protocol    = "-1"
    to_port     = 0
    cidr_blocks = ["0.0.0.0/0"]
  }
}

################################
###  Cloudwatch Log Group
################################

resource "aws_cloudwatch_log_group" "cloudwatch-log-group" {
  name              = "client-vpn-endpoint-${var.domain}"
  retention_in_days = "30"

  tags = {
    Name        = var.domain
    Environment = "global"
    Terraform   = "true"
  }
}

################################
###  Client VPN Endpoint
################################

resource "aws_ec2_client_vpn_endpoint" "client-vpn-endpoint" {
  description            = "terraform-client-vpn-endpoint"
  server_certificate_arn = var.server_cert
  client_cidr_block      = var.client_cidr_block
  split_tunnel           = var.split_tunnel
  dns_servers            = var.dns_servers

  authentication_options {
    type                       = "certificate-authentication"
    root_certificate_chain_arn = var.client_cert
  }

  connection_log_options {
    enabled              = true
    cloudwatch_log_group = aws_cloudwatch_log_group.cloudwatch-log-group.name
  }

  tags = {
    Name        = var.domain
    Environment = "global"
    Terraform   = "true"
  }
}

resource "aws_ec2_client_vpn_network_association" "client-vpn-network-association" {
  count                  = length(var.subnet_id)
  client_vpn_endpoint_id = aws_ec2_client_vpn_endpoint.client-vpn-endpoint.id
  subnet_id              = element(var.subnet_id, count.index)
}

resource "aws_ec2_client_vpn_route" "client-vpn-route" {
  count                  = length(var.subnet_id)
  client_vpn_endpoint_id = aws_ec2_client_vpn_endpoint.client-vpn-endpoint.id
  destination_cidr_block = "0.0.0.0/0"
  target_vpc_subnet_id   = element(var.subnet_id, count.index)
  description            = "Internet Access"

  depends_on = [
    aws_ec2_client_vpn_endpoint.client-vpn-endpoint,
    aws_ec2_client_vpn_network_association.client-vpn-network-association
  ]
}

################################
###  Null Resource
################################

resource "null_resource" "authorize-client-vpn-ingress" {
  provisioner "local-exec" {
    when    = create
    command = "aws --region ${var.region} ec2 authorize-client-vpn-ingress --client-vpn-endpoint-id ${aws_ec2_client_vpn_endpoint.client-vpn-endpoint.id} --target-network-cidr 0.0.0.0/0 --authorize-all-groups"
  }

  lifecycle {
    create_before_destroy = true
  }

  depends_on = [
    aws_ec2_client_vpn_endpoint.client-vpn-endpoint,
    aws_ec2_client_vpn_network_association.client-vpn-network-association
  ]
}

resource null_resource "client-vpn-security-group" {
  provisioner "local-exec" {
    when    = create
    command = "aws ec2 apply-security-groups-to-client-vpn-target-network --client-vpn-endpoint-id ${aws_ec2_client_vpn_endpoint.client-vpn-endpoint.id} --vpc-id ${aws_security_group.client-vpn-access.vpc_id} --security-group-ids ${aws_security_group.client-vpn-access.id}"
  }

  lifecycle {
    create_before_destroy = true
  }

  depends_on = [
    aws_ec2_client_vpn_endpoint.client-vpn-endpoint,
    aws_security_group.client-vpn-access
  ]
}

variables.tf

variable "region" {
  default = "us-east-1"
}

variable "client_cidr_block" {
  description = "The IPv4 address range, in CIDR notation being /22 or greater, from which to assign client IP addresses"
  default     = "18.0.0.0/22"
}

variable "vpc_id" {
  description = "The ID of the VPC to associate with the Client VPN endpoint."
}

variable "subnet_id" {
  type        = list
  description = "The ID of the subnet to associate with the Client VPN endpoint."
}

variable "domain" {
  default = "aws-vpn.reflektion.com"
}

variable "server_cert" {
  description = "Server certificate"
}

variable "client_cert" {
  description = "Client/Root certificate"
}

variable "split_tunnel" {
  default = "false"
}

variable "dns_servers" {
  type = list
  default = [
    "8.8.8.8",
    "1.1.1.1"
  ]
}

output.tf

output "client_vpn_endpoint_id" {
  value = aws_ec2_client_vpn_endpoint.client-vpn-endpoint.id
}

Environment

########################
## Client VPN Endpoint
########################

module "client-vpn" {
  source            = "../../modules/aws_client_vpn_endpoint"
  region            = "us-east-1"
  client_cidr_block = "10.22.0.0/22"                   // 1024
  vpc_id            = "vpc-*****"                      // central
  subnet_id         = ["subnet-*****", "subnet-*****"] // central-backend w/ route table 0.0.0.0/0 which has central-public EIPs
  domain            = "aws-vpn.reflektion.com"
  server_cert       = "arn:aws:acm:us-east-1:*****:certificate/*****-12e7-4da8-836b-*****"
  client_cert       = "arn:aws:acm:us-east-1:*****:certificate/*****-12b7-4995-9f1e-*****"
  dns_servers       = ["10.16.0.2", "8.8.8.8"]
}

Managing User Certificate

Notice! This is sensitive/secure data, DO NOT share with anyone else! as they contain login credentials.

The following script will generate a .ovpn config file which then can be shared with user securely (via secure gist url).

#!/usr/bin/env bash

# This script generates certificate for client vpn
# AWS Client VPN Pricing: https://aws.amazon.com/vpn/pricing/

[ -z "$1" ] && echo "Usage: $0 first.last" && exit 1

export EASYRSA_BATCH=1

client="$1"
cert_domain="aws-vpn.company.com"
client_full="${client}.${cert_domain}"
outfile="/tmp/${client}-cvpn-endpoint.ovpn"
vpn_endpoint_id="cvpn-endpoint-*****"

tput setaf 2; echo ">> clone repo ..."; tput setaf 9
cd `mktemp -d`
git clone https://github.com/OpenVPN/easy-rsa.git
cd easy-rsa/easyrsa3

tput setaf 2; echo ">> init ..."; tput setaf 9
./easyrsa init-pki
./easyrsa build-ca nopass
rm -f pki/ca.crt
rm -f pki/private/ca.key

tput setaf 2; echo ">> get ca crt/key from param store ..."; tput setaf 9
(aws ssm get-parameters \
    --region us-east-1 \
    --names /vpn/crt/ca \
    --with-decryption \
    --query Parameters[0].Value \
    --output text || exit 2) | base64 --decode > pki/ca.crt
(aws ssm get-parameters \
    --region us-east-1 \
    --names /vpn/key/ca \
    --with-decryption \
    --query Parameters[0].Value \
    --output text || exit 3) | base64 --decode > pki/private/ca.key

tput setaf 2; echo ">> create and import crt/key for $client ..."; tput setaf 9
./easyrsa build-client-full ${client_full} nopass
aws acm import-certificate \
    --certificate fileb://pki/issued/${client_full}.crt \
    --private-key fileb://pki/private/${client_full}.key \
    --certificate-chain fileb://pki/ca.crt

tput setaf 2; echo ">> generate vpn client config file ..."; tput setaf 9
aws ec2 export-client-vpn-client-configuration \
    --client-vpn-endpoint-id $vpn_endpoint_id \
    --output text >| $outfile
sed -i~ "s/^remote /remote ${client}./" $outfile
echo "<cert>"                      >> $outfile
cat pki/issued/${client_full}.crt  >> $outfile
echo "</cert>"                     >> $outfile
echo "<key>"                       >> $outfile
cat pki/private/${client_full}.key >> $outfile
echo "</key>"                      >> $outfile

tput setaf 2; echo ">> $outfile ..."; tput setaf 9
ls -alh $outfile

Removing Certificate

When an employee leaves or no longer need VPN access, we simply just go to AWS console and remove their certificate from ACM.

Auditing

Assuming you have aws-utils cloned, you can use the vpnlookup script in the repo to do some basic auditing.

For our environment, we have to update ~/.aws-utils/vpnlookup.conf.

$ cat ~/.aws-utils/vpnlookup.conf
cvpn_endpoint_id="cvpn-endpoint-*****"
cvpn_cert_domain_name="aws-vpn.company.com"

Connections

$ vpnlookup conn | column -ts,
cvpn-connection-*****  cvpn-endpoint-*****  2020-08-13T21:36:04  2020-08-13T21:36:03  3075  2732  8  7  10.22.1.162  calvin.wong.aws-vpn.company.com  terminated  2020-08-13T21:36:04
cvpn-connection-*****  cvpn-endpoint-*****  2020-08-13T21:36:12  2020-08-13T21:36:07  3075  2732  8  7  10.22.0.2    calvin.wong.aws-vpn.company.com  terminated  2020-08-13T21:36:12
cvpn-connection-*****  cvpn-endpoint-*****  2020-08-13T21:36:17  2020-08-13T21:36:17  3075  2732  8  7  10.22.0.34   calvin.wong.aws-vpn.company.com  active      -

Certificates Matching Our Domain Name

$ vpnlookup cert | column -ts,
client.aws-vpn.company.com          arn:aws:acm:us-east-1:*****:certificate/*****-6d38-4b10-8d1c-*****
server.aws-vpn.company.com          arn:aws:acm:us-east-1:*****:certificate/*****-c706-4dc5-aa86-*****
erin.ewing.aws-vpn.company.com      arn:aws:acm:us-east-1:*****:certificate/*****-615e-4306-916b-*****
calvin.wong.aws-vpn.company.com     arn:aws:acm:us-east-1:*****:certificate/*****-adb3-40dc-a93d-*****
dennis.dalton.aws-vpn.company.com   arn:aws:acm:us-east-1:*****:certificate/*****-2056-459c-ad9c-*****
barry.bonds.aws-vpn.company.com     arn:aws:acm:us-east-1:*****:certificate/*****-562d-4792-a783-*****
alice.anderson.aws-vpn.company.com  arn:aws:acm:us-east-1:*****:certificate/*****-96ed-47c7-ba78-*****

AWS Client VPN Download

Mac and Windows user can download the desktop client from AWS and load the .ovpn config file.

For iPhone users, you can download the OpenVPN Connect app from the App Store.