At DEPT, we love pushing the envelope of new technologies and tooling to deliver best-in-class products and solutions that delight our clients.

We use Terraform and Terragrunt to build and provision as much as we possibly can, and recently we've begun leveraging GitHub Workflows for CI/CD. In this post, we'll take a look at how we combine these tools to create a GitOps centric workflow for managing cloud infrastructure.

Specifically we'll:

Define GitOps
Review a sample Terraform app module repo and an associated GitHub Workflow to lint and version the module repo
Review a sample Terragrunt live repo and a GitHub Workflow to apply infrastructure changes

GitOps

So what's GitOps?

The fundamentals are pretty straight forward:

Git as the single source of truth of a system
Git as the single place where we create, change and destroy all environments
All changes are observable / verifiable

As we'll see below, combining Terraform, Terragrunt and GitHub Workflows is as GitOps as it gets.

Terraform

Terraform is a declarative, cloud agnostic tool for provisioning immutable infrastructure.

Terraform modules are a fundamental component. Any set of Terraform configuration files in a folder is a module. That's it.

Within a module, you leverage providers (AWS is a provider) to create resources (EC2 is a resource). Dynamic configuration data is defined in variables (EC2 instance class is a variable) and provisioners can be used to execute specific actions (like installing and configuring software) on hosts to prepare them for service. Each of these are present in the sample repo.

Here's our sample module repo:

https://github.com/rocketinsights/terraform-blog-sample-module

This module leverages the AWS provider to create the following resources:

VPC
Load Balancer
Security Groups
EC2 instance
Installs/starts nginx on the EC2 instance (via a provisioner)

Linting and Versioning with a GitHub Workflow

In addition to the Terraform module code, we also have two GitHub Workflows defined; one for pull requests and one for merges to master.

The pull request workflow lints and validates formatting of the Terraform code:

	name: Lint and Validate Terraform Code

	on:
	pull_request:
	branches:
	- master
	jobs:
	build:
	runs-on: ubuntu-latest
	env:
	AWS_DEFAULT_REGION: us-east-1
	steps:
	- uses: actions/checkout@v1
	- name: Install Terraform and Terragrunt
	run: \|
	brew tap rocketinsights/tgenv
	brew install tfenv tgenv
	tfenv install
	tgenv install

	- name: Get Versions
	run: \|
	terragrunt --version
	terraform --version

	- name: Terraform Init
	run: find . -type f -name "*.tf" -exec dirname {} \;\|sort -u \| while read m; do (cd "$m" && terraform init -input=false -backend=false) \|\| exit 1; done

	- name: Validate Terraform configs
	run: find . -name ".terraform" -prune -o -type f -name "*.tf" -exec dirname {} \;\|sort -u \| while read m; do (cd "$m" && terraform validate && echo "√ $m") \|\| exit 1 ; done

	- name: Check Terraform config formatting
	run: terraform fmt -write=false -recursive

view raw github-workflow-lint-terraform.yml hosted with ❤ by GitHub

The master workflow uses a special action to create and apply an auto-incremented SemVer version tag:

	name: Apply/Increment Tag
	on:
	push:
	branches:
	- master
	jobs:
	build:
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@master
	with:
	fetch-depth: '0'
	- name: Bump version and push tag
	uses: anothrNick/github-tag-action@1.19.0
	env:
	GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	WITH_V: true

view raw github-workflow-apply-tag.yml hosted with ❤ by GitHub

Why do we care about versioning? Enter Terragrunt...

Terragrunt

Terragrunt is thin wrapper around Terraform.

It makes writing DRY Terraform modules easy and facilitates targeted control over when and where module updates are deployed.

As discussed above, our sample Terraform module is in its own repo and every time we merge to master, we automatically get a SemVer tag thanks to our fancy GitHub Workflow.

Now let's look at our Terragrunt live sample repo:

https://github.com/rocketinsights/terraform-blog-sample-live

This repo contains our Terragrunt configuration files for each environment; dev, staging and prod. Each environment specific Terragrunt file references our module at a specific version and sets any module variables required as inputs.

Because we are pinning each environment to a specific version of the module, we can make changes without affecting any running environment.

When we're ready to deploy a module change to a given environment, we simply increment the version tag in the source reference defined in the environment specific Terragrunt file. (This is usually the "ah-ha!" moment.)

Ok, great, but how do we continuously deploy this across different environments in AWS?

Applying infrastructure changes with a Github Workflow

Just as we saw in our module repo, our Terragrunt live repo also has two GitHub Workflows defined - one for pull requests and another for merges to master.

The pull request workflow outputs the plan (think dry run) of what Terragrunt is going to do (what AWS resources Terraform intends to create/update/delete):

	name: Terragrunt Plan All (dry run)
	on:
	pull_request:
	branches:
	- master
	jobs:
	build:
	runs-on: ubuntu-latest
	env:
	AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
	AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
	AWS_DEFAULT_REGION: us-east-1
	steps:
	- uses: actions/checkout@v1
	- name: Install Terraform and Terragrunt
	run: \|
	brew tap rocketinsights/tgenv
	brew install tfenv tgenv
	tfenv install
	tgenv install

	- name: Get Versions
	run: \|
	terragrunt --version
	terraform --version

	- name: Setup infra modules deploy key
	run: \|
	mkdir ~/.ssh
	echo "${{ secrets.INFRA_MODULES_DEPLOY_KEY }}" > ~/.ssh/id_rsa
	chmod 600 ~/.ssh/id_rsa
	ssh-keyscan -t rsa github.com

	- name: Terragrunt plan-all
	run: terragrunt plan-all

view raw github-workflow-terragrunt-plan-all.yml hosted with ❤ by GitHub

The master workflow takes that plan and applies it (actually creates/updates/deletes the resources in AWS based on the plan):

	name: Terragrunt Apply All (deploy)
	on:
	push:
	branches:
	- main
	jobs:
	build:
	runs-on: ubuntu-latest
	env:
	AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
	AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
	AWS_DEFAULT_REGION: us-east-1
	steps:
	- uses: actions/checkout@v1
	- name: Install Terraform and Terragrunt
	run: \|
	brew tap rocketinsights/tgenv
	brew install tfenv tgenv
	tfenv install
	tgenv install

	- name: Get Versions
	run: \|
	terragrunt --version
	terraform --version

	- name: Setup infra modules deploy key
	run: \|
	mkdir ~/.ssh
	echo "${{ secrets.INFRA_MODULES_DEPLOY_KEY }}" > ~/.ssh/id_rsa
	chmod 600 ~/.ssh/id_rsa
	ssh-keyscan -t rsa github.com

	- name: Terragrunt plan-all
	run: terragrunt plan-all

	- name: Terragrunt apply-all
	run: terragrunt apply-all --terragrunt-non-interactive

view raw github-workflow-terragrunt-apply-all.yml hosted with ❤ by GitHub

This downloads the module from the source reference in the terraform block of the Terragrunt configuration file, sets the inputs (which correlate to the module variables) and creates the resources in AWS.

Secrets

Something to take note of regarding these workflows is their use of secrets.

From the GitHub:

libsodium sealed box

Once you create a secret, it can only be decrypted in the workflow (you can't decrypt it in the UI or via the GitHub API) and GitHub automatically redacts secrets printed to the log.

Let's look at each secret we leverage:

INFRA_MODULES_DEPLOY_KEY:
Terragrunt requires the source reference for a private GitHub repo to use the ssh:// format.

To accommodate this, we create a GitHub deploy key under the module repo and add the private key as a secret in the Terragrunt live repo. We then render the private key so it can be used when Terragrunt is called. (Our example repos are public so this is would ONLY be required if you were using private GitHub repos.)

AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY:
These are the keys for the IAM user used by Terragrunt/Terraform to provision resources in AWS.

Final thoughts

Terraform, Terragrunt and GitHub Workflows are incredibly powerful tools that work extremely well together to facilitate a GitOps continuous delivery model.

At DEPT, we've used patterns like this to great effect, resulting in resilient and reliable workflows for efficiently replicating cloud infrastructure and application delivery pipelines on client projects.

We are always looking to work with both existing and new clients to apply these patterns to help evolve and adopt modern, robust and reliable patterns to their application delivery and cloud infrastructure management processes.

If you are interested in learning more about how we can help your organization or you'd like to work with us to apply these patterns to client projects, please let us know!

Subscribe to DEPT® Engineering Blog