At Pixability, we primarily leverage Ansible for provisioning infrastructure — however not too long ago, we began to discover Terraform as properly. Both programs may be understood as agent-less provisioning instruments (like Puppet, for instance). Ansible supplies extra of an crucial language strategy whereby one writes down the steps in a process that, when executed, ends in a desired objective. Terraform takes a extra useful strategy (with generally shocking uncomfortable side effects), whereby one describes the objective and the software computes a plan for reaching the objective.

Terraform is an Infrastructure-as-Code (IaC) software created by HashiCorp. Infrastructure-as-Code pertains to an strategy to managing infrastructure that makes use of versioning and launch practices most generally related to software program artifacts. The rise of virtualization and cloud-based suppliers corresponding to AWS has led to infrastructure turning into “programmable,” and thus extra akin to software program than .

While each instruments fall into the class of infrastructure provisioning, they’re each additionally able to deploying functions. Ansible supplies a big library of duties, together with launching EC2 situations; managing packages; and creating Docker containers, ECS duties, and providers. In distinction, Terraform is targeted virtually solely on offering infrastructure because it pertains to cloud deployment. But it too affords each a Docker supplier and the technique of immediately managing ECS duties and providers by way of its AWS supplier.

As DevOps engineers, our objective is the dependable and secure supply of programs, so the excellence between a part of infrastructure and an utility shouldn’t be significantly helpful. What is helpful is the truth that parts of our programs have distinct life cycle patterns. Some, like our community layer, have a protracted lifespan and alter not often — however a microservice ReST API software program part could change a number of occasions per day.

The Zen of Terragrunt

“Tranquil Zen gardens” is licensed below CC BY 2.0

Terraform offers us the means of making a library of modules which are functionally similar to features. A couple of cumbersome particulars of this course of are dealt with with considerably extra grace by Terragrunt — a skinny wrapper round Terraform that facilitates its use, improves group of code and configuration, reduces repetition, and likewise supplies a method to layer parts.

Terragrunt, along with Terraform S3-based state storage, encourages us to make use of a folder construction that locations the Terraform modules below one hierarchy (or a set of SCM repositories), and the configurations that outline features “name” to the modules below one another.

Configurations are organized below the


folder hierarchy, the construction of which is printed beneath:


Terragrunt robotically handles initialization and state storage backend configuration. It additionally helps us keep away from repeating the identical configuration statements in all of our module calls, enabling a typical configuration in a mother or father terraform.tfvars file. In its regular operation recordsdata named .tfvars are scanned by Terraform as a part of loading a module, and processed as variable definitions. In Terragrunt, we will configure a name to, for instance, a module named config, within the terraform.tfvars of the leaf folder (we name these leaf folders “pods”). When Terragrunt processes this configuration, it would additionally search up the hierarchy for a mother or father terraform.tfvars file, which it finds within the aws folder and which accommodates shared settings for configuring state storage.

terragrunt = 
  # (1) Define terraform entry level
    extra_arguments "retry_lock" 
      instructions  = ["$"]
      arguments = ["-lock-timeout=20m"]
    extra_arguments "custom_vars" 
      instructions = ["$"]
      required_var_files = [
      optional_var_files = [

  # (2) Configure Terragrunt to robotically retailer tfstate recordsdata in an S3 bucket
  remote_state = 
    backend = "s3"
      bucket         = "pix-terraform"
      key            = "$path_relative_to_include()/terraform.tfstate"
      area         = "us-east-1"
      dynamodb_table = "terraform_locks"

The terragrunt block is the place we outline (1) variable recordsdata handed on terraform command line, and (2) terraform state storage backend configuration.

In block (1) we specify that the mother or father terraform.tfvars will provide variables (which may be overridden by subsequent variable recordsdata) as will recordsdata named frequent.tfvars showing in both the mother or father or grandparent of the module listing, and eventually the module listing terraform.tfvars itself.

Block (2) configures using an AWS S3 bucket because the state storage again finish. The S3 key prefix for the terraform.tfstate of a module is derived from the relative path between the mother or father terraform.tfvars and the module itself. In our config instance, the S3 key can be


and the S3 storage location of the module state is


The mechanism employed to reference the state of different parts relies on finding S3 state storage via the worldwide variables:




We declare the previous within the top-level mother or father tfvars, whereas the latter is asserted within the frequent.tfvars belonging to the atmosphere during which the deployed part lives. For our config instance, the atmosphere is




furnishes the S3 state_prefix path part:

state_prefix = "us-east-1/vpca"

The leaf folders (pods) within the /terraform/environments configuration hierarchy is the place we declare a name to certainly one of our modules. As an instance, beneath is the terraform.tfvars for the config pod:

terragrunt = 

vpc_name = "pix-vpc-a"
vpc_short_name = "vpca"

The terragrunt block is the place we inform Terragrunt to seek for our mother or father terraform.tfvars, and likewise the place we declare our module supply, and, optionally, any dependencies on different pods. The the rest of the file consists of plain previous HCL variable assignments corresponding to can be present in any .tfvars file.

In this instance we use a relative path for the terraform supply module however we will additionally use an SCM URL. An SCM reference can specify a selected model of the module supply. In growth environments we frequently use a relative path (or the –terragrunt-source possibility on our terragrunt command line) to simplify growing terraform modules themselves. In manufacturing environments we use versioned SCM URLs to extra tightly management precisely what model we’re deploying.

Modules are organized below the /terraform/modules folder hierarchy:


Each module defines enter variables, output variables, and the assets it builds. A module’s inputs and outputs outline its interface, or contract, with each its direct callers and with different parts that make use of it. Other parts, both in the identical or greater layers, could make use of the assets created via a module invocation by way of information references to its distant state. As such this enter/output contract must be managed with the identical self-discipline and course of as that of any shared software program part. The output contract, particularly, represents the interface via which parts of a system could make use of one another — that’s, via which a service we want to deploy could make use of, for instance, an AWS occasion or an utility load balancer provisioned in a unique layer of the general system.

The config module will get deployed to any variety of environments by defining calls or configurations below our


hierarchy. Other modules can reference the config pod inside the similar atmosphere utilizing a code fragment of the shape:

information "terraform_remote_state" "config" 
backend = "s3"
bucket = "$"
key = "$var.state_prefix/config/terraform.tfstate"
area = "$"

When that module must discuss with a datum provided by the config pod, it merely makes use of expressions of the shape


How This Helps Pixability

While we’re simply getting began with Terraform at Pixability, we consider that the mix of Terraform and Terragrunt will allow us to construct out advanced but simple-to-manage infrastructure and deployments for a while to return. We are already constructing ECS clusters for our UI functions and we’ve begun utilizing these instruments and strategies to provision and orchestrate Zookeeper, Kafka, and Connect parts comprising the premise of a brand new information pipeline — and the early outcomes are fairly promising.

The submit Terraforming Pixability appeared first on .

This article sources info from Tech Blog