Lesson 09 intermediate

Terraform Expressions and Functions

Write conditionals, for expressions, splats and templates in HCL, use the built-in functions you need most, experiment safely in terraform console and use newer additions such as provider-defined functions, templatestring and convert.

HCL is not a programming language: there are no loops you write step by step and no variables you reassign. What it has is expressions, small formulas that compute a value from other values. Combined with about a hundred built-in functions, they cover almost everything a configuration needs: naming conventions, merging settings, filtering maps and rendering config files.

Try everything in terraform console

Before writing an expression into a resource, test it. terraform console evaluates expressions against your configuration and state:

$ terraform console
> upper("orbit")
"ORBIT"
> [for q in ["emails", "exports"] : "worker-${q}"]
[
  "worker-emails",
  "worker-exports",
]
> var.environment
"dev"
> exit

Since Terraform 1.16, terraform console -scope=module.cache evaluates inside a module, so you can inspect its locals and variables.

Strings and templates

Interpolation puts values into strings with ${...}:

locals {
  name_prefix = "orbit-${var.environment}"
}

Heredocs hold multi-line text, and %{ } directives add conditions and loops inside a template:

locals {
  nginx_conf = <<-EOT
    upstream orbit_api {
    %{ for name in local.api_hosts ~}
      server ${name}:9090;
    %{ endfor ~}
    }
    server {
      listen 80;
      location /api/ { proxy_pass http://orbit_api; }
    }
  EOT
}

The ~ trims the newline next to the directive so the output has no blank lines. <<-EOT (with the dash) removes the common indentation, so the template can be indented with the code.

For anything longer than a few lines, keep the template in its own file and render it with templatefile():

resource "docker_container" "web" {
  # ...
  upload {
    file    = "/etc/nginx/conf.d/default.conf"
    content = templatefile("${path.module}/templates/nginx.conf.tftpl", {
      api_hosts = [for k, c in docker_container.api : c.name]
    })
  }
}

Terraform 1.9 added templatestring(), which renders a template stored in a string you got from somewhere else, such as a file fetched by a data source.

Conditionals

The only conditional is the ternary condition ? if_true : if_false. Both sides must have compatible types.

locals {
  is_prod      = var.environment == "prod"
  worker_count = local.is_prod ? 4 : 1
  log_level    = local.is_prod ? "warn" : "debug"
}

for expressions

A for expression transforms one collection into another. Square brackets build a list (or tuple), curly braces build a map (or object):

locals {
  queues = ["emails", "thumbnails", "exports"]

  # list -> list
  container_names = [for q in local.queues : "orbit-worker-${q}"]

  # list -> map
  queue_ports = { for i, q in local.queues : q => 9100 + i }

  # filter with if
  heavy_workers = { for name, w in var.workers : name => w if w.memory_mb > 512 }

  # list of objects -> map keyed by name, ready for for_each
  workers_by_name = { for w in var.worker_list : w.name => w }
}

The last pattern is the standard answer to “for_each does not accept a list of objects”.

If two items produce the same map key, Terraform raises an error. Add ... after the value to group them into lists instead: { for w in var.worker_list : w.team => w.name... }.

Splat expressions

A splat is a shortcut for the most common for expression, reading one attribute from every element:

output "worker_ids" {
  value = docker_container.worker[*].id      # same as [for c in docker_container.worker : c.id]
}

Splat works on lists, including resources with count. For resources with for_each (which are maps), use values(docker_container.worker)[*].id or a for expression.

The functions you will use most

FunctionWhat it doesExample result
merge(a, b)Combines maps, later ones winmerge({a=1}, {a=2,b=3}) gives {a=2,b=3}
lookup(map, key, default)Read a key with a fallbacklookup(var.sizes, "prod", "small")
coalesce(a, b, ...)First value that is not null or emptycoalesce(var.name, "orbit")
join(sep, list) / split(sep, str)List to string and backjoin(",", ["a","b"]) gives "a,b"
format(fmt, ...)printf-style formattingformat("%s-%03d", "web", 7) gives "web-007"
length(x)Size of a list, map or stringlength(var.queues)
contains(list, value)Membership testcontains(["dev","prod"], var.environment)
toset(list) / tomap(obj)Type conversionstoset(["a","a","b"]) gives a set of 2
jsonencode(v) / yamlencode(v)Value to JSON/YAML textpolicies, config files
jsondecode(s) / yamldecode(s)Text to valuereading config files
file(path) / templatefile(path, vars)Read or render a filefile("${path.module}/motd.txt")
cidrsubnet(prefix, bits, n)Carve subnetscidrsubnet("10.20.0.0/16", 8, 1) gives "10.20.1.0/24"
try(expr, fallback)First expression that does not errortry(var.settings.port, 8080)
can(expr)true if the expression worksuseful in validation rules

Converting types explicitly with convert

Terraform converts types automatically most of the time. Where it cannot guess, Terraform 1.15 adds convert(value, type). The classic case is an empty collection: [] is an empty tuple and {} an empty object, which do not always match a list(string) or map(string) on the other side of a conditional:

locals {
  extra_env = var.debug ? ["LOG_LEVEL=debug", "TRACE=1"] : convert([], list(string))
}

convert also fills in optional() defaults of object types, which makes it handy for building complete default settings in one line.

Provider-defined functions

Since Terraform 1.8, providers can ship their own functions. They are called with a provider::<name>::<function> prefix and need no resources at all:

locals {
  role_arn   = "arn:aws:iam::123456789012:role/orbit-deployer"
  role_parts = provider::aws::arn_parse(local.role_arn)
}

output "deployer_account" {
  value = local.role_parts.account_id   # "123456789012"
}

The AWS provider has ARN helpers, the Kubernetes provider can decode manifests, and many others add their own. The provider must be listed in required_providers.

Guarding assumptions with conditions

Expressions also power checks. A precondition stops the plan if an assumption is false; a postcondition checks what a resource or data source returned:

resource "docker_container" "web" {
  # ...
  lifecycle {
    precondition {
      condition     = var.web_port >= 1024 || var.environment == "prod"
      error_message = "Only prod may bind a privileged port below 1024."
    }
  }
}

Quick reference

TaskSyntax
Interpolate"orbit-${var.environment}"
Conditionallocal.is_prod ? 4 : 1
Transform a list[for q in var.queues : upper(q)]
Build a map{ for w in var.list : w.name => w }
Filter[for q in var.queues : q if q != "emails"]
One attribute from all (count)docker_container.worker[*].id
Render a template filetemplatefile("x.tftpl", { ... })
Fallback on errortry(var.settings.port, 8080)
Explicit conversion (1.15)convert([], list(string))
Provider function (1.8)provider::aws::arn_parse(arn)
Interactive testingterraform console

Exercise

You need a map from worker name to the full worker object, to use with for_each. var.worker_list is a list of objects with a name attribute. Which expression builds it?

  1. [for w in var.worker_list : w.name => w]
  2. { for w in var.worker_list : w.name => w }
  3. toset(var.worker_list)
  4. var.worker_list[*].name
Show answer

Answer: B. Curly braces with key => value build a map. Square brackets build a list, toset needs strings for for_each, and the splat returns only the names.

Exercise

A conditional returns a list(string) on one side and [] on the other, and Terraform complains about inconsistent types. What is the clean fix in Terraform 1.15+?

  1. Wrap the whole conditional in tostring()
  2. Use null instead of []
  3. Replace [] with convert([], list(string))
  4. Move the conditional into a variable default
Show answer

Answer: C. [] is an empty tuple. convert([], list(string)) produces an empty list of strings, so both sides of the conditional have the same type.

Key takeaways

  • Test expressions in terraform console before putting them in code; -scope (1.16) works inside modules.
  • Use heredocs for short templates and templatefile() for anything longer.
  • for expressions transform and filter: [...] builds lists, {...} builds maps.
  • The map-from-list-of-objects pattern is how you feed lists of objects to for_each.
  • try, can, coalesce and lookup handle optional values cleanly.
  • Newer tools: provider-defined functions (1.8), templatestring (1.9), convert (1.15).

FAQ

Can I write my own functions in Terraform?

Not in HCL. You can use a module that takes inputs and returns outputs as a “function”, or a provider that defines functions (1.8+). For most needs, a local value with a for expression is enough.

Why does my for_each over a for expression fail with “Invalid for_each argument”?

The keys you build must be known at plan time. Build keys from variables or names you control, not from IDs of resources created in the same run.

What is the difference between try and can?

try(a, b) returns the first expression that evaluates without an error. can(a) returns true or false depending on whether a evaluates. Use can inside validation rules, try for fallback values.

Are there functions to call external programs or HTTP APIs?

No, built-in functions are pure and work offline. To fetch remote data, use a data source such as http from the hashicorp/http provider, or the external data source as a last resort.

↑ ↓ navigate Enter open Esc close