Terraform Series - Iterating Over Local JSON with for_each

This article was last updated on: May 17, 2026 am

Series Articles

Overview

In the previous article Grafana Series - Grafana Terraform Provider Basics, we covered how to create a Datasource using the Grafana Terraform Provider.

Now there’s a real-world requirement:

A large number of datasources of the same type need to be added in bulk, and the basic information for these datasources already exists in JSON format.

We need to parse, simplify, and restructure the JSON, then use it as a Terraform datasource.

The JSON format might look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[
    {
        "env_name": "dev",
        "prom_url": "http://dev-prom.example.com",
        "es_url": "http://dev-es.example.com:9200",
        "jaeger_url": "http://dev-jaeger.example.com"
    },
    {
        "env_name": "test",
        "prom_url": "http://test-prom.example.com",
        "es_url": "http://test-es.example.com:9200",
        "jaeger_url": "http://test-jaeger.example.com"
    }
]

│ 📝Notes:
│
│ By extension, the solution below applies to any arbitrary JSON format.

How do we implement this? 🤔

Solution

This is achieved using Terraform’s locals, jsondecode, for loops, and for_each.

Here’s the approach:

  • Construct a local variable
  • The local variable reads from a .json file and decodes the JSON content into an object using jsondecode + file
  • Use a for loop to reshape the object according to the current requirements, using env_name from the example as the key and the rest as values
  • When creating resources in bulk, use for_each to iterate and create them.

Core Concepts

locals

locals assigns a name to an expression, so you can use that name multiple times within a module without repeating the expression.

If you’re familiar with traditional programming languages, it may be helpful to think of Terraform modules as function definitions:

Once a local value is declared, you can reference it in expressions as local..

Local values help avoid repeating the same value or expression multiple times in a configuration. Use local values in moderation — only when a single value or result is used in many places. The key advantage of local values is the ability to easily change a value in one central place.

file Function

file reads the contents of a file at the given path and returns it as a string.

1
2
> file("${path.module}/hello.txt")
Hello World

jsondecode Function

jsondecode interprets a given string as JSON and returns the decoded representation.

This function maps JSON values to Terraform language types as follows:

JSON type Terraform type
String string
Number number
Boolean bool
Object object(…) with attribute types determined by this table
Array tuple(…) with element types determined by this table
Null The Terraform language null value

Terraform’s automatic type conversion rules mean you typically don’t need to worry about the exact type a given value produces — just use the result intuitively.

1
2
3
4
5
6
> jsondecode("{\"hello\": \"world\"}")
{
  "hello" = "world"
}
> jsondecode("true")
true

│ jsonencode performs the reverse operation, encoding a value as a JSON string.

for Expressions

A for expression creates a complex type value by transforming another complex type value. Each element in the input can correspond to one or zero values in the result, and an arbitrary expression can be used to transform each input element into an output element.

For example, if var.list is a list of strings, the following expression produces a tuple of uppercase strings:

1
[for s in var.list : upper(s)]

This for expression iterates over each element in var.list, then evaluates the expression upper(s) with s set to each respective element. It then builds a new tuple value from all the results in the same order.

The input to a for expression (given after the in keyword) can be a list, a set, a tuple, a map, or an object.

The example above shows a for expression with only one temporary symbol s, but a for expression can optionally declare a pair of temporary symbols to also use each item’s key or index:

1
[for k, v in var.map : length(k) + length(v)]

For map or object types, as shown above, the k symbol refers to the key or attribute name of the current element. You can also use the two-symbol form with lists and tuples, in which case the additional symbol is the index of each element starting from zero — conventionally named i or idx, unless a more specific name is helpful:

1
[for i, v in var.list : "${i} is ${v}"]

The index or key symbol is always optional. If you specify only one symbol after the for keyword, it will always represent the value of each element in the input collection.

The type of brackets around the for expression determines the type of result it produces.

The examples above use [ and ], which produce a tuple. If you use { and } instead, the result is an object, and you must provide two result expressions separated by the => symbol:

1
{for s in var.list : s => upper(s)}

This expression produces an object whose attributes are the original elements from var.list, with corresponding values being the uppercase versions. For example, the resulting value might be:

1
2
3
4
5
{
  foo = "FOO"
  bar = "BAR"
  baz = "BAZ"
}

A single for expression can only produce either an object value or a tuple value, but Terraform’s automatic type conversion rules mean you can typically use the result wherever a list, map, or set is expected.

A for expression can also include an optional if clause to filter elements from the source collection, producing a value with fewer elements than the source:

1
[for s in var.list : upper(s) if s != ""]

A common reason for filtering a collection in a for expression is to split a source collection into two separate collections based on some criteria. For example, if the input var.users is a map of objects where each object has an is_admin attribute, you might want to produce separate maps of admin and non-admin objects:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
variable "users" {
  type = map(object({
    is_admin = bool
  }))
}

locals {
  admin_users = {
    for name, user in var.users : name => user
    if user.is_admin
  }
  regular_users = {
    for name, user in var.users : name => user
    if !user.is_admin
  }
}

Because for expressions can convert from unordered types (maps, objects, sets) to ordered types (lists, tuples), Terraform must choose an implied ordering for elements of unordered collections.

For maps and objects, Terraform sorts elements by key or attribute name using lexical ordering.

For sets of strings, Terraform sorts by value using lexical ordering.

The for expression mechanism is designed for constructing collection values from other collection values within expressions, which you can then assign to individual resource arguments that expect complex values.

for_each Meta-Argument

By default, a resource block configures one real infrastructure object (similarly, a module block includes a child module’s contents into the configuration once). However, sometimes you want to manage several similar objects (like a fixed pool of compute instances) without writing a separate block for each one. Terraform has two ways to do this: count and for_each.

If a resource or module block includes a for_each argument whose value is a map or a set of strings, Terraform creates one instance for each member of that map or set.

Version note: for_each was added in Terraform 0.12.6. Module support for for_each was added in Terraform 0.13; earlier versions could only use it with resources.

Note: A given resource or module block cannot use both count and for_each.

for_each is a meta-argument defined by the Terraform language. It can be used with modules and with every resource type.

The for_each meta-argument accepts a map or a set of strings, and creates an instance for each item in that map or set. Each instance has a distinct infrastructure object associated with it, and each is separately created, updated, or destroyed when the configuration is applied.

Map:

1
2
3
4
5
6
7
8
resource "azurerm_resource_group" "rg" {
  for_each = {
    a_group = "eastus"
    another_group = "westus2"
  }
  name     = each.key
  location = each.value
}

Set of strings:

1
2
3
4
resource "aws_iam_user" "the-accounts" {
  for_each = toset( ["Todd", "James", "Alice", "Dottie"] )
  name     = each.key
}

In blocks where for_each is set, an additional each object is available in expressions, so you can modify the configuration of each instance. This object has two attributes:

  • each.key — The map key (or set member) corresponding to this instance.
  • each.value — The map value corresponding to this instance. (If a set was provided, this is the same as each.key.)

When for_each is set, Terraform distinguishes between the block itself and the multiple resource or module instances associated with it. Instances are identified by a map key (or set member) from the value provided to for_each.

  • . or module. (e.g., azurerm_resource_group.rg) refers to the block.
  • .[] or module.[] (e.g., azurerm_resource_group.rg[“a_group”], azurerm_resource_group.rg[“another_group”], etc.) refers to individual instances.

This differs from resources and modules without count or for_each, which can be referenced without an index or key.

Strings & Templates

Strings are the most complex kind of literal expression in Terraform, and also the most commonly used.

Terraform supports both a quoted string syntax and a heredoc syntax for strings. Both syntaxes support template sequences for interpolation and text manipulation.

Quoted strings are a series of characters delimited by double quote characters (").

There are two special escape sequences that do not use backslashes:

Sequence Replacement
$${ Literal ${, without starting an interpolation sequence.
%%{ Literal %{, without starting a template directive sequence.

The ${ … } sequence is an interpolation, which evaluates the expression given between the markers, converts the result to a string if necessary, and then inserts it into the final string:

1
"Hello, ${var.name}!"

In the example above, the named object var.name is accessed and its value is inserted into the string, producing a result like “Hello, Juan!”.

The %{ ... } sequence is a directive, which allows conditional results and iteration over collections, similar to conditional and for expressions.

The following directives are supported:

  • The %{if <BOOL>}/%{else}/%{endif} directive chooses between two templates based on the value of a bool expression:

    1
    "Hello, %{ if var.name != "" }${var.name}%{ else }unnamed%{ endif }!"

    The else portion can be omitted, in which case the result is an empty string if the condition expression returns false.

  • The %{for <NAME> in <COLLECTION>}/%{endfor} directive iterates over the elements of a given collection or structural value, evaluating the given template once for each element and concatenating the results:

    1
    2
    3
    4
    5
    <<EOT
    %{ for ip in aws_instance.example.*.private_ip }
    server ${ip}
    %{ endfor }
    EOT

Hands-On

Requirement:

A large number of datasources of the same type need to be added in bulk, and the basic information for these datasources already exists in JSON format.

We need to parse, simplify, and restructure the JSON, then use it as a Terraform datasource.

The JSON format might look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[
    {
        "env_name": "dev",
        "prom_url": "http://dev-prom.example.com",
        "es_url": "http://dev-es.example.com:9200",
        "jaeger_url": "http://dev-jaeger.example.com"
    },
    {
        "env_name": "test",
        "prom_url": "http://test-prom.example.com",
        "es_url": "http://test-es.example.com:9200",
        "jaeger_url": "http://test-jaeger.example.com"
    }
]

Solution:

  • Construct a local variable
  • The local variable reads from a .json file and decodes the JSON content into an object using jsondecode + file
  • Use a for loop to reshape the object according to the current requirements, using env from the example as the key and the rest as values
  • When creating resources in bulk, use for_each to iterate and create them.

Putting it all together, the final result looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
locals {
  # Convert the JSON file into an object
  user_data = jsondecode(file("${path.module}/env-details.json"))
  # Construct a map
  # key is env_name
  # value is another map, where the key is the Grafana datasource type and the value is the URL
  envs = { for env in local.user_data : env.env_name =>
    {
      prometheus = env.prom_url
      # Use ${} to construct a new URL
      jaeger     = "${env.jaeger_url}/trace/"
      es         = env.es_url
    }
  }
}

resource "grafana_data_source" "prometheus" {
  # Iterate using for_each
  for_each = local.envs

  type = "prometheus"
  name = "${each.key}_prom"
  uid  = "${each.key}_prom"
  url  = each.value.prometheus

  json_data_encoded = jsonencode({
    httpMethod = "POST"
  })
}

resource "grafana_data_source" "jaeger" {
  for_each = local.envs

  type = "jaeger"
  name = "${each.key}_jaeger"
  uid  = "${each.key}_jaeger"
  url  = each.value.jaeger
}

resource "grafana_data_source" "elasticsearch" {
  for_each = local.envs

  type          = "elasticsearch"
  name          = "${each.key}_es"
  uid           = "${each.key}_es"
  url           = each.value.es
  database_name = "[example.*-]YYYY.MM.DD"

  json_data_encoded = jsonencode({
    esVersion = "6.0.0"

    interval = "Daily"
    includeFrozen              = false
    maxConcurrentShardRequests = 256
    timeField                  = "@timestamp"

    logLevelField   = "level"
    logMessageField = "message"
  })
}

Done! 🎉🎉🎉

📚️ References