Terraform Series - How to Conditionally Create Resources Based on a Field When Batch Creating

Series Articles

• Terraform Series Articles
• Grafana Series Articles

Overview

The previous article Grafana Series - Grafana Terraform Provider Basics introduced how to create Datasources using the Grafana Terraform Provider.

I recently ran into a real-world requirement:

When using Terraform to batch create log data sources, some data source types are ElasticSearch while others are Opensearch. So how do you conditionally create resources based on a specific field (e.g., es_type)?

Also, I recommend reading the previous article first: Terraform Series - Iterating Over Local JSON with for-each to quickly get up to speed on the context.

The data source for creating datasources is a JSON file. After the transformation described in the previous article, the format looks like this:

{
    "dev": 
    {
        "env_name": "dev",
        "prom_url": "http://dev-prom.example.com",
        "jaeger_url": "http://dev-jaeger.example.com",
        "es_url": "http://dev-es.example.com:9200",
        "es_type": "elasticsearch"
    },
    "test": 
    {
        "env_name": "test",
        "prom_url": "http://test-prom.example.com",
        "jaeger_url": "http://test-jaeger.example.com",
        "es_url": "http://test-es.example.com:9200",
        "es_type": "opensearch"
    }
}

How do we implement this? 🤔

Solution

Use a for loop + if to reconstruct the map.

Here’s the approach:

• When batch creating resources, use for_each for bulk creation.
• Within for_each, use a for loop + if to reconstruct the map, filtering based on local.env.es_type to determine whether to create the resource.

Here’s the implementation:

locals {
  # Convert the JSON file to an object
  user_data = jsondecode(file("${path.module}/env-details.json"))
  # Construct a map
  # key is env_name
  # value is another map, where key is grafana datasource type and value is url
  envs = { for env in local.user_data : env.env_name =>
    {
      prometheus = env.prom_url
      # Use ${} to construct a new url
      jaeger     = "${env.jaeger_url}/trace/"
      es         = env.es_url
      es_type    = env.es_type
    }
  }
}


resource "grafana_data_source" "elasticsearch" {
  for_each = {
    for env_name, env_info in local.envs : env_name => env_info
    if env_info.es_type == "elasticsearch"
  }

  type          = "elasticsearch"
  name          = "${each.key}_es"
  uid           = "${each.key}_es"
  url           = each.value.es
  database_name = "[example.*-]YYYY.MM.DD"

  json_data_encoded = jsonencode({
    esVersion = "6.0.0"

    interval = "Daily"
    includeFrozen              = false
    maxConcurrentShardRequests = 256
    timeField                  = "@timestamp"

    logLevelField   = "level"
    logMessageField = "message"
  })
}

resource "grafana_data_source" "opensearch" {
  for_each = {
    for env_name, env_info in local.envs : env_name => env_info
    if env_info.es_type == "opensearch"
  }

  type = "grafana-opensearch-datasource"
  name = "${each.key}_opensearch"
  uid  = "${each.key}_opensearch"
  url  = each.value.es
  basic_auth_enabled  = true
  basic_auth_username = "readonly"

  json_data_encoded = jsonencode({
    database = "[example.*-]YYYY.MM.DD"
    version  = "6.8.0"
    flavor   = "elasticsearch"

    interval = "Daily"
    pplEnabled                 = true
    maxConcurrentShardRequests = 256
    timeField                  = "@timestamp"

    logLevelField   = "level"
    logMessageField = "message"
  })

  secure_json_data_encoded = jsonencode({
    basicAuthPassword = "Changeme!"
  })
}

Don’t let the length of this code intimidate you — most of it isn’t directly relevant to the topic at hand. The key to the implementation lies in this code snippet:

for_each = {
  for env_name, env_info in local.envs : env_name => env_info
  if env_info.es_type == "elasticsearch"
}

It’s quite straightforward and self-explanatory. If es_type is elasticsearch, then the object gets included in the map.

After that, different DataSource types will have different parameters, as shown above:

• Opensearch has a different type from ES, and Opensearch has authentication enabled
• Opensearch uses the database field instead of database_name
• Opensearch additionally has the flavor and pplEnabled fields.

Alternative Solution

If your raw data or the constructed locals is a list rather than a map, you can use count + the condition ? true_val : false_val conditional expression to achieve the same result.

Here’s an example:

The decision is based on whether var.cloudflare is true or false.

resource "cloudflare_record" "record" {
  count = var.cloudflare ? 1 : 0
  zone_id = "${data.cloudflare_zones.domain.zones[0].id}"
  name    = "${var.subdomain}"
  value   = "${var.origin_server}"
  type    = "CNAME"
  ttl     = 1
  proxied = true
}

The key point is the conditional expression: count = var.cloudflare ? 1 : 0

Also very clean and clear.

Done! 🎉🎉🎉

📚️ References

• Terraform: Conditional creation of a resource based on a variable in .tfvars - Stack Overflow
• Conditionally create resources when a for_each loop is involved - Terraform - HashiCorp Discuss