How to Monitor URLs with Blackbox Exporter

This article was last updated on: May 17, 2026 am

Introduction

Monitoring domains and URLs is an important aspect of observability, primarily used for diagnosing availability issues. This article provides a detailed guide on how to implement URL monitoring in Kubernetes using Blackbox Exporter and Prometheus.

Overview of Blackbox Exporter

Blackbox Exporter is an optional component of Prometheus. Like other exporters, its main purpose is to convert monitoring data into a metrics format that Prometheus can understand, namely the Prometheus exposition format.

Endpoint Monitoring

Endpoint monitoring refers to monitoring various parameters of both internal and external endpoints (HTTP/S, DNS, TCP, ICMP, and gRPC), including HTTP response time, DNS query latency, SSL certificate expiration information, TLS version, and more.

In Kubernetes, it’s not just external endpoints that need monitoring — internal endpoints also need to be monitored for response time and other parameters. These metrics are a critical part of infrastructure to ensure service continuity, availability, and compliance with certain security certifications.

WhiteBox vs. Blackbox Monitoring

WhiteBox monitoring refers to monitoring the internals of a system, including application logging, handlers, tracing, and metrics. In contrast, Blackbox monitoring primarily initiates probes from the outside, probing user-impacting behaviors such as server downtime, non-functioning pages, or degraded website performance.

Blackbox Exporter

Blackbox Exporter is used to probe endpoints such as HTTPS, HTTP, TCP, DNS, ICMP, and gRPC. After you define the endpoints, Blackbox Exporter generates metrics that can be visualized using tools like Grafana. One of the most important features of Blackbox Exporter is measuring endpoint availability.

The following diagram shows the workflow of Blackbox Exporter monitoring an endpoint:

Blackbox Exporter workflow diagram

Installing and Configuring Blackbox Exporter

Installing Blackbox Exporter with Helm

Installing Blackbox Exporter is straightforward and can be done via a Helm Chart:

1
2
3
4
5
6
# Add repo
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install chart
helm install [RELEASE_NAME] prometheus-community/prometheus-blackbox-exporter

🎉

Basic Blackbox Configuration

Below is a default module defined in the Blackbox Exporter configuration:

blackbox.yaml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
modules:
  http_2xx:
    prober: http
    timeout: 15s  
    http:
      fail_if_not_ssl: true
      ip_protocol_fallback: false
      method: GET
      follow_redirects: true
      preferred_ip_protocol: ip4
      valid_http_versions:
        - HTTP/1.1
        - HTTP/2.0
      valid_status_codes:
        - 200
        - 204

You can configure your own blackbox.yml accordingly so that the probe returns success/failure based on your configuration. Using the configuration above as an example, here is a detailed explanation of the module and http probe settings:

  • prober: The protocol for probing (can be: http, tcp, dns, icmp, grpc).
  • timeout: Probe timeout duration.
  • http: HTTP probe configuration.

Next are the HTTP probe configuration options:

  • valid_status_codes: , … | default = 2xx: Acceptable status codes for this probe. Defaults to 2xx. Using the default is recommended.
  • valid_http_versions: HTTP versions accepted by this probe. Options: HTTP/1.1 HTTP/2.0
  • method: | default = “GET”: HTTP method used by the probe.
  • headers: Headers used by the probe. For example, you can add headers like user-agent to avoid being blocked by a WAF.
  • body_size_limit: | default = 0: Maximum uncompressed body length (in bytes) to be processed. A value of 0 means no limit.
  • compression: Compression algorithm for decompressing the response (gzip, br, deflate, ident).
  • follow_redirects: | default = true: Whether to follow redirects.
  • fail_if_ssl: Probe fails if SSL is present.
  • fail_if_not_ssl: Probe fails if SSL is not present.
  • fail_if_body_matches_regexp: Fails if the returned body matches this regex.
  • fail_if_body_not_matches_regexp: Fails if the returned body does not match this regex.
  • fail_if_header_matches: Fails if the returned header matches this regex. For headers with multiple values, it fails if at least one matches.
  • fail_if_header_not_matches: Fails if the returned header does not match this regex.
  • tls_config: TLS protocol configuration for the HTTP probe, commonly used for private certificates.
  • basic_auth: HTTP basic auth credentials for the target.
  • bearer_token: : Bearer token for the target.
  • proxy_url: Configuration for the proxy server used to connect to the target.
  • skip_resolve_phase_with_proxy: Skips DNS resolution and URL changes when an HTTP proxy (proxy_url) is set.
  • oauth2: OAuth 2.0 configuration for connecting to the target.
  • enable_http2: Whether to enable HTTP/2.
  • preferred_ip_protocol: IP protocol for the HTTP probe (ip4, ip6).
  • ip_protocol_fallback
  • body: HTTP request body used in the probe.

You can check this example.yml for detailed examples and more information. Additionally, some configuration changes need to be made in Prometheus for Blackbox Exporter to send metrics related to the application’s configuration.

Configuration in Prometheus

You need to configure scrape settings in Prometheus, as well as Blackbox-related Alert Rules.

Prometheus Scrape Configuration for Blackbox

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
scrape_configs:
  - job_name: blackbox-exporter
    params:
      module:
        - http_2xx
    scrape_interval: 1m
    scrape_timeout: 10s
    metrics_path: /probe
    scheme: http
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: prometheus-blackbox-exporter.monitoring:9115
        action: replace
    static_configs:
      - targets:
          - https://ewhisper.cn
          - https://www.ewhisper.cn
          - https://rancher.ewhisper.cn
        labels:
          domain: ewhisper
          environment: test
          cluster: home-k3s

Directly modifying the Prometheus configuration like this is error-prone. If you already have Prometheus Operator installed, you can configure it directly through the Probe CRD, which is much more convenient:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: monitoring.coreos.com/v1
kind: Probe
metadata:
  name: ewhisper
  namespace: monitoring
spec:
  jobName: http-get
  interval: 60s
  module: http_2xx
  prober:
    url: prometheus-blackbox-exporter.monitoring:9115
    scheme: http
    path: /probe
  targets:
    staticConfig:
      static:
      - targets:
          - https://ewhisper.cn
          - https://www.ewhisper.cn
          - https://rancher.ewhisper.cn
        labels:
          domain: ewhisper
          environment: test
          cluster: home-k3s

Blackbox Exporter Probing Scenarios

When it comes to URLs specifically, Blackbox Exporter covers the following probing scenarios:

  1. Probing external URLs
  2. Probing internal Kubernetes Services
  3. Probing internal Kubernetes Ingresses
  4. Probing internal Kubernetes Pods

Scenario 1: Probing External URLs

The configuration was already covered above, so it won’t be repeated here.

Scenario 2: Probing Internal Kubernetes Services

In a Kubernetes system, resources and endpoints appear and disappear over time. A very useful type of probing is dynamic probing of resources, including Pods, Services, and Ingresses.

Using Kubernetes service discovery configuration in Prometheus, we can achieve dynamic endpoint probing. Kubernetes service discovery configuration allows fetching scrape targets from the Kubernetes API and staying in sync with the cluster state at all times. You can find the list of available roles that can be configured as discovery targets in the kubernetes_sd_config section of the documentation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
kubernetes_sd_configs:
  - role: service
    metrics_path: /probe
    params:
      module:
      - http_2xx
    relabel_configs:
    - action: keep
      regex: true
      source_labels:
      - __meta_kubernetes_service_annotation_prometheus_io_probe
    - source_labels:
      - __address__
      target_label: __param_target
    - replacement: prometheus-blackbox-exporter.monitoring:9115
      target_label: __address__
    - source_labels:
      - __param_target
      target_label: instance
    - action: labelmap
      regex: __meta_kubernetes_service_label_(.+)
    - source_labels:
      - __meta_kubernetes_namespace
      target_label: kubernetes_namespace
    - source_labels:
      - __meta_kubernetes_service_name
      target_label: kubernetes_name

Here we can use [__meta_kubernetes_service_annotation_prometheus_io_probe] to only check services that have the prometheus.io/probe = true annotation. Example:

1
2
3
4
➜ kubectl describe svc nginx
...
Annotations:              prometheus.io/probe: true
...

Scenario 3: Probing Internal Kubernetes Ingresses

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
- job_name: "blackbox-kubernetes-ingresses"
  metrics_path: /probe
  params:
    module: [http_2xx]
  kubernetes_sd_configs:
  - role: ingress
  relabel_configs:
  # Example relabeling to only probe some ingresses that have "prometheus.io/probe = true" annotation.
  #  - source_labels: [__meta_kubernetes_ingess_annotation_prometheus_io_probe]
  #    action: keep
  #    regex: true
    - source_labels:
        [
          __meta_kubernetes_ingress_scheme,
          __address__,
          __meta_kubernetes_ingress_path,
        ]
      regex: (.+);(.+);(.+)
      replacement: ${1}://${2}${3}
      target_label: __param_target
    - target_label: __address__
      replacement: prometheus-blackbox-exporter.monitoring:9115
    - source_labels: [__param_target]
      target_label: instance
    - action: labelmap
      regex: __meta_kubernetes_ingress_label_(.+)
    - source_labels: [__meta_kubernetes_namespace]
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_ingress_name]
      target_label: ingress_name

Scenario 4: Probing Internal Kubernetes Pods

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
- job_name: "blackbox-kubernetes-pods"
  metrics_path: /probe    
  params:
    module: [http_2xx]
  kubernetes_sd_configs:
  - role: pod        
  relabel_configs:
  # Example relabeling to only probe pods that have
  # "prometheus.io/probe = true" annotation.
  #  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_probe]
  #    action: keep
  #    regex: true
    - source_labels: [__address__]
      target_label: __param_target
    - target_label: __address__
      replacement:  prometheus-blackbox-exporter.monitoring:9115
    - source_labels: [__param_target]
      replacement: ${1}/health
      target_label: instance          
    - action: labelmap
      regex: __meta_kubernetes_pod_label_(.+)
    - source_labels: [__meta_kubernetes_namespace]
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_pod_name]
      target_label: kubernetes_pod_name

Verifying Generated Metrics in Prometheus

Blackbox Alert

Once the changes are applied and the Blackbox Exporter resources are deployed, we can verify the status of targets in Prometheus. We can check whether Blackbox Exporter has started with the registered targets by navigating to the Status tab and selecting Targets in the Prometheus UI.

Here you can see that we used https://rancher.ewhisper.cn as an external target for reference, and its status is 404. We can also check whether metrics are being collected by looking for metrics prefixed with probe_.

Prometheus probe metrics

Here you can see a list of some generated probe_ metrics.

Metric Name Description
probe_duration_seconds Returns the time (in seconds) for the probe to complete.
probe_http_status_code Response HTTP status code.
probe_http_version Returns the HTTP version of the probe response.
probe_success Indicates whether the probe was successful.
probe_dns_lookup_time_seconds Returns the DNS lookup time for the probe, in seconds.
probe_ip_protocol Specifies whether the probe IP protocol is IP4 or IP6.
probe_ssl_earliest_cert_expiry metric Returns the earliest SSL certificate expiry time in unixtime.
probe_tls_version_info Contains the TLS version in use.
probe_failed_due_to_regex Indicates whether the probe failed due to a regex match.
probe_http_content_length Length of the HTTP content response.

Monitoring Configured URLs with Grafana

You can directly reuse some dashboards on Grafana to view URL metrics:

dashboard

dashboard

Click here 👉 Blackbox Grafana to search and download the corresponding Grafana dashboards.

Summary of Blackbox Advantages

  1. An open-source, free Blackbox endpoint monitoring tool.
  2. Supports DNS, TCP, ICMP, and gRPC in addition to HTTP/S.
  3. Rich HTTP blackbox monitoring configuration options, including headers, authentication, proxies, regex matching, and more.
  4. Leverages Prometheus + Kubernetes kubernetes_sd_config for dynamic metric generation, enabling dynamic endpoint monitoring.
  5. Can monitor certificate expiration times.

Industry Use Cases for Blackbox Exporter

Why do you need Blackbox Exporter?

Taking the insurance industry I’m familiar with as an example, medium to large insurance companies typically adopt an organizational structure like:

  • Headquarters
  • Provincial branches
  • Central sub-branches
  • Sub-branches
  • Sales offices

Branch offices are usually connected to headquarters via dedicated lines and use various insurance business systems provided by headquarters.

While there are various domestic and international tools and services available for monitoring domains and URLs, such as Tingyun, Dynatrace, etc.:

  • On one hand, these services charge per probe, and if the probing frequency or number of URLs is high, the cost can be significant.
  • On the other hand, these commercial services may not be able to cover the nearly intranet-like network architecture of the insurance industry.

In this scenario, Blackbox Exporter serves as an open-source alternative to existing solutions, maintained by the Prometheus community.

Moreover, Prometheus + Blackbox Exporter + Kubernetes dynamic discovery can significantly reduce the manual effort of configuring large numbers of URL probes.

Additionally, for the scenario described above, you can also use Prometheus + Blackbox Exporter + a lightweight K8s solution like K3s to deploy probe nodes to each branch office, achieving the exact same access path as branch office employees. The network availability between branches and headquarters systems becomes clear at a glance, enabling timely detection of network issues.

Conclusion

In this article, we discussed:

  • What Blackbox Exporter is
  • How to install and configure it
  • Several typical configuration scenarios, especially leveraging Prometheus + Blackbox Exporter + Kubernetes dynamic discovery
  • Advantages of Blackbox Exporter
  • Industry use cases for Blackbox Exporter

Hope this is helpful to you.

🎉🎉🎉

📚️ Reference