I needed to measure DNS resolution performance of two domains from two different locations using three DNS resolvers at each location.

The example configuration demonstrates how to use Prometheus relabel configs to pass multiple targets to one blackbox_exporter. Two web search results (referenced below) hinted on using regexes to pass more than one parameter for each target. I did a little cleanup of the regex pattern and used the @ separator instead of :. The resulting configuration is easy to follow and can be adapted for other blackbox use-cases.

Each DNS query must be configured as separate modules in blackbox.yml. The blackbox.yml content is copied on both blackboxes.

modules:
  dns_udp_host1_cname:
    prober: dns
    timeout: 5s
    dns:
      preferred_ip_protocol: ip4
      query_name: "alias.example.com"
      query_type: "CNAME"
      valid_rcodes:
      - NOERROR
      validate_answer_rrs:
        fail_if_not_matches_regexp:
        - "alias.example.com.\t.*\tIN\tCNAME\thost.example.com."
  dns_udp_host1_a:
    prober: dns
    timeout: 5s
    dns:
      preferred_ip_protocol: ip4
      query_name: "host.example.com"
      query_type: "A"
      valid_rcodes:
      - NOERROR
      validate_answer_rrs:
        fail_if_not_matches_regexp:
        - "host.example.com.\t.*\tIN\tA\t.*"

This is what the scrape_config in prometheus.yml looks like:

  - job_name: dns_probes
    scrape_interval: 60s
    metrics_path: /probe
    static_configs:
      - targets:
        - blackbox1:9115@dns_udp_host1_cname@192.168.13.1:53
        - blackbox1:9115@dns_udp_host1_cname@ns1.example.com:53
        - blackbox1:9115@dns_udp_host1_cname@ns2.example.com:53
        - blackbox1:9115@dns_udp_host1_a@192.168.13.1:53
        - blackbox1:9115@dns_udp_host1_a@ns1.example.com:53
        - blackbox1:9115@dns_udp_host1_a@ns2.example.com:53
        - blackbox2:9115@dns_udp_host1_cname@8.8.8.8:53
        - blackbox2:9115@dns_udp_host1_cname@ns1.example.com:53
        - blackbox2:9115@dns_udp_host1_cname@ns2.example.com:53
        - blackbox2:9115@dns_udp_host1_a@8.8.8.8:53
        - blackbox2:9115@dns_udp_host1_a@ns1.example.com:53
        - blackbox2:9115@dns_udp_host1_a@ns2.example.com:53
    relabel_configs:
      - source_labels: [__address__]
        regex: '(.*)@.*@.*'
        replacement: $1
        target_label: 'instance'      # instance label for Prometheus datapoints
      - source_labels: [__address__]
        regex: '.*@(.*)@.*'
        replacement: $1
        target_label: __param_module  # module parameter to blackbox exporter
      - source_labels: [__address__]
        regex: '.*@(.*)@.*'
        replacement: $1
        target_label: module          # module label for Prometheus datapoints
      - source_labels: [__address__]
        regex: '.*@.*@(.*)'
        replacement: $1
        target_label: __param_target  # target parameter to blackbox exporter
      - source_labels: [__address__]
        regex: '.*@.*@(.*)'
        replacement: $1
        target_label: resolver        # resolver label for Prometheus datapoints
      - source_labels: [__address__]
        regex: '(.*)@.*@.*'
        replacement: $1
        target_label: __address__  # The blackbox exporter's real hostname:port.

This pattern of @ separators, regexes and replacement string is easy to read and expand to pass more labels or parameters.

To debug your DNS queries and validation, you can curl the following URL on your blackbox_exporter:

http://blackbox1:9115/probe?module=dns_udp_host1_a&target=ns1.example.com&debug=true

References

Upstream documentation:

Ideas from community discussion:

Comments

Comment Atom Feed

There are no comments yet.

Add a Comment

You can use the Markdown syntax to format your comment.