Blackbox Exporter

Worldwide probes

Blackbox Exporter gives you the ability to probe endpoints over HTTP, HTTPS, DNS, TCP and ICMP.

You should deploy blackbox exporters in multiple Points of Presence around the globe to monitor latency. Feel free to use the following endpoints for your own projects:

• https://probe-montreal.cleverapps.io
• https://probe-paris.cleverapps.io
• https://probe-jeddah.cleverapps.io
• https://probe-singapore.cleverapps.io
• https://probe-sydney.cleverapps.io
• https://probe-warsaw.cleverapps.io

☝️ Server logs have been disabled. More probes from the community would be appreciated — contribute here! These blackbox exporters use the following configuration.

Prometheus Configuration

Blackbox exporters and endpoints must be declared in Prometheus. Here is a simple configuration, inspired by Hayk Davtyan's medium post:

# sd/blackbox.yml

- targets:
  #
  # Montreal
  #
  # http
  - probe-montreal.cleverapps.io:_:http_2xx:_:Montreal:_:f229cy:_:https://api.screeb.app
  - probe-montreal.cleverapps.io:_:http_2xx:_:Montreal:_:f229cy:_:https://t.screeb.app/tag.js
  # icmp
  - probe-montreal.cleverapps.io:_:icmp_ipv4:_:Montreal:_:f229cy:_:api.screeb.app
  - probe-montreal.cleverapps.io:_:icmp_ipv4:_:Montreal:_:f229cy:_:t.screeb.app


  #
  # Paris
  #
  # http
  - probe-paris.cleverapps.io:_:http_2xx:_:Paris:_:u09tgy:_:https://api.screeb.app
  - probe-paris.cleverapps.io:_:http_2xx:_:Paris:_:u09tgy:_:https://t.screeb.app/tag.js
  # icmp
  - probe-paris.cleverapps.io:_:icmp_ipv4:_:Paris:_:u09tgy:_:api.screeb.app
  - probe-paris.cleverapps.io:_:icmp_ipv4:_:Paris:_:u09tgy:_:t.screeb.app


  #
  # Sydney
  #
  # http
  - probe-sydney.cleverapps.io:_:http_2xx:_:Sydney:_:r3gpkn:_:https://api.screeb.app
  - probe-sydney.cleverapps.io:_:http_2xx:_:Sydney:_:r3gpkn:_:https://t.screeb.app/tag.js
  # icmp
  - probe-sydney.cleverapps.io:_:icmp_ipv4:_:Sydney:_:r3gpkn:_:api.screeb.app
  - probe-sydney.cleverapps.io:_:icmp_ipv4:_:Sydney:_:r3gpkn:_:t.screeb.app

  # ...

# prometheus.yml

global:
  # ...

scrape_configs:

  - job_name: 'blackbox'
    metrics_path: /probe
    scrape_interval: 30s
    scheme: https
    file_sd_configs:
      - files:
        - /etc/prometheus/sd/blackbox.yml
    relabel_configs:
      # adds "module" label in the final labelset
      - source_labels: [__address__]
        regex: '.*:_:(.*):_:.*:_:.*:_:.*'
        target_label: module
      # adds "geohash" label in the final labelset
      - source_labels: [__address__]
        regex: '.*:_:.*:_:.*:_:(.*):_:.*'
        target_label: geohash
      # rewrites "instance" label with corresponding URL
      - source_labels: [__address__]
        regex: '.*:_:.*:_:.*:_:.*:_:(.*)'
        target_label: instance
      # rewrites "pop" label with corresponding location name
      - source_labels: [__address__]
        regex: '.*:_:.*:_:(.*):_:.*:_:.*'
        target_label: pop
      # passes "module" parameter to Blackbox exporter
      - source_labels: [module]
        target_label: __param_module
      # passes "target" parameter to Blackbox exporter
      - source_labels: [instance]
        target_label: __param_target
      # the Blackbox exporter's real hostname:port
      - source_labels: [__address__]
        regex: '(.*):_:.*:_:.*:_:.*:_:.*'
        target_label: __address__

  # ...

Geohash

To display nice maps in Grafana, you need to instruct blackbox exporters about the location. Grafana map panel speaks the "geohash" format:

1. Go to Google Maps
2. Extract the lat/long from the URL
3. Convert lat/long to geohash at geohash.co

Grafana

Some great dashboards have been created by the community: grafana.com/grafana/dashboards/?search=blackbox

Since Grafana v5.0.0, a map panel is available: Geomap panel documentation