What is the Prometheus alert rule for "Stackdriver exporter scrape error"?

Stackdriver exporter failed to scrape metrics from Google Cloud Monitoring API for project {{ $labels.project_id }}. PromQL expression: stackdriver_monitoring_last_scrape_error > 0. Severity: warning. Duration: 5m.

What is the Prometheus alert rule for "Stackdriver exporter slow scrape"?

Stackdriver exporter scrape for project {{ $labels.project_id }} is taking more than 5 minutes ({{ $value }}s). PromQL expression: stackdriver_monitoring_last_scrape_duration_seconds > 300. Severity: warning. Duration: 5m.

What is the Prometheus alert rule for "Stackdriver exporter scrape errors increasing"?

Stackdriver exporter has had {{ $value }} scrape errors in the last 15 minutes for project {{ $labels.project_id }}. PromQL expression: increase(stackdriver_monitoring_scrape_errors_total[15m]) > 5. Severity: warning.

What is the Prometheus alert rule for "Stackdriver exporter high API calls"?

Stackdriver exporter is making {{ $value }} API calls per minute for project {{ $labels.project_id }}. This may hit Google Cloud Monitoring API quotas. PromQL expression: rate(stackdriver_monitoring_api_calls_total[5m]) * 60 > 100. Severity: warning.

What is the Prometheus alert rule for "Stackdriver exporter scrape stale"?

Stackdriver exporter has not successfully scraped metrics for project {{ $labels.project_id }} in the last 10 minutes. PromQL expression: time() - stackdriver_monitoring_last_scrape_timestamp > 600. Severity: warning.

⚠️

Alert thresholds depend on the nature of your applications. Some queries may have arbitrary tolerance thresholds. Building an efficient monitoring platform takes time. 😉

Google Cloud Stackdriver Prometheus Alert Rules

5 Prometheus alerting rules for Google Cloud Stackdriver. Exported via prometheus-community/stackdriver_exporter. These rules cover critical and warning conditions — copy and paste the YAML into your Prometheus configuration.

groups:
- name: StackdriverExporter
  rules:
    - alert: StackdriverExporterScrapeError
      expr: stackdriver_monitoring_last_scrape_error > 0
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: Stackdriver exporter scrape error (instance {{ $labels.instance }})
        description: "Stackdriver exporter failed to scrape metrics from Google Cloud Monitoring API for project {{ $labels.project_id }}.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"
    
    - alert: StackdriverExporterSlowScrape
      expr: stackdriver_monitoring_last_scrape_duration_seconds > 300
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: Stackdriver exporter slow scrape (instance {{ $labels.instance }})
        description: "Stackdriver exporter scrape for project {{ $labels.project_id }} is taking more than 5 minutes ({{ $value }}s).\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"
    
    - alert: StackdriverExporterScrapeErrorsIncreasing
      expr: increase(stackdriver_monitoring_scrape_errors_total[15m]) > 5
      for: 0m
      labels:
        severity: warning
      annotations:
        summary: Stackdriver exporter scrape errors increasing (instance {{ $labels.instance }})
        description: "Stackdriver exporter has had {{ $value }} scrape errors in the last 15 minutes for project {{ $labels.project_id }}.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"
    
    - alert: StackdriverExporterHighAPICalls
      expr: rate(stackdriver_monitoring_api_calls_total[5m]) * 60 > 100
      for: 0m
      labels:
        severity: warning
      annotations:
        summary: Stackdriver exporter high API calls (instance {{ $labels.instance }})
        description: "Stackdriver exporter is making {{ $value }} API calls per minute for project {{ $labels.project_id }}. This may hit Google Cloud Monitoring API quotas.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"
    
    - alert: StackdriverExporterScrapeStale
      expr: time() - stackdriver_monitoring_last_scrape_timestamp > 600
      for: 0m
      labels:
        severity: warning
      annotations:
        summary: Stackdriver exporter scrape stale (instance {{ $labels.instance }})
        description: "Stackdriver exporter has not successfully scraped metrics for project {{ $labels.project_id }} in the last 10 minutes.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

11.2. prometheus-community/stackdriver_exporter (5 rules)

Self-monitoring metrics use the stackdriver_monitoring_* prefix.
All self-monitoring metrics include a project_id label.

wget https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/refs/heads/master/dist/rules/google-cloud-stackdriver/stackdriver-exporter.yml

warning

11.2.1. Stackdriver exporter scrape error

Stackdriver exporter failed to scrape metrics from Google Cloud Monitoring API for project {{ $labels.project_id }}.

- alert: StackdriverExporterScrapeError
  expr: stackdriver_monitoring_last_scrape_error > 0
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: Stackdriver exporter scrape error (instance {{ $labels.instance }})
    description: "Stackdriver exporter failed to scrape metrics from Google Cloud Monitoring API for project {{ $labels.project_id }}.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

warning

11.2.2. Stackdriver exporter slow scrape

Stackdriver exporter scrape for project {{ $labels.project_id }} is taking more than 5 minutes ({{ $value }}s).

- alert: StackdriverExporterSlowScrape
  expr: stackdriver_monitoring_last_scrape_duration_seconds > 300
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: Stackdriver exporter slow scrape (instance {{ $labels.instance }})
    description: "Stackdriver exporter scrape for project {{ $labels.project_id }} is taking more than 5 minutes ({{ $value }}s).\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

warning

11.2.3. Stackdriver exporter scrape errors increasing

Stackdriver exporter has had {{ $value }} scrape errors in the last 15 minutes for project {{ $labels.project_id }}.

- alert: StackdriverExporterScrapeErrorsIncreasing
  expr: increase(stackdriver_monitoring_scrape_errors_total[15m]) > 5
  for: 0m
  labels:
    severity: warning
  annotations:
    summary: Stackdriver exporter scrape errors increasing (instance {{ $labels.instance }})
    description: "Stackdriver exporter has had {{ $value }} scrape errors in the last 15 minutes for project {{ $labels.project_id }}.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

warning

11.2.4. Stackdriver exporter high API calls

Stackdriver exporter is making {{ $value }} API calls per minute for project {{ $labels.project_id }}. This may hit Google Cloud Monitoring API quotas.

- alert: StackdriverExporterHighAPICalls
  expr: rate(stackdriver_monitoring_api_calls_total[5m]) * 60 > 100
  for: 0m
  labels:
    severity: warning
  annotations:
    summary: Stackdriver exporter high API calls (instance {{ $labels.instance }})
    description: "Stackdriver exporter is making {{ $value }} API calls per minute for project {{ $labels.project_id }}. This may hit Google Cloud Monitoring API quotas.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

warning

11.2.5. Stackdriver exporter scrape stale

Stackdriver exporter has not successfully scraped metrics for project {{ $labels.project_id }} in the last 10 minutes.

- alert: StackdriverExporterScrapeStale
  expr: time() - stackdriver_monitoring_last_scrape_timestamp > 600
  for: 0m
  labels:
    severity: warning
  annotations:
    summary: Stackdriver exporter scrape stale (instance {{ $labels.instance }})
    description: "Stackdriver exporter has not successfully scraped metrics for project {{ $labels.project_id }} in the last 10 minutes.\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

More in Cloud providers

AWS CloudWatch DigitalOcean Azure