What is the Prometheus alert rule for "Linkerd high error rate"?

⚠️

Alert thresholds depend on the nature of your applications. Some queries may have arbitrary tolerance thresholds. Building an efficient monitoring platform takes time. 😉

Linkerd Prometheus Alert Rules

1 Prometheus alerting rule for Linkerd. Exported via Embedded exporter. These rules cover critical and warning conditions — copy and paste the YAML into your Prometheus configuration.

groups:
- name: EmbeddedExporter
  rules:
      # Linkerd does not expose request_errors_total. Errors are tracked via response_total{classification="failure"}.
    - alert: LinkerdHighErrorRate
      expr: sum(rate(response_total{classification="failure"}[1m])) by (deployment, statefulset, daemonset) / sum(rate(response_total[1m])) by (deployment, statefulset, daemonset) * 100 > 10 and sum(rate(response_total[1m])) by (deployment, statefulset, daemonset) > 0
      for: 1m
      labels:
        severity: warning
      annotations:
        summary: Linkerd high error rate (instance {{ $labels.instance }})
        description: "Linkerd error rate for {{ $labels.deployment }}{{ $labels.statefulset }}{{ $labels.daemonset }} is over 10%\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

4.7. Embedded exporter (1 rules)

wget https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/refs/heads/master/dist/rules/linkerd/embedded-exporter.yml

warning

4.7.1. Linkerd high error rate

Linkerd error rate for {{ $labels.deployment }}{{ $labels.statefulset }}{{ $labels.daemonset }} is over 10%

  # Linkerd does not expose request_errors_total. Errors are tracked via response_total{classification="failure"}.
- alert: LinkerdHighErrorRate
  expr: sum(rate(response_total{classification="failure"}[1m])) by (deployment, statefulset, daemonset) / sum(rate(response_total[1m])) by (deployment, statefulset, daemonset) * 100 > 10 and sum(rate(response_total[1m])) by (deployment, statefulset, daemonset) > 0
  for: 1m
  labels:
    severity: warning
  annotations:
    summary: Linkerd high error rate (instance {{ $labels.instance }})
    description: "Linkerd error rate for {{ $labels.deployment }}{{ $labels.statefulset }}{{ $labels.daemonset }} is over 10%\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}"

More in Proxies, load balancers and service meshes

Nginx Apache HaProxy Traefik Caddy Envoy Istio