Alert thresholds depend on the nature of your applications. Some queries may have arbitrary tolerance thresholds. Building an efficient monitoring platform takes time. 😉
6 Prometheus alerting rules for OpenSearch. Exported via opensearch-project/opensearch-prometheus-exporter. These rules cover critical and warning conditions — copy and paste the YAML into your Prometheus configuration.
groups:
- name: OpensearchProjectOpensearchPrometheusExporter
rules:
- alert: OpenSearchIsUnhealthy
expr: opensearch_cluster_status != 0
for: 0m
labels:
severity: critical
annotations:
summary: OpenSearch is unhealthy (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} is unhealthy\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: OpenSearchHighHeapUsage
expr: opensearch_jvm_mem_heap_used_percent > 90
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch high heap usage (instance {{ $labels.instance }})
description: "OpenSearch heap usage on cluster {{ $labels.cluster }} is too high\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: OpenSearchCircuitbreakerTripped
expr: opensearch_circuitbreaker_tripped_count > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch circuitbreaker tripped (instance {{ $labels.instance }})
description: "The circuitbreaker on OpenSearch cluster {{ $labels.cluster }} has tripped to prevent Java OutOfMemoryError\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: OpenSearchHasPendingTasks
expr: opensearch_cluster_pending_tasks_number > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch has pending tasks (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} has pending tasks\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: OpenSearchIndexingIsThrottled
expr: opensearch_indices_indexing_is_throttled_bool > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch indexing is throttled (instance {{ $labels.instance }})
description: "Indexing on OpenSearch cluster {{ $labels.cluster }} is throttled\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
- alert: OpenSearchHasInactiveShards
expr: opensearch_cluster_shards_active_percent < 100.0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch has inactive shards (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} has inactive shards\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" wget https://raw.githubusercontent.com/samber/awesome-prometheus-alerts/refs/heads/master/dist/rules/opensearch/opensearch-project-opensearch-prometheus-exporter.yml
OpenSearch cluster {{ $labels.cluster }} is unhealthy
- alert: OpenSearchIsUnhealthy
expr: opensearch_cluster_status != 0
for: 0m
labels:
severity: critical
annotations:
summary: OpenSearch is unhealthy (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} is unhealthy\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" OpenSearch heap usage on cluster {{ $labels.cluster }} is too high
- alert: OpenSearchHighHeapUsage
expr: opensearch_jvm_mem_heap_used_percent > 90
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch high heap usage (instance {{ $labels.instance }})
description: "OpenSearch heap usage on cluster {{ $labels.cluster }} is too high\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" The circuitbreaker on OpenSearch cluster {{ $labels.cluster }} has tripped to prevent Java OutOfMemoryError
- alert: OpenSearchCircuitbreakerTripped
expr: opensearch_circuitbreaker_tripped_count > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch circuitbreaker tripped (instance {{ $labels.instance }})
description: "The circuitbreaker on OpenSearch cluster {{ $labels.cluster }} has tripped to prevent Java OutOfMemoryError\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" OpenSearch cluster {{ $labels.cluster }} has pending tasks
- alert: OpenSearchHasPendingTasks
expr: opensearch_cluster_pending_tasks_number > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch has pending tasks (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} has pending tasks\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" Indexing on OpenSearch cluster {{ $labels.cluster }} is throttled
- alert: OpenSearchIndexingIsThrottled
expr: opensearch_indices_indexing_is_throttled_bool > 0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch indexing is throttled (instance {{ $labels.instance }})
description: "Indexing on OpenSearch cluster {{ $labels.cluster }} is throttled\n VALUE = {{ $value }}\n LABELS = {{ $labels }}" OpenSearch cluster {{ $labels.cluster }} has inactive shards
- alert: OpenSearchHasInactiveShards
expr: opensearch_cluster_shards_active_percent < 100.0
for: 5m
labels:
severity: warning
annotations:
summary: OpenSearch has inactive shards (instance {{ $labels.instance }})
description: "OpenSearch cluster {{ $labels.cluster }} has inactive shards\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"