Skip to main content

Overview

The Recurring Alerts page provides a comprehensive view of your most frequently triggered alerts. Use this data to identify candidates for tuning, create suppression rules, and reduce alert fatigue across your team.

Alert Ranking

See which alerts fire most frequently

Trend Analysis

Identify increasing or decreasing patterns

Export & Reports

Export data and send email reports

Quick Actions

Create suppression rules directly

Summary Statistics

Four metrics provide an overview of recurring alert patterns:
MetricDescription
Unique AlertsNumber of distinct alert types
Total OccurrencesSum of all alert instances
Increasing TrendAlerts getting worse over time
Avg Alert ShareAverage percentage of total each alert represents
High “Increasing Trend” count indicates growing problems that need attention.

Alert List Table

The main table displays detailed information about each recurring alert:
ColumnDescription
#Rank by occurrence count
Alert TitleAlert name and associated service
CountNumber of times alert fired
TrendIncreasing, stable, or decreasing
VolumePercentage of total alerts
MTTAAverage acknowledgment time
MTTRAverage resolution time
SeveritiesBreakdown by severity level
ActionsQuick action menu
TrendIconMeaning
Increasing🔴 ↑Alert firing more frequently
StableNo significant change
Decreasing🟢 ↓Alert firing less frequently
Alerts with Increasing trends require investigation—they often indicate growing underlying problems.

Search across multiple fields:
  • Alert title
  • Service name
  • Host names
  • Custom tags

Trend Filter

Filter to see only:
  • All Trends — Show everything
  • Increasing — Alerts getting worse
  • Stable — Consistent alerts
  • Decreasing — Improving alerts

Group by Service

Toggle “Group by Service” to aggregate alerts by service:
ModeUse Case
OffSee individual alert patterns
OnIdentify noisy services overall
Use “Group by Service” to find services that need comprehensive alert review.

Taking Action

From the Actions Menu

For each alert, you can:
1

View Incidents

Opens the incidents page filtered to this alert title
2

Create Suppress Rule

Pre-populates an alert rule with this alert’s details

Export Options

Click the download button to export all filtered data to CSV:Included Fields:
  • Rank, Alert Title, Service
  • Count, Percent of Total
  • MTTA, MTTR
  • Severity breakdown
  • Trend information
  • Last occurrence
  • Affected services and hosts

Identifying Tuning Candidates

High-Priority Candidates

Alerts that should be reviewed first:
Pattern: Alert fires frequently but is mostly low/medium severityAction Options:
  • Increase threshold to reduce triggers
  • Convert to informational alert
  • Suppress during non-business hours
Pattern: Alert firing more frequently over timeAction Options:
  • Investigate root cause of increase
  • Fix underlying issue
  • Temporarily suppress while fixing
Pattern: Alert takes long time to resolveAction Options:
  • Create or improve runbook
  • Automate remediation
  • Review if alert is actionable
Pattern: Alert fires and resolves quickly without actionAction Options:
  • Increase alert delay/threshold
  • Convert to warning level
  • Implement hysteresis

Creating Effective Suppress Rules

1

Identify Pattern

Determine what makes this alert non-actionable:
  • Specific time windows?
  • Certain environments?
  • Below specific threshold?
2

Define Conditions

Create precise conditions that match:
  • Alert title patterns
  • Source/service
  • Severity level
3

Set Appropriate Action

Choose the right response:
  • Suppress — Don’t create incident
  • Reduce Severity — Lower priority
  • Route Differently — Send to different team
4

Monitor Results

After implementing, verify:
  • Desired alerts are suppressed
  • Important alerts still come through
  • Overall noise reduced

Best Practices

Schedule weekly or bi-weekly reviews of recurring alerts:
  • Monday morning review of previous week
  • Include in team standup agenda
  • Track progress on noise reduction
Focus on the top 10 recurring alerts:
  • These represent most of the noise
  • Improvements have biggest impact
  • More manageable scope
For each reviewed alert, document:
  • Decision made (tune, suppress, keep as-is)
  • Reasoning
  • Expected outcome
  • Review date
Track metrics over time:
  • Total unique alerts
  • Total occurrences
  • Percentage of alerts suppressed
  • Team feedback on noise levels
Before suppressing, ask:
  • Has this alert ever caught a real issue?
  • Could we miss something important?
  • Is there a better alternative (tuning vs. suppressing)?
Suppressions can become stale:
  • Services change
  • Thresholds should be reconsidered
  • Set reminders to review suppression rules quarterly

Common Patterns and Solutions

Pattern: Frequent disk space warnings that self-resolveSolutions:
  • Increase threshold (e.g., 80% → 90%)
  • Implement auto-cleanup scripts
  • Add hysteresis (alert only after X minutes)
  • Separate critical partition alerts from non-critical
Pattern: Brief spikes in connection pool usageSolutions:
  • Increase pool size if appropriate
  • Add averaging/smoothing to alert
  • Alert on sustained high usage, not spikes
Pattern: Same job fails and succeeds on retrySolutions:
  • Improve job retry logic
  • Alert only after N failures
  • Separate transient vs. persistent failures
Pattern: Health checks failing/recovering rapidlySolutions:
  • Add dead time between alerts
  • Require multiple consecutive failures
  • Review health check timeout settings
Pattern: Alerts during deploymentsSolutions:
  • Implement deployment windows with suppression
  • Use canary/gradual deployments
  • Improve deployment health checks

Pagination and Large Datasets

For organizations with many alerts:

Pagination Controls

  • Rows per page: 10, 25, 50, or 100
  • Navigation: Previous/Next page buttons
  • Position indicator: “1-25 of 150”

Working with Large Lists


Troubleshooting

  • Verify incidents exist in the selected date range
  • Check that incidents have title information
  • Ensure incidents are assigned to your tenant
  • Trends compare current period to previous period
  • Short date ranges may show variable trends
  • Try longer date range for more accurate trends
  • Verify incidents have service metadata
  • Check integration is sending service information
  • Review alert payload configuration
  • Exports include current filter results
  • Clear filters to export all data
  • Maximum export is 100 alerts
  • Check spam/junk folders
  • Verify email address in profile
  • Contact admin if email delivery issues persist

URL Parameters

The page supports URL parameters for deep linking:
ParameterDescriptionExample
daysDate range in days?days=14
groupByServiceEnable service grouping?groupByService=true
searchPre-fill search?search=database
trendFilter by trend?trend=increasing
Bookmark filtered views for quick access to specific alert categories.