Alerts of various kinds related to CPU, Disk, Network, Yarn Applications can be created through Acceldata. A unique feature of these alerts is the granularity at which the alerts can be created.
As an example, one can create an alert for all Yarn MR jobs, which monitors the number of mappers, or the number of reducers and raises an alarm if there is an abnormality. Another example could be the time that a spark task takes, if the task takes more than a second, raise an alert.
Creating an alert
Alerts can be created in multiple categories, which encompass the following areas:
- Infrastructure metrics
- Hadoop Services
- Yarn Applications
- Ingestion Alerts with Kafka Support
- Hive and Spark Specific Applications
Alert Expressions allow the addition/selection of various conditions:
- Execution Interval - This is the delay between two consecutive evaluatins of an alert. The initial delay is also equal to
execution interval. This is specified in seconds.
- Threshold - An alert if evaluated to be breaching consecutively for a threshold time(number of times or time duration), then an incident is created for the alert. This is specified in either number or seconds.
- Incident - Incident is a result of an alert continuously getting raised.
|Raised||An Incident is created with the value as |
|Cleared||The existing incident will be marked as |
|Muted||When any operator takes action on the incident, it is marked as |
Editing an alert
This is an easy step, which can be completed using the same screen, by clicking on the alert definition. The Edit options allow the administrator to change the conditions, filter conditions.
Alert notifications are delivered to users/groups via Email and Slack Channels. For the email, the SMTP configuration needs to be provided.
Email configuration is managed by environment variables.
|ALERTS_SMTP_HOST||False||SMTP host for the email provider|
|ALERTS_SMTP_PORT||False||SMTP port for the email provider|
|ALERTS_SMTP_AUTH_ENABLED||False||true, if auth is enabled.|
|ALERTS_SMTP_USE_TLS||True||false if |
|ALERTS_SMTP_USE_SSL||True||false if |
|ALERTS_FROM_EMAIL||False||Email id from which the emails will be sent.|
|ALERTS_DEFAULT_TO_EMAILS||True||If provided, this will be used to configure stock alerts. This comma separated email ids will be configured as the senders email for the stock alert.|
|ALERTS_MAX_EMAIL_THRESHOLD||True||To prevent spamming, this value can be specified. The system will stop sending message for existing incident when this value is crossed. Defaults to 5 if not provided.|