Dynatrace
Dynatrace
Dynatrace allows you to receive notifications using their Events REST API. See the Dynatrace documentation about POSTing an event in the Events API for more details. You can send notifications to Dynatrace using Netdata’s Agent alert notification feature, which supports dozens of endpoints, user roles, and more.
Setup
Prerequisites
- A Dynatrace Server. You can use the same on all your Netdata servers but make sure the server is network visible from your Netdata hosts. The Dynatrace server should be with protocol prefixed (http:// or https://), for example: https://monitor.example.com.
- An API Token. Generate a secure access API token that enables access to your Dynatrace monitoring data via the REST-based API. See Dynatrace API - Authentication for more details.
- An API Space. This is the URL part of the page you have access in order to generate the API Token. For example, the URL for a generated API token might look like: https://monitor.illumineit.com/e/2a93fe0e-4cd5-469a-9d0d-1a064235cfce/#settings/integration/apikeys;gf=all In that case, the Space is 2a93fe0e-4cd5-469a-9d0d-1a064235cfce.
- A Server Tag. To generate one on your Dynatrace Server, go to Settings –> Tags –> Manually applied tags and create the Tag. The Netdata alarm is sent as a Dynatrace Event to be correlated with all those hosts tagged with this Tag you have created.
- Terminal access to the Agent you wish to configure
Configuration
File
The configuration file name for this integration is health_alarm_notify.conf.
You can edit the configuration file using the edit-config script from the
Netdata config directory.
cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config health_alarm_notify.conf
Options
The following options can be defined for this notification
| Name | Description | Default | Required |
|---|---|---|---|
| SEND_DYNATRACE | Set SEND_DYNATRACE to YES |
YES | True |
| DYNATRACE_SERVER | Set DYNATRACE_SERVER to the Dynatrace server with the protocol prefix, for example https://monitor.example.com. |
True | |
| DYNATRACE_TOKEN | Set DYNATRACE_TOKEN to your Dynatrace API authentication token |
True | |
| DYNATRACE_SPACE | Set DYNATRACE_SPACE to the API Space, it is the URL part of the page you have access in order to generate the API Token. |
True | |
| DYNATRACE_TAG_VALUE | Set DYNATRACE_TAG_VALUE to your Dynatrace Server Tag. |
True | |
| DYNATRACE_ANNOTATION_TYPE | DYNATRACE_ANNOTATION_TYPE can be left to its default value Netdata Alarm, but you can change it to better fit your needs. |
Netdata Alarm | False |
| DYNATRACE_EVENT | Set DYNATRACE_EVENT to the Dynatrace eventType you want. |
Netdata Alarm | False |
DYNATRACE_SPACE
For example, the URL for a generated API token might look like: https://monitor.illumineit.com/e/2a93fe0e-4cd5-469a-9d0d-1a064235cfce/#settings/integration/apikeys;gf=all In that case, the Space is 2a93fe0e-4cd5-469a-9d0d-1a064235cfce.
DYNATRACE_EVENT
AVAILABILITY_EVENT, CUSTOM_ALERT, CUSTOM_ANNOTATION, CUSTOM_CONFIGURATION, CUSTOM_DEPLOYMENT, CUSTOM_INFO, ERROR_EVENT,
MARKED_FOR_TERMINATION, PERFORMANCE_EVENT, RESOURCE_CONTENTION_EVENT.
You can read more here.
Examples
Basic Configuration
#------------------------------------------------------------------------------
# Dynatrace global notification options
SEND_DYNATRACE="YES"
DYNATRACE_SERVER="https://monitor.example.com"
DYNATRACE_TOKEN="XXXXXXX"
DYNATRACE_SPACE="2a93fe0e-4cd5-469a-9d0d-1a064235cfce"
DYNATRACE_TAG_VALUE="SERVERTAG"
DYNATRACE_ANNOTATION_TYPE="Netdata Alert"
DYNATRACE_EVENT="AVAILABILITY_EVENT"
Troubleshooting
Test Notification
You can run the following command by hand, to test alerts configuration:
# become user netdata
sudo su -s /bin/bash netdata
# enable debugging info on the console
export NETDATA_ALARM_NOTIFY_DEBUG=1
# send test alarms to sysadmin
/usr/libexec/netdata/plugins.d/alarm-notify.sh test
# send test alarms to any role
/usr/libexec/netdata/plugins.d/alarm-notify.sh test "ROLE"
Note that this will test all alert mechanisms for the selected role.