Webhook

From the Netdata Cloud UI, you can manage your space’s notification settings and enable the configuration to deliver notifications on a webhook using a predefined schema.

Setup

Prerequisites

A Netdata Cloud account
Access to the Netdata Space as an administrator
The Netdata Space needs to be on Pro plan or higher
You need to have an app that allows you to receive webhooks following a predefined schema.

Netdata Configuration Steps

Click on the Space settings cog (located above your profile icon)
Click on the Notification tab
Click on the + Add configuration button (near the top-right corner of your screen)
On the Webhook card click on + Add
A modal will be presented to you to enter the required details to enable the configuration:

Notification settings are Netdata specific settings
- Configuration name - you can optionally provide a name for your configuration you can easily refer to it
- Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
- Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
Integration configuration are the specific notification integration required settings, which vary by notification method. For Webhook:
- Webhook URL - webhook URL is the url of the service that Netdata will send notifications to. In order to keep the communication secured, we only accept HTTPS urls.
- Extra headers - these are optional key-value pairs that you can set to be included in the HTTP requests sent to the webhook URL.
- Authentication Mechanism - Netdata webhook integration supports 3 different authentication mechanisms.
  - Mutual TLS (recommended) - default authentication mechanism used if no other method is selected.
  - Basic - the client sends a request with an Authorization header that includes a base64-encoded string in the format username:password. These will settings will be required inputs.
  - Bearer - the client sends a request with an Authorization header that includes a bearer token. This setting will be a required input.

Webhook service

A webhook integration allows your application to receive real-time alerts from Netdata by sending HTTP requests to a specified URL. In this document, we’ll go over the steps to set up a generic webhook integration, including adding headers, and implementing different types of authorization mechanisms.

Netdata webhook integration

A webhook integration is a way for one service to notify another service about events that occur within it. This is done by sending an HTTP POST request to a specified URL (known as the “webhook URL”) when an event occurs.

Netdata webhook integration service will send alert notifications to the destination service as soon as they are detected.

The notification content sent to the destination service will be a JSON object having these properties:

field	type	description
message	string	A summary message of the alert.
alarm	string	The alarm the notification is about.
info	string	Additional info related with the alert.
chart	string	The chart associated with the alert.
context	string	The chart context.
space	string	The space where the node that raised the alert is assigned.
rooms	object[object(string,string)]	Object with list of rooms names and urls where the node belongs to.
family	string	Context family.
class	string	Classification of the alert, e.g. “Error”.
severity	string	Alert severity, can be one of “warning”, “critical” or “clear”.
date	string	Date of the alert in ISO8601 format.
duration	string	Duration the alert has been raised.
additional_active_critical_alerts	integer	Number of additional critical alerts currently existing on the same node.
additional_active_warning_alerts	integer	Number of additional warning alerts currently existing on the same node.
alarm_url	string	Netdata Cloud URL for this alarm.

Extra headers

When setting up a webhook integration, the user can specify a set of headers to be included in the HTTP requests sent to the webhook URL.

By default, the following headers will be sent in the HTTP request

Header	Value
Content-Type	application/json

Authentication mechanisms

Netdata webhook integration supports 3 different authentication mechanisms:

Mutual TLS authentication (recommended)

In mutual Transport Layer Security (mTLS) authentication, the client and the server authenticate each other using X.509 certificates. This ensures that the client is connecting to the intended server, and that the server is only accepting connections from authorized clients.

This is the default authentication mechanism used if no other method is selected.

To take advantage of mutual TLS, you can configure your server to verify Netdata’s client certificate. In order to achieve this, the Netdata client sending the notification supports mutual TLS (mTLS) to identify itself with a client certificate that your server can validate.

The steps to perform this validation are as follows:

Store Netdata CA certificate on a file in your disk. The content of this file should be:

Netdata CA certificate

Enable client certificate validation on the web server that is doing the TLS termination. Below we show you how to perform this configuration in NGINX and Apache

NGINX

server {
    listen 443 ssl default_server;

    # ... existing SSL configuration for server authentication ...
    ssl_verify_client on;
    ssl_client_certificate /path/to/Netdata_CA.pem;

    location / {
        if ($ssl_client_s_dn !~ "CN=app.netdata.cloud") {
            return 403;
        }
       # ... existing location configuration ...
    }
}

Apache

Listen 443
<VirtualHost *:443>
    # ... existing SSL configuration for server authentication ...
    SSLVerifyClient require
    SSLCACertificateFile "/path/to/Netdata_CA.pem"
</VirtualHost>
<Directory /var/www/>
    Require expr "%{SSL_CLIENT_S_DN_CN} == 'app.netdata.cloud'"
    # ... existing directory configuration ...
</Directory>

Basic authentication

In basic authorization, the client sends a request with an Authorization header that includes a base64-encoded string in the format username:password. The server then uses this information to authenticate the client. If this authentication method is selected, the user can set the user and password that will be used when connecting to the destination service.

Bearer token authentication

In bearer token authentication, the client sends a request with an Authorization header that includes a bearer token. The server then uses this token to authenticate the client. Bearer tokens are typically generated by an authentication service, and are passed to the client after a successful authentication. If this method is selected, the user can set the token to be used for connecting to the destination service.

Challenge secret

To validate that you has ownership of the web application that will receive the webhook events, we are using a challenge response check mechanism.

This mechanism works as follows:

The challenge secret parameter that you provide is a shared secret between you and Netdata only.
On your request for creating a new Webhook integration, we will make a GET request to the url of the webhook, adding a query parameter crc_token, consisting of a random string.
You will receive this request on your application and it must construct an encrypted response, consisting of a base64-encoded HMAC SHA-256 hash created from the crc_token and the shared secret. The response will be in the format:

{
  "response_token": "sha256=9GKoHJYmcHIkhD+C182QWN79YBd+D+Vkj4snmZrfNi4="
}

We will compare your application’s response with the hash that we will generate using the challenge secret, and if they are the same, the integration creation will succeed.

We will do this validation everytime you update your integration configuration.

Response requirements:
- A base64 encoded HMAC SHA-256 hash created from the crc_token and the shared secret.
- Valid response_token and JSON format.
- Latency less than 5 seconds.
- 200 HTTP response code.

Example response token generation in Python:

Here you can see how to define a handler for a Flask application in python 3:

import base64
import hashlib
import hmac
import json

key ='YOUR_CHALLENGE_SECRET'

@app.route('/webhooks/netdata')
def webhook_challenge():
  token = request.args.get('crc_token').encode('ascii')

  # creates HMAC SHA-256 hash from incomming token and your consumer secret
  sha256_hash_digest = hmac.new(key.encode(),
                                msg=token,
                                digestmod=hashlib.sha256).digest()

  # construct response data with base64 encoded hash
  response = {
    'response_token': 'sha256=' + base64.b64encode(sha256_hash_digest).decode('ascii')
  }

  # returns properly formatted json response
  return json.dumps(response)

Industry

Technology

Use cases

Webhook

Webhook

Setup

Prerequisites

Netdata Configuration Steps

Webhook service

Netdata webhook integration

Extra headers

Authentication mechanisms

Mutual TLS authentication (recommended)

Basic authentication

Bearer token authentication

Challenge secret

Get Netdata