Configuring webhook by using the mmhealth command

Configure a web server for a user, which accepts webhook POST requests from the IBM Storage Scale mmhealth command. The procedure includes the HTTP server authentication process, and how it validates UUID value and acts based on the incoming events.

A webhook is a basic API that allows one-way sharing of data between a client and an HTTP server. In most cases, the data is sent when an event or a trigger mechanism occurs. When a webhook is configured in IBM Storage Scale by using the mmhealth command, health events occur during the system health monitoring. These health events trigger the one-way transfer of data to the HTTP server that is defined in the webhook.

Setting up an HTTP server

Consider an HTTP server that responds only on a specific webhook API. When the mmhealth command posts data to this webhook API, the HTTP server prints a simple summary of the received health events. The posted data can be examined for specific health events and based on the health events, further actions can be taken. For example, if a disk_down health event is encountered, then the HTTP server can trigger an email or alarm mechanism to inform the service team to take immediate action.

You can use programming languages, such as Rust, GO, or Python to configure a webhook server.

Example 1: Setting up a webhook server by using Rust


#![allow(non_camel_case_types)]
#![allow(non_snake_case)]

use actix_web::{web, App, HttpResponse, HttpServer};
use serde::Deserialize;
use std::collections::HashMap;
use std::env;

#[derive(Deserialize)]
struct HealthEvent {
  cause: String,
  code: String,
  component: String,
  container_restart: bool,
  container_unready: bool,
  description: String,
  entity_name: String,
  entity_type: String,
  event: String,
  event_type: String,
  ftdc_scope: String,
  identifier: String,
  internalComponent: String,
  is_resolvable: bool,
  message: String,
  node: String,
  priority: u64,
  remedy: Option<String>,
  requireUnique: bool,
  scope: String,
  severity: String,
  state: String,
  time: String,
  TZONE: String,
  user_action: String,
}

#[derive(Deserialize)]
struct PostMsg {
  version: String,
  reportingController: String,
  reportingInstance: String,
  events: Vec<HealthEvent>,
}

fn post_handler(events: web::Json<PostMsg>) -> HttpResponse {
  let mut counts: HashMap<String, u64> = HashMap::new();
  for e in &events.events {
    *counts.entry(e.severity.clone()).or_insert(0) += 1;
  }
  println!("{}: {:?}", events.reportingInstance, counts);
  HttpResponse::Ok().content_type("text/html").body("OK")
}

pub fn main() {
  let args: Vec<String> = env::args().collect();
  if args.len() != 2 {
    println!("Usage: {} IPAddr:Port", args[0]);
    return;
  }

  println!("Listening on {}", args[1]);
  HttpServer::new(|| App::new().route("/webhook", web::post().to(post_handler)))
    .bind(&args[1])
    .expect("error binding server to IP address:port")
    .run()
    .expect("error running server");
}

Example 2: Setting up a webhook server by using GO


import (
       "encoding/json"
       "fmt"
       "log"
       "net/http"
       "os"
       "time"
)

type healthEvent struct {
      Cause string `json:"cause"`
      Code string `json:"code"`
      Component string `json:"component"`
      ContainerRestart bool `json:"container_restart"`
      ContainerUnready bool `json:"container_unready"`
      Description string `json:"description"`
      EntityName string `json:"entity_name"`
      EntityType string `json:"entity_type"`
      Event string `json:"event"`
      EventType string `json:"event_type"`
      FTDCScope string `json:"ftdc_scope"`
      Identifier string `json:"identifier"`
      InternalComponent string `json:"internalComponent"`
      IsResolvable bool `json:"is_resolvable"`
      Message string `json:"message"`
      Node string `json:"node"`
      Priority int `json:"priority"`
      Remedy string `json:"remedy"`
      RequireUnique bool `json:"requireUnique"`
      Scope string `json:"scope"`
      Severity string `json:"severity"`
      State string `json:"state"`
      Time time.Time `json:"time"`
      TZone string `json:"TZONE"`
      UserAction string `json:"user_action"`
}

type postMsg struct {
      Version string `json:"version"`
      ReportingController string `json:"reportingController"`
      ReportingInstance string `json:"reportingInstance"`
      Events []healthEvent `json:"events"`
}

func webhook(w http.ResponseWriter, r *http.Request) {
       decoder := json.NewDecoder(r.Body)
       var data postMsg
       if err := decoder.Decode(&data); err != nil {
              fmt.Println(err)
              return
       }
       counts := make(map[string]int)
       for _, event := range data.Events {
             _, matched := counts[event.Severity]
             switch matched {
             case true:
                   counts[event.Severity] += 1
             case false:
                   counts[event.Severity] = 1
             }
       }
       fmt.Println(data.ReportingInstance, ":", counts)
}

func main() {
       http.HandleFunc("/webhook", webhook)

       if len(os.Args) != 2 {
              log.Fatal(fmt.Errorf("usage: %s IPAddr:Port", os.Args[0]))
       }
       fmt.Printf("Starting server listening on %s ...\n", os.Args[1])
       if err := http.ListenAndServe(os.Args[1], nil); err != nil {
              log.Fatal(err)
       }
}

The following example shows starting the GO program that is compiled to a binary named webhook. You must provide the IP address and port number the HTTP server can use.


[httpd-dev]# ./webhook 192.0.2.48:9000
Starting server listening on 192.0.2.48:9000 …

Example 3: Setting up a webhook server by using Python


#!/usr/bin/env python3
import argparse
from collections import Counter
import cherrypy
import json

class DataView(object):
       exposed = True
       @cherrypy.tools.accept(media='application/json')
       def POST(self):
              rawData = cherrypy.request.body.read(int(cherrypy.request.headers['Content-Length'])) 
              b = json.loads(rawData)
              eventCounts = Counter([e['severity'] for e in b['events']]) 
              print(f"{b['reportingInstance']}: {json.dumps(eventCounts)}")
              if dump_json: print(json.dumps(b, indent=4))
              return "OK"

if __name__ == '__main__':
      parser = argparse.ArgumentParser()
      parser.add_argument('--json', action='store_true')
      parser.add_argument('IP', type=str)
      parser.add_argument('PORT', type=int)
      args = parser.parse_args()  
      conf = {
             '/': {'request.dispatch': cherrypy.dispatch.MethodDispatcher(),}
      }
      dump_json = args.json  
      cherrypy.server.socket_host = args.IP
      cherrypy.server.socket_port = args.PORT
      cherrypy.quickstart(DataView(), '/webhook/', conf)

Configuring a webhook

In an IBM Storage Scale cluster, you can configure the mmhealth framework to interact with the webhook server by using the mmhealth config webhook add command. For more information, see mmhealth command.

For example, run the mmhealth command by using the IP address 192.0.2.48 and port 9000 that was used when starting up the webhook server.
[root@frodo-master ~]# mmhealth config webhook add http://192.0.2.48:9000/webhook
Successfully connected to http://192.0.2.48:9000/webhook
To list all webhooks that are currently configured, use the command:
[root@frodo-master ~]# mmhealth config webhook list http://192.0.2.48:9000/webhook
You can add the -Y option for extended information and to make the output that is more easily processed by other programs and scripts:
[root@frodo-master ~]# mmhealth config webhook list -Y
mmhealth:webhook:HEADER:version:reserved:reserved:url:uuid:status:
mmhealth:webhook:0:1:::http%3A//192.0.2.48%3A9000/webhook:af0f2f36-771e-4e3d-930d-40bc30ab41f9:enabled:

The –Y output shows the UUID value that is associated with this webhook. The UUID value is set in the HTTP POST header when the mmhealth command posts to the webhook.

When the webhook is configured in the mmhealth framework, the webhook server starts to receive health events.

Note: Events are sent only when a health event is triggered in IBM Storage Scale.
[httpd-dev]# ./webhook 192.0.2.48:9000
Starting server listening on 192.0.2.48:9000 ...
frodo-master.fyre.ibm.com : map[INFO:29]
frodo-master.fyre.ibm.com : map[INFO:8]
frodo-master.fyre.ibm.com : map[INFO:3]
frodo-master.fyre.ibm.com : map[INFO:9]
frodo-master.fyre.ibm.com : map[INFO:1]
frodo-master.fyre.ibm.com : map[INFO:4]
frodo-master.fyre.ibm.com : map[INFO:2]
frodo-master.fyre.ibm.com : map[INFO:2]
Important: If the mmhealth webhook framework faces issues with the configured webhook URL, then it is disabled over time with a default setting of 24 hours. The -Y output shows the enabled or disabled status of each webhook URL. If the webhook URL gets disabled, then rerun the mmhealth config webhook command to readd the URL.