How do I configure Webhook notifications for Cribl?

1. Send Canary Console Webhooks into Cribl Stream 

1.1 Configure a HTTP Source in Cribl Stream

• Select Products.

• Select Worker Groups.

  

• Select the Worker Group you'd like to set up the HTTP source under. We've selected the default - you will most likely have more than just the one Worker Group.

• Go to Data -> Sources 

  

• Select HTTP from the list of available sources.

• At this point you can either add a new HTTP source or edit an existing one. In this example we will edit the default HTTP source.

• Open the http source configuration and add an Auth Token.

• Generate a new Token or use an existing one.
• Set a description for the Auth Token, for example: Thinkst Canary HTTP auth Token.

• Save your changes, then commit and deploy your new configuration.

1.2 Create a Webhook Destination in Canary Console

• In your Canary Console, go to Global Settings -> Webhooks -> + (Add Webhook) -> Generic Webhook.

• Here we're going to use the Ingress Address from our Cribl Worker Group overview in our webhook URL:

• Prepend https:// to the Ingress address and append :10080/cribl/_bulk.
• Add a custom header named Authorization, and set the value to the Auth Token you created earlier. 

• Click Save.

   

2. Send a Test Payload From the Canary Console

• Return to the HTTP source you configured in Cribl and click Live to watch incoming webhook alerts in real time.

• Use the Capture button to keep the live capture open for 60 seconds and start the capture.

• Now go back to your Canary Console, open the webhook you created, and click Test webhook.

• The test alert payload will appear in the Live Data window, similar to the example below:

3. (Optional) Store Webhook Data in Cribl Lake

This section is optional and demonstrates how to store webhook data in Cribl Lake.

3.1 Create a Dataset in Cribl Lake for Canary Alerts

• In Cribl Lake, go to Datasets.

• Click Add Dataset in the upper-right corner

• Create the new Dataset:
  • Give it a name, for example: canary_dataset
  • Give it a description, for example: Thinkst Canary Dataset
  • Save the new Dataset.

 3.2 Configure a Lake Destination

• In Cribl Stream, go to Worker Groups -> Data -> Destinations.

• Choose Cribl Lake and click Add Destination.

Configure the destination:

• Give it a name, for example: canary_lake_dest
• Give it a description, for example: Canary Lake Destination
• Select the Lake dataset we created earlier, in this case  we will use canary_dataset
• Leave everything else as default.
• Save your changes, then commit and deploy your new configuration.

• We now need to route the incoming webhook data to our Cribl Lake destination.

3.3 Create a Pipeline for Canary Webhook Events

• Go to Processing -> Pipelines-> Add Pipeline.

• Set an ID, for example: canary_webhook_pipeline.
• Set a Description, for example: Thinkst Canary webhook Pipeline
• Save your changes, then commit and deploy your new configuration.

3.4 Route Canary Events to Cribl Lake

• Go to Routing -> Data Routes -> Add Route:

• Set a route name, for example: canary_webhook_to_lake_route
• For the filter we will filter on the input ID of the HTTP source we have configured.
  • In this case it will be __inputId.startsWith('http:http:')

• Set a Pipeline, which in this case we will use the one we created earlier called canary_webhook_pipeline.
• Set the destination to the Cribl Lake we created earlier, in our example, this will be cribl_lake:canary_lake_dest.
• Set the description, for example: Route from http input to Cribl Lake.
• Lastly move this route above your other data routes.

• Save your changes, then commit and deploy your new configuration.

3.5 Test the End-to-End Flow

• In the Cribl UI, go to Lake -> Datasets
• Select the Lake Dataset you configured for example canary_dataset and click Search.
  • Note: It can take up to 5-10 minutes for your alerts to show up in the dataset search.

• Send over some test alerts using the built in webhook test function or trigger real Canary alerts.
• You should now see all your alert data flowing into Cribl Lake.