Handy Tips #31: Detecting invalid metrics with Zabbix validation preprocessing

Anonymous Anonymous

Zabbix Certified Expert & Trainer

June 9, 2022

Monitor and react to unexpected or faulty outputs from your monitoring targets by using Zabbix validation preprocessing. In case of a failure, some monitoring endpoints like sensors or specific application or OS level counters can start outputting faulty metrics. Such behavior needs to be detected and reacted to as soon as possible. Use Zabbix preprocessing […]

Monitor and react to unexpected or faulty outputs from your monitoring targets by using Zabbix validation preprocessing.

In case of a failure, some monitoring endpoints like sensors or specific application or OS level counters can start outputting faulty metrics. Such behavior needs to be detected and reacted to as soon as possible.

Use Zabbix preprocessing to validate the collected metrics:

Select from and combine multiple preprocessing validation steps
Display a custom error message in case of an unexpected metric

Discard or change the value in case of an unexpected metric
Create an internal action to react to items becoming not supported

Check out the video to learn how to use preprocessing to detect invalid metrics.

Define preprocessing steps and react on invalid metrics:

Navigate to Configuration → Hosts and find your host
Click on the Items button
Find the item for which the preprocessing steps will be defined
Open the item and click on the Preprocessing tab
For our example, we will use the Temperature item
Select the In range preprocessing step
Define the min and max preprocessing parameters
Mark the Custom on fail checkbox
Press the Set error to button and enter your custom error message
Press the Update button
Simulate an invalid metric by sending an out-of-range value to this item
Navigate to Configuration → Hosts → Your Host → Items
Observe the custom error message being displayed next to your item

Tips and best practices

Validation preprocessing can check for errors in JSON, XML, or unstructured text with JSONPath, XPath, or Regex
User macros and low-level discovery macros can be used to define the In range validation values
The Check for not supported value preprocessing step is always executed as the first preprocessing step
Internal actions can be used to define action conditions and receive alerts about specific items receiving invalid metrics