Context Anomaly Detection

Context Anomaly Detection

Anomaly detection

Anomalies are something deviating from usual. Anomalies in online e-commerce businesses visitor metrics may indicate something good or bad about the monitored web site. Companies can use this information to react accordingly. It is a vital part of monitoring online business activity. To gain better insight into data anomalies the context of the data can be discovered and predicted using software Artificial Intelligence. (A.I.)

Detecting anomalies in business data offers huge potential to uncover activity such as success, faults, and User Experience (UX), as well as offering future insights for businesses to plan for. Applying anomaly detection creatively and thoughtfully widens scope dramatically.

RAW data

Anomaly detection methods implemented frequently used in anomaly detection software applies rules to RAW data, in other words, data without context. Inconsistencies in RAW data causes problems for anomaly detection in the following way:

Example of data showing repeating trends.

RAW data metrics graph.
RAW data trace with regular trends.

The data has a large dynamic range yet the data range in given periods is less dynamic. Outliers or anomaly’s can be detected only in areas on the graph outside of the data value range limits.

RAW data anomalies are found in the areas in red.
RAW data anomalies are found in the areas in red.

This technique shows areas on the chart that would be anomalous values, but are not covered in the detection rule area. In this example, values at more than half or twice their usual values would not be detected as anomalous in some cases. Adjusting the rule to account for those values would result in many false positive alerts being generated.

Context data

The data needs context. Machine learning and AI comes in especially useful for the task of gaining data context and trends to set expectations and make predictions about data coming in.
Assume the data is not yet collected. With the information gathered from processing historical data, predictions can be made about the range of data values expected.

A.I. predicted value range expectations
A.I. predicted value range expectations.

The predicted data value ranges can be used as the basis for anomaly detection rules.

The anomalous areas on the chart that were unreachable in the raw data are now anomalous areas according to their trend and context.

Context driven anomaly data areas in red.
Context driven anomaly data areas in red.

Monitoring rules

Rules can be created that monitor how far into the red the blue line might go. Anomalies found in the red area values are appropriate to the context of the expected metrics values.

Benefits

This method not only improves the quality of the alerts generated by providing more context for the alert, but also prevents generally unproductive and possibly misleading false positives.

Summary

Having worked with Anodot using their AI anomaly detection on models that feed fast and vital metrics to large business web campaign stakeholders, I have benefited from the accuracy of their predictive technology. Anodot are perhaps the envy of the anomaly detection industry for outstanding AI and machine learning software performance. In my experience, Anodot’s products reflected that sentiment very well.

To get the best data insights from detected anomalies, use a top quality, well tested, known working and efficient solution such as Anodot’s AI anomaly detection.

© Graft Computing Ltd.

Tags: