Anomaly Detection

What is anomaly detection?

Anomaly detection, also known as outlier analysis or outlier detection, is the process of identifying data points or events that diverge significantly from the majority of the dataset. Developers, operations teams, and other stakeholders rely on a number of anomaly detection techniques, powered by machine learning and AI, to pinpoint bugs, glitches, and rare events. This helps teams to identify new business opportunities and drive conversion rates.

Most anomaly detection methods look at 1 of 3 broad categories: global outliers, contextual outliers, and collective outliers.

• Global outliers are generally single points or a small cluster of points of a very high or very low value that deviate significantly from the majority of the data.
• Contextual outliers occur when a value or data point deviates from the majority of the data set when viewed in the same context.

Collective outliers, on the other hand, are a subset of data points that deviate or are anomalous to the entire dataset.

Using anomaly detection to uncover user struggle and customer friction

For companies to conduct anomaly detection and better understand consumer behavior, they must collect customer data, measure key metrics, and perform a statistical analysis. You might be thinking that outliers and anomalies are always a bad thing, but that’s not necessarily the case—they’re simply a deviation from what is expected. Depending on the use case, an anomaly can actually be a positive signal, such as a sudden surge in sales or traffic to your website. Your team can employ anomaly detection to better understand every single aspect of your company’s business activity, especially unexpected changes in metrics such as:

• Web page views
• Daily active users
• Bounce rate
• Volume of transactions
• Cart abandonment rate
• Macro and micro conversion rates

undefined

Who uses anomaly detection tools?

IT/ops, digital marketers, UX designers, developers, and other stakeholders can benefit from using anomaly detection technology.

Anomaly detection use cases

There are a number of anomaly detection use cases across industries such as retail, financial services, banking, travel, and media.

• Detect Fraud: Anomaly detection technology can identify unusual user activity from bots, or nonhuman traffic, such as copying-&-pasting information into a form hundreds of times or making hundreds of login attempts.
• Enhance digital products: With the help of anomaly detection tools, teams can identify instances of friction in the user experience, especially design flaws that decrease conversion rates. By quickly identifying moments of friction with anomaly detection technology, teams can continue improving their user experience based on actual user data.
• Monitor key performance indicators (KPI): Anomaly detection can also help teams to understand sudden changes in bounce rates, conversion rates, checkout success, and other conversion blocking issues. Teams can assess API errors, load-time glitches, server downtime, and more.

Anomaly detection tools and software

Some examples of anomaly detection tools and software include:

• Quantum Metric
• Numenta
• AVORA
• Splunk Enterprise
• Loom Systems
• Elastic X-Pack
• Anodot
• CrunchMetrics

Many anomaly detection tools come with features such as:

• Behavior monitoring, which allows teams to spot unusual patterns, behaviors, events, and network traffic.
• Real time alerts, which notify teams of any critical alerts in real-time by sending alerts via email, Slack, and other third-party services.
• Dashboards, which enable teams to display metrics and visualize important data using charts, lines, and bars.