Azure Cognitive Services: Anomaly Detector
Author: Ronald Fung
Creation Date: 31 May 2023
Next Modified Date: 31 May 2024
A. Introduction
Anomaly Detector is an AI service with a set of APIs, which enables you to monitor and detect anomalies in your time series data with little machine learning (ML) knowledge, either batch validation or real-time inference.
Anomaly Detector capabilities
With Anomaly Detector, you can either detect anomalies in one variable using Univariate Anomaly Detector, or detect anomalies in multiple variables with Multivariate Anomaly Detector.
Feature |
Description |
|---|---|
Univariate Anomaly Detection |
Detect anomalies in one variable, like revenue, cost, etc. The model was selected automatically based on your data pattern. |
Multivariate Anomaly Detection |
Detect anomalies in multiple variables with correlations, which are usually gathered from equipment or other complex system. The underlying model used is a Graph Attention Network. |
Univariate Anomaly Detection
The Univariate Anomaly Detection API enables you to monitor and detect abnormalities in your time series data without having to know machine learning. The algorithms adapt by automatically identifying and applying the best-fitting models to your data, regardless of industry, scenario, or data volume. Using your time series data, the API determines boundaries for anomaly detection, expected values, and which data points are anomalies.
Using the Anomaly Detector doesn’t require any prior experience in machine learning, and the REST API enables you to easily integrate the service into your applications and processes.
With the Univariate Anomaly Detector, you can automatically detect anomalies throughout your time series data, or as they occur in real-time.
Feature |
Description |
|---|---|
Streaming detection |
Detect anomalies in your streaming data by using previously seen data points to determine if your latest one is an anomaly. This operation generates a model using the data points you send, and determines if the target point is an anomaly. By calling the API with each new data point you generate, you can monitor your data as it’s created. |
Batch detection |
Use your time series to detect any anomalies that might exist throughout your data. This operation generates a model using your entire time series data, with each point analyzed with the same model. |
Change points detection |
Use your time series to detect any trend change points that exist in your data. This operation generates a model using your entire time series data, with each point analyzed with the same model. |
Multivariate Anomaly Detection
The Multivariate Anomaly Detection APIs further enable developers by easily integrating advanced AI for detecting anomalies from groups of metrics, without the need for machine learning knowledge or labeled data. Dependencies and inter-correlations between up to 300 different signals are now automatically counted as key factors. This new capability helps you to proactively protect your complex systems such as software applications, servers, factory machines, spacecraft, or even your business, from failures.
Imagine 20 sensors from an auto engine generating 20 different signals like rotation, fuel pressure, bearing, etc. The readings of those signals individually may not tell you much about system level issues, but together they can represent the health of the engine. When the interaction of those signals deviates outside the usual range, the multivariate anomaly detection feature can sense the anomaly like a seasoned expert. The underlying AI models are trained and customized using your data such that it understands the unique needs of your business. With the new APIs in Anomaly Detector, developers can now easily integrate the multivariate time series anomaly detection capabilities into predictive maintenance solutions, AIOps monitoring solutions for complex enterprise software, or business intelligence tools.
B. How is it used at Seagen
As a biopharma research company that uses Microsoft Azure, you can use Azure Cognitive Services: Anomaly Detector to help detect and diagnose anomalies in your data. Here are some ways you can use Azure Cognitive Services: Anomaly Detector:
Detect anomalies in clinical trial data: You can use Azure Cognitive Services: Anomaly Detector to detect anomalies in clinical trial data, such as patient vitals, medication dosages, and lab results. This can help you identify potential issues with the trial and make data-driven decisions to improve patient outcomes.
Diagnose manufacturing issues: You can use Azure Cognitive Services: Anomaly Detector to diagnose anomalies in manufacturing data, such as equipment performance, product quality, and supply chain logistics. This can help you identify potential issues with the manufacturing process and make data-driven decisions to improve efficiency and reduce costs.
Monitor environmental sensors: You can use Azure Cognitive Services: Anomaly Detector to monitor anomalies in environmental sensor data, such as temperature, humidity, and air quality. This can help you identify potential issues with the environment and make data-driven decisions to improve safety and compliance.
Analyze financial transactions: You can use Azure Cognitive Services: Anomaly Detector to analyze anomalies in financial transaction data, such as credit card transactions, insurance claims, and stock market data. This can help you identify potential fraud or compliance issues and make data-driven decisions to mitigate risks and improve financial performance.
Overall, Azure Cognitive Services: Anomaly Detector provides a powerful and flexible tool for detecting and diagnosing anomalies in your data. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately detect anomalies, identify potential issues, and make data-driven decisions to improve your business outcomes.
C. Features
Azure Cognitive Services: Anomaly Detector is a machine learning-based service that enables you to monitor and detect anomalies in your time-series data. Here are some of the key features of Azure Cognitive Services: Anomaly Detector:
Easy to use: Azure Cognitive Services: Anomaly Detector is easy to use, with a simple REST API that allows you to quickly integrate the service into your applications and workflows.
Customizable: Azure Cognitive Services: Anomaly Detector allows you to customize the service to your specific needs, including setting the sensitivity of anomaly detection, defining the time window for analysis, and specifying the type of data to analyze.
Scalable: Azure Cognitive Services: Anomaly Detector is scalable, allowing you to analyze large volumes of data and handle high traffic loads.
Accurate: Azure Cognitive Services: Anomaly Detector uses machine learning algorithms to analyze your data and accurately detect anomalies, reducing false positives and false negatives.
Flexible: Azure Cognitive Services: Anomaly Detector is flexible, allowing you to use it with a wide range of data types, including numerical, categorical, and text data.
Real-time detection: Azure Cognitive Services: Anomaly Detector provides real-time detection of anomalies, allowing you to quickly identify potential issues and take action.
Integration with Azure services: Azure Cognitive Services: Anomaly Detector is integrated with other Azure services, such as Azure Stream Analytics and Azure Time Series Insights, allowing you to easily incorporate anomaly detection into your existing workflows.
Overall, Azure Cognitive Services: Anomaly Detector provides a powerful and flexible tool for detecting and diagnosing anomalies in your time-series data. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately detect anomalies, identify potential issues, and make data-driven decisions to improve your business outcomes.
D. Where Implemented
E. How it is tested
Testing Azure Cognitive Services: Anomaly Detector involves verifying that the service is properly configured and that it can accurately detect anomalies in your time-series data. Here are some steps you can take to test Azure Cognitive Services: Anomaly Detector:
Verify configuration: Verify that Azure Cognitive Services: Anomaly Detector is properly configured and integrated with your Azure account and resources.
Test performance: Test Azure Cognitive Services: Anomaly Detector by uploading sample data and verifying that it can accurately detect anomalies in the data. You can also test the performance of the service by analyzing large volumes of data and measuring the time it takes to detect anomalies.
Test accuracy: Test the accuracy of Azure Cognitive Services: Anomaly Detector by comparing the results of the service with manually identified anomalies in your data. You can also test the accuracy of the service by injecting synthetic anomalies into your data and verifying that the service detects them.
Test customization: Test the customization options of Azure Cognitive Services: Anomaly Detector by adjusting the sensitivity of anomaly detection, defining the time window for analysis, and specifying the type of data to analyze. Verify that the service accurately detects anomalies according to your specifications.
Test scalability: Test the scalability of Azure Cognitive Services: Anomaly Detector by analyzing large volumes of data and measuring the time it takes to detect anomalies. You can also test the scalability of the service by analyzing data in real-time and measuring the response time of the service.
Test integration: Test the integration of Azure Cognitive Services: Anomaly Detector with other Azure services and resources by verifying that data and workflows are properly integrated and that there are no compatibility issues.
Test documentation: Test the documentation of Azure Cognitive Services: Anomaly Detector by verifying that it is up-to-date, accurate, and comprehensive.
Overall, testing Azure Cognitive Services: Anomaly Detector involves verifying that the service is properly configured and functioning as expected, testing performance, accuracy, customization, scalability, integration, and documentation. By testing Azure Cognitive Services: Anomaly Detector, you can ensure that you are effectively using the service to monitor and detect anomalies in your time-series data, and that you are benefiting from the accuracy, flexibility, and scalability it provides.
F. 2023 Roadmap
????
G. 2024 Roadmap
????
H. Known Issues
As with any software or service, there may be known issues or limitations that users should be aware of when using Azure Cognitive Services: Anomaly Detector. Here are some of the known issues for Azure Cognitive Services: Anomaly Detector:
Data preprocessing: Azure Cognitive Services: Anomaly Detector requires time-series data to be preprocessed before it can be analyzed. This can be time-consuming and can limit the flexibility of the service for certain types of data.
Limited customization: Azure Cognitive Services: Anomaly Detector has limited customization options, which can limit the ability of users to configure the service to their specific needs.
Limited data types: Azure Cognitive Services: Anomaly Detector is designed to analyze time-series data, which can limit its usefulness for certain types of data, such as non-time-series data or data that requires more complex analysis.
Sensitivity: The sensitivity of anomaly detection in Azure Cognitive Services: Anomaly Detector can be difficult to configure, which can result in false positives or false negatives.
Cost: Azure Cognitive Services: Anomaly Detector can be expensive for users with limited budgets, particularly if they use it frequently or for large volumes of data.
Limited integration: Azure Cognitive Services: Anomaly Detector has limited integration with third-party tools and services, which can limit the ability of users to incorporate it into their existing workflows.
Overall, while Azure Cognitive Services: Anomaly Detector offers a powerful and flexible tool for detecting and diagnosing anomalies in time-series data, users must be aware of these known issues and take steps to mitigate their impact. This may include carefully preprocessing data to ensure that it can be properly analyzed, carefully configuring the service to meet the specific needs of their data, carefully monitoring the cost and sensitivity of the service to ensure that it is a good fit for their budget and data requirements, and carefully integrating the service into their existing workflows to ensure that it is effectively utilized. By taking these steps, users can ensure that they are effectively using Azure Cognitive Services: Anomaly Detector to monitor and detect anomalies in their time-series data and that they are benefiting from the accuracy, flexibility, and scalability it provides.
[x] Reviewed by Enterprise Architecture
[x] Reviewed by Application Development
[x] Reviewed by Data Architecture