Azure Cognitive Services: Custom Vision
Author: Ronald Fung
Creation Date: 31 May 2023
Next Modified Date: 31 May 2024
A. Introduction
Azure Custom Vision is an image recognition service that lets you build, deploy, and improve your own image identifier models. An image identifier applies labels to images, according to their visual characteristics. Each label represents a classification or object. Unlike the Computer Vision service, Custom Vision allows you to specify your own labels and train custom models to detect them.
Tip
The Azure Computer Vision Image Analysis API now supports custom models. Use Image Analysis 4.0 to create custom image identifier models using the latest technology from Azure. To migrate a Custom Vision project to the new Image Analysis 4.0 system, see the Migration guide.
You can use Custom Vision through a client library SDK, REST API, or through the Custom Vision web portal. Follow a quickstart to get started.
This documentation contains the following types of articles:
The quickstarts are step-by-step instructions that let you make calls to the service and get results in a short period of time.
The how-to guides contain instructions for using the service in more specific or customized ways.
The tutorials are longer guides that show you how to use this service as a component in broader business solutions.
For a more structured approach, follow a Training module for Custom Vision:
How it works
The Custom Vision service uses a machine learning algorithm to analyze images. You submit sets of images that have and don’t have the visual characteristics you’re looking for. Then you label the images with your own custom labels (tags) at the time of submission. The algorithm trains to this data and calculates its own accuracy by testing itself on the same images. Once you’ve trained your model, you can test, retrain, and eventually use it in your image recognition app to classify images or detect objects. You can also export the model for offline use.
Classification and object detection
Custom Vision functionality can be divided into two features. Image classification applies one or more labels to an entire image. Object detection is similar, but it returns the coordinates in the image where the applied label(s) can be found.
Optimization
The Custom Vision service is optimized to quickly recognize major differences between images, so you can start prototyping your model with a small amount of data. 50 images per label are generally a good start. However, the service is not optimal for detecting subtle differences in images (for example, detecting minor cracks or dents in quality assurance scenarios).
Additionally, you can choose from several variations of the Custom Vision algorithm that are optimized for images with certain subject material—for example, landmarks or retail items. See Select a domain for more information.
B. How is it used at Seagen
As a biopharma research company that uses Microsoft Azure, you can use Azure Cognitive Services: Custom Vision to build and train custom image recognition models that can recognize and classify images relevant to your research. Here are some ways you can use Azure Cognitive Services: Custom Vision:
Identification of microscopic images: You can use Azure Cognitive Services: Custom Vision to train a custom image recognition model that can identify microscopic images of cells, tissues, or other biological structures relevant to your research.
Identification of molecular structures: You can use Azure Cognitive Services: Custom Vision to train a custom image recognition model that can identify molecular structures, such as proteins, enzymes, and DNA, relevant to your research.
Identification of disease markers: You can use Azure Cognitive Services: Custom Vision to train a custom image recognition model that can identify disease markers, such as tumors or abnormalities, in medical images relevant to your research.
Identification of drug interactions: You can use Azure Cognitive Services: Custom Vision to train a custom image recognition model that can identify drug interactions and their effects on biological structures relevant to your research.
Customization: Azure Cognitive Services: Custom Vision allows you to customize the image recognition models and criteria to meet your specific needs, ensuring that the service accurately recognizes and classifies images based on your unique requirements.
Overall, Azure Cognitive Services: Custom Vision provides a powerful and flexible tool for building and training custom image recognition models that can recognize and classify images relevant to your research. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately analyze and extract information from your images, and gain insights into the biological structures, molecular interactions, and disease markers relevant to your research.
C. Features
Azure Cognitive Services: Custom Vision is a machine learning-based service that enables you to build and train custom image recognition models that can recognize and classify images relevant to your research or business. Here are some of the key features of Azure Cognitive Services: Custom Vision:
Customization: Azure Cognitive Services: Custom Vision allows you to customize the image recognition models and criteria to meet your specific needs, ensuring that the service accurately recognizes and classifies images based on your unique requirements.
Image recognition: Azure Cognitive Services: Custom Vision allows you to build and train custom image recognition models that can recognize and classify images relevant to your research or business. This can include anything from identifying microscopic images to recognizing molecular structures and disease markers.
Training and testing: Azure Cognitive Services: Custom Vision provides a user-friendly interface that makes it easy to train and test custom image recognition models, allowing you to quickly iterate and refine your models as needed.
Integration: Azure Cognitive Services: Custom Vision can be easily integrated with other Azure services, such as Azure Blob Storage and Azure Functions, as well as third-party services and tools.
Scalability: Azure Cognitive Services: Custom Vision is highly scalable, allowing you to process large volumes of images and data quickly and efficiently.
Pre-built models: Azure Cognitive Services: Custom Vision also includes a number of pre-built models that can be used to recognize common objects, such as people, animals, and vehicles.
Custom domains: Azure Cognitive Services: Custom Vision allows you to create custom domains, which can be used to create more specific image recognition models that are tailored to your unique business or research needs.
Overall, Azure Cognitive Services: Custom Vision provides a powerful and flexible tool for building and training custom image recognition models that can recognize and classify images relevant to your research or business. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately analyze and extract information from your images, and gain insights into the biological structures, molecular interactions, and disease markers relevant to your research.
D. Where Implemented
E. How it is tested
Testing Azure Cognitive Services: Custom Vision involves verifying that the service is properly configured, that the image recognition models are accurately trained, and that the service can accurately recognize and classify images relevant to your research or business. Here are some steps you can take to test Azure Cognitive Services: Custom Vision:
Verify configuration: Verify that Azure Cognitive Services: Custom Vision is properly configured and integrated with your Azure account and resources.
Test image recognition: Test Azure Cognitive Services: Custom Vision by submitting sample images that contain known objects or structures relevant to your research or business, and verifying that the service accurately recognizes and classifies the images.
Test customization: Test the customization capabilities of Azure Cognitive Services: Custom Vision by configuring the image recognition models and criteria to meet your specific needs, and verifying that the service accurately recognizes and classifies images based on your unique requirements.
Test training and testing: Test the training and testing capabilities of Azure Cognitive Services: Custom Vision by iteratively training and testing custom image recognition models, and verifying that the service accurately recognizes and classifies images based on the training data.
Test integration: Test the integration capabilities of Azure Cognitive Services: Custom Vision by integrating it with other Azure services or third-party tools, and verifying that the service works seamlessly with your existing workflows and platforms.
Test scalability: Test the scalability of Azure Cognitive Services: Custom Vision by submitting large volumes of images and verifying that the service can process them quickly and efficiently.
Test documentation: Test the documentation of Azure Cognitive Services: Custom Vision by verifying that it is up-to-date, accurate, and comprehensive.
Overall, testing Azure Cognitive Services: Custom Vision involves verifying that the service is properly configured and functioning as expected, testing image recognition, customization, training and testing, integration, scalability, and documentation. By testing Azure Cognitive Services: Custom Vision, you can ensure that you are effectively using the service to recognize and classify images relevant to your research or business, and that you are benefiting from the accuracy, flexibility, and scalability it provides.
F. 2023 Roadmap
????
G. 2024 Roadmap
????
H. Known Issues
As with any software or service, there may be known issues or limitations that users should be aware of when using Azure Cognitive Services: Custom Vision. Here are some of the known issues for Azure Cognitive Services: Custom Vision:
Limited accuracy: While Azure Cognitive Services: Custom Vision provides accurate results in many cases, it may not always accurately recognize and classify images, particularly in cases where the images are highly complex or contain subtle nuances.
Limited customization: Azure Cognitive Services: Custom Vision has limited customization options, which can limit the ability of users to configure the service to their specific needs.
Limited training data: Azure Cognitive Services: Custom Vision requires a large amount of high-quality training data to accurately recognize and classify images, which can be difficult to obtain and may require significant time and resources.
Limited integration: Azure Cognitive Services: Custom Vision has limited integration with third-party tools and services, which can limit the ability of users to incorporate it into their existing workflows.
Cost: Azure Cognitive Services: Custom Vision can be expensive for users with limited budgets, particularly if they use it frequently or for large volumes of data.
Limited language support: Azure Cognitive Services: Custom Vision has limited language support, which can limit its usefulness for users who work with images in languages other than English.
Overall, while Azure Cognitive Services: Custom Vision offers a powerful and flexible tool for building and training custom image recognition models that can recognize and classify images relevant to your research or business, users must be aware of these known issues and take steps to mitigate their impact. This may include carefully configuring the service to meet the specific needs of their data, carefully monitoring the cost and accuracy of the service to ensure that it is a good fit for their budget and data requirements, and carefully integrating the service into their existing workflows to ensure that it is effectively utilized. By taking these steps, users can ensure that they are effectively using Azure Cognitive Services: Custom Vision to recognize and classify images relevant to their research or business, and that they are benefiting from the accuracy, flexibility, and scalability it provides.
[x] Reviewed by Enterprise Architecture
[x] Reviewed by Application Development
[x] Reviewed by Data Architecture