Azure Cognitive Services: Computer Vision
Author: Ronald Fung
Creation Date: 31 May 2023
Next Modified Date: 31 May 2024
A. Introduction
Azure’s Computer Vision service gives you access to advanced algorithms that process images and return information based on the visual features you’re interested in.
Service |
Description |
|---|---|
The Optical Character Recognition (OCR) service extracts text from images. You can use the new Read API to extract printed and handwritten text from photos and documents. It uses deep-learning-based models and works with text on a variety of surfaces and backgrounds. These include business documents, invoices, receipts, posters, business cards, letters, and whiteboards. The OCR APIs support extracting printed text in several languages. Follow the OCR quickstart to get started. |
|
The Image Analysis service extracts many visual features from images, such as objects, faces, adult content, and auto-generated text descriptions. Follow the Image Analysis quickstart to get started. |
|
The Face service provides AI algorithms that detect, recognize, and analyze human faces in images. Facial recognition software is important in many different scenarios, such as identity verification, touchless access control, and face blurring for privacy. Follow the Face quickstart to get started. |
|
The Spatial Analysis service analyzes the presence and movement of people on a video feed and produces events that other systems can respond to. Install the Spatial Analysis container to get started. |
Computer Vision for digital asset management
Computer Vision can power many digital asset management (DAM) scenarios. DAM is the business process of organizing, storing, and retrieving rich media assets and managing digital rights and permissions. For example, a company may want to group and identify images based on visible logos, faces, objects, colors, and so on. Or, you might want to automatically generate captions for images and attach keywords so they’re searchable. For an all-in-one DAM solution using Cognitive Services, Azure Cognitive Search, and intelligent reporting, see the Knowledge Mining Solution Accelerator Guide on GitHub. For other DAM examples, see the Computer Vision Solution Templates repository.
B. How is it used at Seagen
As a biopharma research company that uses Microsoft Azure, you can use Azure Cognitive Services: Computer Vision to help analyze and extract information from images and videos. Here are some ways you can use Azure Cognitive Services: Computer Vision:
Analyze microscope images: You can use Azure Cognitive Services: Computer Vision to analyze microscope images of cells, tissues, and other biological samples. This can help you identify patterns, structures, and abnormalities that may be relevant to your research.
Analyze medical images: You can use Azure Cognitive Services: Computer Vision to analyze medical images, such as x-rays, MRIs, and CT scans. This can help you identify potential health issues, track disease progression, and evaluate treatment outcomes.
Analyze research images: You can use Azure Cognitive Services: Computer Vision to analyze research images, such as gel electrophoresis images, western blot images, and histology images. This can help you identify patterns, structures, and abnormalities that may be relevant to your research.
Extract text from images: You can use Azure Cognitive Services: Computer Vision to extract text from images, such as scanned documents, receipts, and labels. This can help you automate data entry and reduce errors.
Analyze security footage: You can use Azure Cognitive Services: Computer Vision to analyze security footage, such as surveillance videos and images. This can help you identify potential security threats, track suspicious behavior, and improve safety.
Overall, Azure Cognitive Services: Computer Vision provides a powerful and flexible tool for analyzing images and videos. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately analyze images and videos, extract information, and make data-driven decisions to improve your business outcomes.
C. Features
Azure Cognitive Services: Computer Vision is a machine learning-based service that enables you to analyze and extract information from images and videos. Here are some of the key features of Azure Cognitive Services: Computer Vision:
Image analysis: Azure Cognitive Services: Computer Vision allows you to analyze images to extract information such as objects, text, and faces. You can also analyze images to detect and classify visual features such as colors, tags, and landmarks.
Video analysis: Azure Cognitive Services: Computer Vision allows you to analyze videos to extract information such as motion, faces, and speech. You can also analyze videos to detect and classify visual features such as frames, shots, and keyframes.
Optical character recognition (OCR): Azure Cognitive Services: Computer Vision includes OCR capabilities that allow you to extract text from images and videos, such as scanned documents, labels, and receipts.
Custom vision: Azure Cognitive Services: Computer Vision allows you to train your own custom vision models using your own images and labels. You can use these models to classify images and detect specific objects and features.
Emotion recognition: Azure Cognitive Services: Computer Vision includes emotion recognition capabilities that allow you to detect emotions such as happiness, sadness, and anger in faces.
Face detection and recognition: Azure Cognitive Services: Computer Vision includes face detection and recognition capabilities that allow you to detect and identify faces in images and videos.
Content moderation: Azure Cognitive Services: Computer Vision includes content moderation capabilities that allow you to detect and filter inappropriate or offensive content in images and videos.
Overall, Azure Cognitive Services: Computer Vision provides a powerful and flexible tool for analyzing images and videos. By leveraging the machine learning and AI capabilities of the service, you can quickly and accurately analyze images and videos, extract information, and make data-driven decisions to improve your business outcomes.
D. Where Implemented
E. How it is tested
Testing Azure Cognitive Services: Computer Vision involves verifying that the service is properly configured and that it can accurately analyze and extract information from images and videos. Here are some steps you can take to test Azure Cognitive Services: Computer Vision:
Verify configuration: Verify that Azure Cognitive Services: Computer Vision is properly configured and integrated with your Azure account and resources.
Test image analysis: Test Azure Cognitive Services: Computer Vision by uploading sample images and verifying that it can accurately detect and classify visual features such as objects, text, and faces.
Test video analysis: Test Azure Cognitive Services: Computer Vision by uploading sample videos and verifying that it can accurately detect and classify visual features such as motion, faces, and speech.
Test OCR: Test the OCR capabilities of Azure Cognitive Services: Computer Vision by uploading sample images and verifying that it can accurately extract text from the images.
Test custom vision: Test the custom vision capabilities of Azure Cognitive Services: Computer Vision by training your own models using your own images and labels, and verifying that the service can accurately classify images and detect specific objects and features.
Test emotion recognition and face detection: Test the emotion recognition and face detection capabilities of Azure Cognitive Services: Computer Vision by uploading sample images and verifying that it can accurately detect emotions and identify faces.
Test content moderation: Test the content moderation capabilities of Azure Cognitive Services: Computer Vision by uploading sample images and videos and verifying that it can accurately detect and filter inappropriate or offensive content.
Test documentation: Test the documentation of Azure Cognitive Services: Computer Vision by verifying that it is up-to-date, accurate, and comprehensive.
Overall, testing Azure Cognitive Services: Computer Vision involves verifying that the service is properly configured and functioning as expected, testing image and video analysis, OCR, custom vision, emotion recognition, face detection, content moderation, and documentation. By testing Azure Cognitive Services: Computer Vision, you can ensure that you are effectively using the service to analyze and extract information from images and videos, and that you are benefiting from the accuracy, flexibility, and scalability it provides.
F. 2023 Roadmap
????
G. 2024 Roadmap
????
H. Known Issues
As with any software or service, there may be known issues or limitations that users should be aware of when using Azure Cognitive Services: Computer Vision. Here are some of the known issues for Azure Cognitive Services: Computer Vision:
Limited accuracy: While Azure Cognitive Services: Computer Vision provides accurate results in many cases, it may not always accurately analyze images and videos, particularly in cases where the images or videos are low-quality or contain unusual or complex features.
Limited customization: Azure Cognitive Services: Computer Vision has limited customization options, which can limit the ability of users to configure the service to their specific needs.
Limited language support: Azure Cognitive Services: Computer Vision has limited language support, which can limit its usefulness for users who work with images and videos in languages other than English.
Sensitivity: The sensitivity of image and video analysis in Azure Cognitive Services: Computer Vision can be difficult to configure, which can result in false positives or false negatives.
Cost: Azure Cognitive Services: Computer Vision can be expensive for users with limited budgets, particularly if they use it frequently or for large volumes of data.
Limited integration: Azure Cognitive Services: Computer Vision has limited integration with third-party tools and services, which can limit the ability of users to incorporate it into their existing workflows.
Overall, while Azure Cognitive Services: Computer Vision offers a powerful and flexible tool for analyzing images and videos, users must be aware of these known issues and take steps to mitigate their impact. This may include carefully preprocessing data to ensure that it can be properly analyzed, carefully configuring the service to meet the specific needs of their data, carefully monitoring the cost and sensitivity of the service to ensure that it is a good fit for their budget and data requirements, and carefully integrating the service into their existing workflows to ensure that it is effectively utilized. By taking these steps, users can ensure that they are effectively using Azure Cognitive Services: Computer Vision to analyze and extract information from their images and videos and that they are benefiting from the accuracy, flexibility, and scalability it provides.
[x] Reviewed by Enterprise Architecture
[x] Reviewed by Application Development
[x] Reviewed by Data Architecture