Azure Snowflake
Author: Ronald Fung
Creation Date: 1 June 2023
Next Modified Date: 1 June 2024
A. Introduction
This article outlines how to register Snowflake, and how to authenticate and interact with Snowflake in Microsoft Purview. For more information about Microsoft Purview, read the introductory article.
Supported capabilities
Metadata Extraction |
Full Scan |
Incremental Scan |
Scoped Scan |
Classification |
Labeling |
Access Policy |
Lineage |
Data Sharing |
|---|---|---|---|---|---|---|---|---|
No |
No |
No |
No |
When scanning Snowflake source, Microsoft Purview supports:
Extracting technical metadata including:
Server
Databases
Schemas
Tables including the columns, foreign keys and unique constraints
Views including the columns
Stored procedures including the parameter dataset and result set
Functions including the parameter dataset
Pipes
Stages
Streams including the columns
Tasks
Sequences
Fetching static lineage on assets relationships among tables, views, and streams.
When setting up scan, you can choose to scan one or more Snowflake database(s) entirely, or further scope the scan to a subset of schemas matching the given name(s) or name pattern(s).
B. How is it used at Seagen
As a biopharma research company using Microsoft Azure, you can use Snowflake to store, manage, and analyze your data in the cloud. Here are some ways you can use Snowflake:
Data warehousing: Snowflake can be used as a cloud-based data warehouse, allowing you to store and manage large volumes of data related to your biopharma research efforts.
Data integration: Snowflake can be integrated with other Azure services, allowing you to move data between systems and perform data integration tasks.
Data analysis: Snowflake provides powerful data analysis tools, allowing you to explore and analyze your research data to uncover patterns and insights.
Data sharing: Snowflake allows you to securely share data with other researchers and stakeholders, helping to facilitate collaboration and knowledge sharing.
Security: Snowflake provides advanced security features, helping to protect the privacy and security of your research data.
Scalability: Snowflake is highly scalable, allowing you to easily scale up or down as your data needs change.
Performance: Snowflake provides high performance data processing capabilities, allowing you to quickly and efficiently analyze your research data.
Overall, by leveraging Snowflake, you can effectively store, manage, and analyze your research data in the cloud, and collaborate with other researchers and stakeholders to achieve your biopharma research goals. By using Snowflake for data warehousing, data integration, data analysis, data sharing, security, scalability, and performance, you can effectively manage your research data and make data-driven decisions that support your biopharma research efforts.
C. Features
Azure Snowflake is a cloud-based data warehousing and analytics platform that provides powerful data warehousing, data integration, and data analysis capabilities. Here are some of the key features of Azure Snowflake:
Data warehousing: Azure Snowflake provides a cloud-based data warehousing solution that allows you to store and manage large volumes of data related to your biopharma research efforts.
Data integration: Azure Snowflake can be integrated with other Azure services, allowing you to move data between systems and perform data integration tasks.
Data analysis: Azure Snowflake provides powerful data analysis tools, allowing you to explore and analyze your research data to uncover patterns and insights.
Data sharing: Azure Snowflake allows you to securely share data with other researchers and stakeholders, helping to facilitate collaboration and knowledge sharing.
Security: Azure Snowflake provides advanced security features, helping to protect the privacy and security of your research data.
Scalability: Azure Snowflake is highly scalable, allowing you to easily scale up or down as your data needs change.
Performance: Azure Snowflake provides high performance data processing capabilities, allowing you to quickly and efficiently analyze your research data.
Cost-effectiveness: Azure Snowflake provides a cost-effective solution for data warehousing and analytics, allowing you to pay only for the resources you use.
Availability: Azure Snowflake provides high availability and disaster recovery capabilities, ensuring that your research data is always available and protected.
Overall, by leveraging Azure Snowflake, you can effectively store, manage, and analyze your research data in the cloud, and collaborate with other researchers and stakeholders to achieve your biopharma research goals. By using Azure Snowflake for data warehousing, data integration, data analysis, data sharing, security, scalability, performance, cost-effectiveness, and availability, you can effectively manage your research data and make data-driven decisions that support your biopharma research efforts.
D. Where Implemented
E. How it is tested
Testing Azure Snowflake involves verifying that the service is properly configured and that it is effectively storing, managing, and analyzing your research data in the cloud. Here are some steps you can take to test Azure Snowflake:
Verify configuration: Verify that Azure Snowflake is properly configured and integrated with your Azure account and applications and websites.
Test data warehousing: Test Azure Snowflake by storing and managing large volumes of data related to your biopharma research efforts, ensuring that the service is effectively storing and managing your research data.
Test data integration: Test Azure Snowflake by integrating with other Azure services, allowing you to move data between systems and perform data integration tasks.
Test data analysis: Test Azure Snowflake by using its powerful data analysis tools to explore and analyze your research data, ensuring that the service is effectively analyzing your research data to uncover patterns and insights.
Test data sharing: Test Azure Snowflake by securely sharing data with other researchers and stakeholders, helping to facilitate collaboration and knowledge sharing.
Test security: Test Azure Snowflake by verifying that the service provides advanced security features, helping to protect the privacy and security of your research data.
Test scalability: Test Azure Snowflake by verifying that the service is highly scalable, allowing you to easily scale up or down as your data needs change.
Test performance: Test Azure Snowflake by verifying that the service provides high performance data processing capabilities, allowing you to quickly and efficiently analyze your research data.
Test cost-effectiveness: Test Azure Snowflake by verifying that the service provides a cost-effective solution for data warehousing and analytics, allowing you to pay only for the resources you use.
Test availability: Test Azure Snowflake by verifying that the service provides high availability and disaster recovery capabilities, ensuring that your research data is always available and protected.
Overall, testing Azure Snowflake involves verifying that the service is effectively storing, managing, and analyzing your research data in the cloud, and providing powerful data warehousing, data integration, and data analysis capabilities. By taking these steps, you can ensure that you are effectively using Azure Snowflake to manage your research data and make data-driven decisions that support your biopharma research efforts.
F. 2023 Roadmap
????
G. 2024 Roadmap
????
H. Known Issues
Like any software or service, there may be known issues or limitations with Azure Snowflake that users should be aware of. Here are some of the known issues with Azure Snowflake:
Cost: Azure Snowflake can be expensive, particularly for organizations that require advanced data warehousing and analytics capabilities or that require a high level of support and customization.
Complexity: Azure Snowflake can be complex to configure and use, particularly for organizations with limited experience in data warehousing and analytics workflows.
Integration: While Azure Snowflake can be integrated with other Azure services, there may be compatibility issues that need to be addressed.
Reliability: Azure Snowflake may experience occasional outages or service disruptions, which can impact the storage, management, and analysis of your research data.
Performance: Azure Snowflake may experience performance issues when working with large datasets or complex data warehousing and analytics workflows.
Security: While Azure Snowflake provides advanced security features, there may be security risks associated with storing and managing research data in the cloud.
Overall, while Azure Snowflake offers a powerful tool for storing, managing, and analyzing research data in the cloud, users must be aware of these known issues and take steps to mitigate their impact. This may include carefully managing costs to ensure that they stay within their budget, carefully configuring the service to meet the specific needs of their data warehousing and analytics workflows, and carefully monitoring data warehousing and analytics activity to ensure that the service is effectively storing, managing, and analyzing research data. By taking these steps, users can ensure that they are effectively using Azure Snowflake to manage their research data and make data-driven decisions that support their biopharma research efforts.
[x] Reviewed by Enterprise Architecture
[x] Reviewed by Application Development
[x] Reviewed by Data Architecture