Azure Cost Management and Billing

Author: Ronald Fung
Creation Date: May 12, 2023
Next Modified Date: May 12, 2024

A. Introduction

Microsoft Cost Management is a suite of tools that help organizations monitor, allocate, and optimize the cost of their Microsoft Cloud workloads. Cost Management is available to anyone with access to a billing or resource management scope. The availability includes anyone from the cloud finance team with access to the billing account. And, to DevOps teams managing resources in subscriptions and resource groups.

Billing is where you can manage your accounts, invoices, and payments. Billing is available to anyone with access to a billing account or other billing scope, like billing profiles and invoice sections. The cloud finance team and organizational leaders are typically included.

Together, Cost Management and Billing are your gateway to the Microsoft Commerce system that’s available to everyone throughout the journey. From initial sign-up and billing account management, to the purchase and management of Microsoft and third-party Marketplace offers, to financial operations (FinOps) tools.

A few examples of what you can do in Cost Management and Billing include:

Report on and analyze costs in the Azure portal, Microsoft 365 admin center, or externally by exporting data.
Monitor costs proactively with budget, anomaly, and scheduled alerts.
Split shared costs with cost allocation rules.
Create and organize subscriptions to customize invoices.
Configure payment options and pay invoices.
Manage your billing information, such as legal entity, tax information, and agreements.

B. How is it used at Seagen

As a biopharma research company using Microsoft Azure, you can use Azure Cognitive cost management and billing features to monitor and optimize your cloud spending. Here are some ways you can use Azure Cognitive cost management and billing:

Monitor and track cloud spending: Azure Cognitive cost management and billing can be used to monitor and track your cloud spending across various Azure services. You can use the Azure portal or APIs to view detailed reports and analysis of your usage and spending.
Set budgets and alerts: Azure Cognitive cost management and billing allows you to set budgets and alerts to help you stay within your spending limits. You can receive alerts when your spending exceeds a certain threshold or when you are close to reaching your budget.
Optimize cloud spending: Azure Cognitive cost management and billing can help you optimize your cloud spending by identifying areas where you can reduce costs. You can use the Azure Advisor to get recommendations for optimizing your Azure resources and services.
Allocate costs and chargebacks: Azure Cognitive cost management and billing allows you to allocate costs and chargebacks to different departments or projects. You can use tags and resource groups to categorize your spending and allocate costs accordingly.
Integration with other systems: Azure Cognitive cost management and billing can be integrated with other systems and applications used in your research process. You can use Azure Functions or Azure Logic Apps to trigger actions or exchange data between different systems, such as sending notifications when spending exceeds a certain threshold.

C. Features

How charges are processed

To understand how Cost Management and Billing works, you should first understand the Commerce system. At its core, Microsoft Commerce is a data pipeline that underpins all Microsoft commercial transactions, whether consumer or commercial. There are many inputs and connections to the pipeline. It includes the sign-up and Marketplace purchase experiences. However, we’ll focus on the pieces that make up your cloud billing account and how charges are processed within the system.

Commerce data pipeline

In the left side of the diagram, your Azure, Microsoft 365, Dynamics 365, and Power Platform services are all pushing data into the Commerce data pipeline. Each service publishes data on a different cadence. In general, if data for one service is slower than another, it’s due to how frequently those services are publishing their usage and charges.

As the data makes its way through the pipeline, the rating system applies discounts based on your specific price sheet and generates rated usage, which includes price and quantity for each cost record. It’s the basis for what you see in Cost Management, but we’ll cover that later. At the end of the month, credits are applied and the invoice is published. The process starts 72 hours after your billing period ends, which is usually the last day of the calendar month for most accounts. For example, if your billing period ends on March 31, charges will be finalized on April 4 at midnight.

Important - Credits are applied like a gift card or other payment instrument before the invoice is generated. While credit status is tracked as new charges flow into the data pipeline, credits aren’t explicitly applied to these charges until the end of the month.

Everything up to this point makes up the billing process. It’s where charges are finalized, discounts are applied, and invoices are published. Billing account and billing profile owners may be familiar with this process as part of the Billing experience within the Azure portal or Microsoft 365 admin center. The Billing experience allows you to review credits, manage your billing address and payment methods, pay invoices, and more – everything related to managing your billing relationship with Microsoft.

After discounts are applied, cost details then flow into Cost Management, where:

The anomaly detection model identifies anomalies daily based on normalized usage (not rated usage).
The cost allocation engine applies tag inheritance and splits shared costs.
AWS cost and usage reports are pulled based on any connectors for AWS you may have configured.
Azure Advisor cost recommendations are pulled in to enable cost savings insights for subscriptions and resource groups.
Cost alerts are sent out for budgets, anomalies, scheduled alerts, and more based on the configured settings. Lastly, cost details are made available from cost analysis in the Azure portal and published to your storage account via scheduled exports.

How Cost Management and Billing relate

Cost Management is a set of FinOps tools that enable you to analyze, manage, and optimize your costs.

Billing provides all the tools you need to manage your billing account and pay invoices.

Cost Management is available from within the Billing experience. It’s also available from every subscription, resource group, and management group in the Azure portal. The availability is to ensure everyone has full visibility into the costs they’re responsible for. And, so they can optimize their workloads to maximize efficiency. Cost Management is also available independently to streamline the process for managing cost across multiple billing accounts, subscriptions, resource groups, and management groups.

billing organization relates to Cost Management

What data is included in Cost Management and Billing?

Within the Billing experience, you can manage all the products, subscriptions, and recurring purchases you use; review your credits and commitments; and view and pay your invoices. Invoices are available online or as PDFs and include all billed charges and any applicable taxes. Credits are applied to the total invoice amount when invoices are generated. This invoicing process happens in parallel to Cost Management data processing, which means Cost Management doesn’t include credits, taxes, and some purchases, like support charges in non-Microsoft Customer Agreement (MCA) accounts.

The classic Cloud Solution Provider (CSP) and sponsorship subscriptions aren’t supported in Cost Management. These subscriptions will be supported after they transition to MCA.

For more information about supported offers, what data is included, or how data is refreshed and retained in Cost Management, see Understand Cost Management data.

Manage your billing account and invoices

Microsoft has several types of billing accounts. Each type has a slightly different experience to support the unique aspects of the billing account. To learn more, see Billing accounts and scopes.

You use billing account management tasks to:

View invoices and make payments.
Configure your billing address and PO numbers.
Create and organize subscriptions into departments or billing profiles.
Renew or cancel products you’ve purchased.
Enable access to Cost Management, Reservations, and Marketplace offers.
View agreements, credits, and commitments.

Management for classic Cloud Solution Provider (CSP) and classic sponsorship subscriptions isn’t available in Billing or Cost Management experiences because they’re billed differently.

Report on and analyze costs

Cost Management and Billing include several tools to help you understand, report on, and analyze your invoiced Microsoft Cloud and AWS costs.

Cost analysis is a tool for ad-hoc cost exploration. Get quick answers with lightweight insights and analytics. Power BI is an advanced solution to build more extensive dashboards and complex reports or combine costs with other data. Power BI is available for billing accounts and billing profiles.
Exports and the Cost Details API enable you to integrate cost details into external systems or business processes.
The Credits page shows your available credit or prepaid commitment balance. They aren’t included in cost analysis.
The Invoices page provides a list of all previously invoiced charges and their payment status for your billing account.
Connectors for AWS enable you to ingest your AWS cost details into Azure to facilitate managing Azure and AWS costs together. After configured, the connector also enables other capabilities, like budget and scheduled alerts.

For more information, see Get started with Cost Management and Billing reporting.

Organize and allocate costs

Organizing and allocating costs are critical to ensuring invoices are routed to the correct business units and can be further split for internal billing, also known as chargeback. Cost Management and Billing offer the following options to organize resources and subscriptions:

MCA billing profiles and invoice sections are used to group subscriptions into invoices. Each billing profile represents a separate invoice that can be billed to a different business unit and each invoice section is segmented separately within those invoices. You can also view costs by billing profile or invoice section in costs analysis.
EA departments and enrollment accounts are conceptually similar to invoice sections, as groups of subscriptions, but they aren’t represented within the invoice PDF. They’re included within the cost details backing each invoice, however. You can also view costs by department or enrollment account in costs analysis.
Management groups also allow grouping subscriptions together, but offer a few key differences:
- Management group access is inherited down to the subscriptions and resources.
- Management groups can be layered into multiple levels and subscriptions can be placed at any level.
- Management groups aren’t included in cost details.
- All historical costs are returned for management groups based on the subscriptions currently within that hierarchy. When a subscription moves, all historical cost moves.
- Management groups are supported by Azure Policy and can have rules assigned to automate compliance reporting for your cost governance strategy.
Subscriptions and resource groups are the lowest level at which you can organize your cloud solutions. At Microsoft, every product – sometimes even limited to a single region – is managed within its own subscription. It simplifies cost governance but requires more overhead for subscription management. Most organizations use subscriptions for business units and separating dev/test from production or other environments, then use resource groups for the products. It complicates cost management because resource group owners don’t have a way to manage cost across resource groups. On the other hand, it’s a straightforward way to understand who’s responsible for most resource-based charges. Keep in mind that not all charges come from resources and some don’t have resource groups or subscriptions associated with them. It also changes as you move to MCA billing accounts.
Resource tags are the only way to add your own business context to cost details and are perhaps the most flexible way to map resources to applications, business units, environments, owners, etc. For more information, see How tags are used in cost and usage data for limitations and important considerations.

In addition to organizing resources and subscriptions using the subscription hierarchy and metadata (tags), Cost Management also offers the ability to move or split shared costs via cost allocation rules. Cost allocation doesn’t change the invoice. Cost allocation simply moves charges from one subscription, resource group, or tag to another subscription, resource group, or tag. The goal of cost allocation is to split and move shared costs to reduce overhead. And, to more accurately report on where charges are ultimately coming from (albeit indirectly), which should drive more complete accountability. For more information, see Allocate Azure costs.

How you organize and allocate costs plays a huge role in how people within your organization can manage and optimize costs. Be sure to plan ahead and revisit your allocation strategy yearly.

Monitor costs with alerts

Cost Management and Billing offer many different types of emails and alerts to keep you informed and help you proactively manage your account and incurred costs.

Budget alerts notify recipients when cost exceeds a predefined cost or forecast amount. Budgets can be visualized in cost analysis and are available on every scope supported by Cost Management. Subscription and resource group budgets can also be configured to notify an action group to take automated actions to reduce or even stop further charges.
Anomaly alerts notify recipients when an unexpected change in daily usage has been detected. It can be a spike or a dip. Anomaly detection is only available for subscriptions and can be viewed within the cost analysis preview. Anomaly alerts can be configured from the cost alerts page.
Scheduled alerts notify recipients about the latest costs on a daily, weekly, or monthly schedule based on a saved cost view. Alert emails include a visual chart representation of the view and can optionally include a CSV file. Views are configured in cost analysis, but recipients don’t require access to cost in order to view the email, chart, or linked CSV.
EA commitment balance alerts are automatically sent to any notification contacts configured on the EA billing account when the balance is 90% or 100% used.
Invoice alerts can be configured for MCA billing profiles and Microsoft Online Services Program (MOSP) subscriptions. For details, see View and download your Azure invoice.

For for information, see Monitor usage and spending with cost alerts.

Optimize costs

Microsoft offers a wide range of tools for optimizing your costs. Some of these tools are available outside the Cost Management and Billing experience, but are included for completeness.

There are many free services available in Azure. Be sure to pay close attention to the constraints. Different services are free indefinitely, for 12 months, or 30 days. Some are free up to a specific amount of usage and some may have dependencies on other services that aren’t free.
The Azure pricing calculator is the best place to start when planning a new deployment. You can tweak many aspects of the deployment to understand how you’ll be charged for that service and identify which SKUs/options will keep you within your desired price range. For more information about pricing for each of the services you use, see pricing details.
Azure Advisor cost recommendations should be your first stop when interested in optimizing existing resources. Advisor recommendations are updated daily and are based on your usage patterns. Advisor is available for subscriptions and resource groups. Management group users can also see recommendations but will need to select the desired subscriptions. Billing users can only see recommendations for subscriptions they have resource access to.
Azure savings plans save you money when you have consistent usage of Azure compute resources. A savings plan can significantly reduce your resource costs by up to 65% from pay-as-you-go prices.
Azure reservations help you save up to 72% compared to pay-as-you-go rates by pre-committing to specific usage amounts for a set time duration.
Azure Hybrid Benefit helps you significantly reduce costs by using on-premises Windows Server and SQL Server licenses or RedHat and SUSE Linux subscriptions on Azure.

For other options, see Azure benefits and incentives.

D. Where implemented

LeanIX

E. How it is tested

Testing Azure Cost Management and Billing involves ensuring that the cost management and billing service is functioning correctly, securely, and meeting the needs of all stakeholders involved in the project. Here are some steps to follow to test Azure Cost Management and Billing:

Define the scope and requirements: Define the scope of the project and the requirements of all stakeholders involved in the project. This will help ensure that Azure Cost Management and Billing is designed to meet the needs of all stakeholders.
Develop test cases: Develop test cases that cover all aspects of Azure Cost Management and Billing functionality, including cost tracking, forecasting, and analysis. The test cases should be designed to meet the needs of the organization, including scalability and resilience.
Conduct unit testing: Test the individual components of Azure Cost Management and Billing to ensure that they are functioning correctly. This may involve using tools like PowerShell or Azure CLI for automated testing.
Conduct integration testing: Test Azure Cost Management and Billing in an integrated environment to ensure that it works correctly with other systems and applications. This may involve testing Azure Cost Management and Billing with different operating systems, browsers, and devices.
Conduct user acceptance testing: Test Azure Cost Management and Billing with end-users to ensure that it meets their needs and is easy to use. This may involve conducting surveys, interviews, or focus groups to gather feedback from users.
Automate testing: Automate testing of Azure Cost Management and Billing to ensure that it is functioning correctly and meeting the needs of all stakeholders. This may involve using tools like Azure DevOps to set up automated testing pipelines.
Monitor performance: Monitor the performance of Azure Cost Management and Billing in production to ensure that it is meeting the needs of all stakeholders. This may involve setting up monitoring tools, such as Azure Monitor, to track usage and identify performance issues.
Address issues: Address any issues that are identified during testing and make necessary changes to ensure that Azure Cost Management and Billing is functioning correctly and meeting the needs of all stakeholders.

By following these steps, you can ensure that Azure Cost Management and Billing is tested thoroughly and meets the needs of all stakeholders involved in the project. This can help improve the quality of Azure Cost Management and Billing and ensure that it functions correctly in a production environment.

F. 2023 Roadmap

May 2023

Support for up to 30 shards for clustered Azure Cache for Redis instances

Azure Cache for Redis now supports clustered caches with upto 30 shards which means your applications can store more data and scale better with your workloads.

For more information, see Configure clustering for Azure Cache for Redis instance.

March 2023

In-place scale up and scale out for the Enterprise tiers (preview)

The Enterprise and Enterprise Flash tiers now support the ability to scale cache instances up and out without requiring downtime or data loss. Scale up and scale out actions can both occur in the same operation.

For more information, see Scale an Azure Cache for Redis instance

Support for RedisJSON in active geo-replicated caches (preview)

Cache instances using active geo-replication now support the RedisJSON module.

For more information, see Configure active geo-replication.

Flush operation for active geo-replicated caches (preview)

Caches using active geo-replication now include a built-in flush operation that can be initiated at the control plane level. Use the flush operation with your cache instead of the FLUSH ALL and FLUSH DB operations, which are blocked by design for active geo-replicated caches.

For more information, see Flush operation

Customer managed key (CMK) disk encryption (preview)

Redis data that is saved on disk can now be encrypted using customer managed keys (CMK) in the Enterprise and Enterprise Flash tiers. Using CMK adds another layer of control to the default disk encryption.

For more information, see Enable disk encryption

Connection event audit logs (preview)

Enterprise and Enterprise Flash tier caches can now log all connection, disconnection, and authentication events through diagnostic settings. Logging this information helps in security audits. You can also monitor who has access to your cache resource.

For more information, see Enabling connection audit logs

G. 2024 Roadmap

????

H. Known Issues

There are several known issues that can impact Azure Cost Management and Billing. Here are some of the most common issues to be aware of:

Integration issues: Integration issues can arise when integrating Azure Cost Management and Billing with other systems and applications. It is important to ensure that Azure Cost Management and Billing is designed to work seamlessly with other systems and applications to avoid integration issues.
Data accuracy issues: Data accuracy issues can arise when the billing data is not updated in real-time. It is important to ensure that billing data is updated in real-time to avoid data accuracy issues.
Data processing issues: Data processing issues can arise when processing large amounts of billing data. It is important to ensure that Azure Cost Management and Billing can handle large amounts of data and that it can process data quickly.
Security issues: Security is a critical concern when it comes to Azure Cost Management and Billing. It is important to ensure that Azure Cost Management and Billing is secured and that access to the solution is restricted to authorized personnel.
Performance issues: If the billing service is not properly sized, it can impact performance and availability, causing issues with the speed and reliability of Azure Cost Management and Billing.
Compatibility issues: Azure Cost Management and Billing may not be compatible with all platforms, devices, or languages. It is important to ensure that Azure Cost Management and Billing is compatible with the organization’s existing infrastructure before implementation.
Testing issues: Testing issues can arise when testing Azure Cost Management and Billing. It is important to ensure that testing is carried out thoroughly and that all aspects of Azure Cost Management and Billing functionality are tested.

Overall, Azure Cost Management and Billing requires careful planning and management to ensure that it is functioning correctly and meeting the needs of all stakeholders involved in the project. By being aware of these known issues and taking steps to address them, you can improve the quality of Azure Cost Management and Billing and ensure the success of your project.

[x] Reviewed by Enterprise Architecture

[x] Reviewed by Application Development

[x] Reviewed by Data Architecture