01 september 2022
The “whales” on which clouds stand. Let's talk about data centers on the example of Microsoft Azure
Cloud technologies became a big and important part of our live and in most cases, modern IT engineers can’t imagine their work without interacting with cloud services. But, clouds don't work by themselves somewhere in the sky. Somewhere on our planet, there are data centers with a lot of servers that host all SaaS, PaaS and IaaS services we use. In this article, I will talk a little about data centers - “whales” on which clouds stand.
I would like to start with a couple of words about what clouds are.
The National Institute of Standards and Technology (NIST) says that a real cloud must meet the following criteria:
- On-demand self-service. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.
- Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations).
- Resource pooling. The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand.
- Rapid elasticity. Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear unlimited and can be appropriated in any quantity at any time.
- Measured service. Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.
So the Cloud is a certain amount of computing resources shared between all users, that have universal, platform-independent, network access, have the possibility of self-management for users, and have the function of billing only for consumed resources.
By computing resources, we mean the CPU, GPU, RAM, etc., which physically must exist somewhere. And they exist in data centers, that have everything to allow the clouds to meet the criteria above.
Let's look at the map of Azure data centers.
You can view this map yourself on a very cool website about Azure infrastructure: https://infrastructuremap.microsoft.com/
To understand what is depicted on this map, you need to know several definitions that Microsoft uses:
As you can see, data centers are grouped into regions, regions into geographies and all of this is connected by a global network
Within a region, there are Azure Availability Zones, consisting of one or more data centers, that allow you to distribute your resources in such a way that they will not be affected by problems with a specific data center.
So, in the picture above, blue color indicates existing regions, gray - announced ones, and dashed lines - networks between regions.
Azure, with 39 existing regions, has at least 78 data centers (assuming that one region has two Availability Zones). But I think that in reality, their number is bigger than one hundred, and maybe even two hundred. It is very mpressive.
But what is this Data Center?
If we believe this tweet, then in 2019, the Azure region in Dublin looked like this.
To understand what is depicted on this map, you need to know several definitions that Microsoft uses:
- Azure datacenters are unique physical buildings—located all over the globe—that house a group of networked computer servers.
- An Azure region is a set of datacenters, deployed within a latency-defined perimeter and connected through a dedicated regional low-latency network.
- An Azure geography is a discrete market, typically containing at least one or more regions, that preserves data residency and compliance boundaries
- Azure Availability Zones are unique physical locations within an Azure region and offer high availability to protect your applications and data from datacenter failures. Each zone is made up of one or more datacenters equipped with independent power, cooling, and networking.
- The Azure global network refers to all of the components in networking and is comprised of the Microsoft global wide-area network (WAN), points of presence (PoPs), fiber, and others.
As you can see, data centers are grouped into regions, regions into geographies and all of this is connected by a global network
Within a region, there are Azure Availability Zones, consisting of one or more data centers, that allow you to distribute your resources in such a way that they will not be affected by problems with a specific data center.
So, in the picture above, blue color indicates existing regions, gray - announced ones, and dashed lines - networks between regions.
Azure, with 39 existing regions, has at least 78 data centers (assuming that one region has two Availability Zones). But I think that in reality, their number is bigger than one hundred, and maybe even two hundred. It is very mpressive.
But what is this Data Center?
If we believe this tweet, then in 2019, the Azure region in Dublin looked like this.
From the picture, we can see very clearly that the data centers themselves consist of identical modules, combined into a single unit. Why? Because it is easier and cheaper to assemble something large from small identical parts.
I do not know exactly what is inside these modules, but logic suggests that all the communications must be laid out during production: electricity, network, cooling, and spatial frames for servers. Most likely, the servers and networking equipment are also pre-assembled in the factory. So the data center is assembled like a constructor - by connecting containers into a single network of communications.
And what else should be included in the data center? Of course, things that makes those communications work: power sources and cooling systems. These are the two most important components that directly affect the reliability of your services.
Let's take a brief look at each of them.
I do not know exactly what is inside these modules, but logic suggests that all the communications must be laid out during production: electricity, network, cooling, and spatial frames for servers. Most likely, the servers and networking equipment are also pre-assembled in the factory. So the data center is assembled like a constructor - by connecting containers into a single network of communications.
And what else should be included in the data center? Of course, things that makes those communications work: power sources and cooling systems. These are the two most important components that directly affect the reliability of your services.
Let's take a brief look at each of them.
The power supply system should solve the following tasks:
- The energy source for hundred kilowatts - better direct connection with powerful Power plant
- Stability of power supply characteristics (current and voltage) + “rectification” system in case the characteristics go out of acceptable ranges.
- Backup power system, consisting of generators, batteries and automatic emergency switching to offline mode.
https://pixabay.com/
Cooling system:
- Heat dissipation
- Humidity (and static) control
- Minimization of resource consumption and impact on the environment
https://pixabay.com/
Sounds impressive, doesn't it? And now, add to all of this the ecological requirements for the impact on the environment, and it turns out that what seemed to be the most important thing - servers with computing resources and network equipment - is almost the simplest component of the data center.
The complexity and value of the energy and cooling systems are truly enormous and require highly skilled engineers who deal with them. After all, if one or two servers fail or the network gateway goes down, the damage will not be as great as if the power or cooling system of the entire data center fails.
But that's not all. There are other departments in the data center without which its life is impossible. These are security personnel responsible for both protecting against information incidents and for the "physical" security of the data center. And if we are all more or less familiar with information security, when it comes to "physical" security, we think of towers with automatic weapons and patrols with dogs :) But I think in reality it is more likely to be a number of cameras with an intelligent image analysis system and a security squad capable of catching fast perimeter violators.
And one more thing we need to mention is management. All the parts and teams o the data centers need to be managed somehow. Moreover, managed in such a way that the number of incidents leading to hardware and connectivity failures tends toward zero. And as we know from Murphy's Laws, anything can break down at the most inappropriate moment.
Here are several rules that data center managers follow:
We can talk about data centers for a very long time, because the topic is very complex, and at the same time extremely interesting. However, the article format is not infinite. I have tried to explain in simple language how data centers work and show it’s complexity (and attractiveness).
If you are interested in this topic, I highly recommend checking out the Azure Infrastructure website:
https://azure.microsoft.com/en-us/explore/global-infrastructure
Look at the global map of Azure regions:
https://infrastructuremap.microsoft.com/
And take a virtual tour over the Azure datacenter:
https://news.microsoft.com/stories/microsoft-datacenter-tour/
Well, that's all I have for now. Thank you for your attention!
The complexity and value of the energy and cooling systems are truly enormous and require highly skilled engineers who deal with them. After all, if one or two servers fail or the network gateway goes down, the damage will not be as great as if the power or cooling system of the entire data center fails.
But that's not all. There are other departments in the data center without which its life is impossible. These are security personnel responsible for both protecting against information incidents and for the "physical" security of the data center. And if we are all more or less familiar with information security, when it comes to "physical" security, we think of towers with automatic weapons and patrols with dogs :) But I think in reality it is more likely to be a number of cameras with an intelligent image analysis system and a security squad capable of catching fast perimeter violators.
And one more thing we need to mention is management. All the parts and teams o the data centers need to be managed somehow. Moreover, managed in such a way that the number of incidents leading to hardware and connectivity failures tends toward zero. And as we know from Murphy's Laws, anything can break down at the most inappropriate moment.
Here are several rules that data center managers follow:
- Every action is regulated by protocols.
- Every incident must be analyzed, and the corresponding protocol should be revised to prevent similar incidents from occurring in the future.
- Regular training, certification, and testing of employees.
- Regular "training" incidents and practical exercises.
- Regular external audits.
- Maximum independence from the human factor.
We can talk about data centers for a very long time, because the topic is very complex, and at the same time extremely interesting. However, the article format is not infinite. I have tried to explain in simple language how data centers work and show it’s complexity (and attractiveness).
If you are interested in this topic, I highly recommend checking out the Azure Infrastructure website:
https://azure.microsoft.com/en-us/explore/global-infrastructure
Look at the global map of Azure regions:
https://infrastructuremap.microsoft.com/
And take a virtual tour over the Azure datacenter:
https://news.microsoft.com/stories/microsoft-datacenter-tour/
Well, that's all I have for now. Thank you for your attention!