Azure Storage is a Microsoft-managed cloud service that provides storage that is highly available, secure, durable, scalable and redundant. Whether it is images, audio, video, logs, configuration files, or sensor data from an IoT array, data needs to be stored in a way that can be easily accessible for analysis purposes, and Azure Storage provides options for each one of these possible use cases.
Within Azure there are two types of storage accounts, four types of storage, four levels of data redundancy and three tiers for storing files.
Two types of storage accounts:
- standard
- premium
Four types of storage:
- Blob storage
- File storage
- Table storage
- Queue storage
Four levels of data redundancy
- LRS – Locally-redundant storage
- ZRS – Zone-redundant storage
- GRS/RA-GRS – Geo-redundant storage (GRS)
- GZRS/RA-GZRS – Geo-zone-redundant storage (GZRS)
Three tiers:
- Hot
- Cool
- Archive
Storage Accounts
An Azure storage account contains all of your Azure Storage data objects: blobs, file shares, queues, tables, and disks. The storage account provides a unique namespace for your Azure Storage data that’s accessible from anywhere in the world over HTTP or HTTPS. Data in your storage account is durable and highly available, secure, and massively scalable.
Azure Document : How to create a storage account.
A storage account provides a unique namespace in Azure for your data. Every object that you store in Azure Storage has an address that includes your unique account name. The combination of the account name and the Azure Storage service endpoint forms the endpoints for your storage account.
The Azure Storage platform comprises the following data services:
- Azure Blobs are an immensely scalable object store for text and binary data. Also includes support for big data analytics through Data Lake Storage Gen2.
- Azure Files are organized file shares for cloud or on-premises deployments.
- Azure Queue is a messaging store for consistent messaging between application components.
- Azure Tables are NoSQL store for schema-less storage of structured data.
- Azure Disks are block-level storage volumes for Azure Virtual Machines.
- Standard general-purpose v2
- Premium block blobs
- Premium file shares
- Premium page blobs
Note: https://cloud.netapp.com/blog/azure-anf-blg-azure-storage-limits-at-a-glance
|
Resource |
Regions |
Limit per Storage Account |
|
Number of storage accounts per region |
N/A |
250 |
|
Maximum storage capacity |
All |
5 PB |
|
Maximum request rate |
All |
20,000 requests per second |
|
Maximum ingress |
US and Europe |
10 Gbps |
|
Maximum ingress |
Other Regions |
5 Gbps** |
|
Maximum egress |
US |
50 Gbps* |
|
Maximum egress |
Other Regions |
10 Gbps** |
|
Maximum number of virtual network rules |
All |
200 |
|
Maximum number of IP address rules |
All |
Container
A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.
Azure Blob Storage
Blob storage includes three types of resources explained below:
The following points describe the use case scenarios:
- Serving images or documents directly to a browser
- Storing Files for distributed access
- Streaming video and audio
- Writing to log Files
- Storing data for backup, restore, disaster recovery and archiving
- Storing data for analysis by an on-premises or Azure-hosted service
Azure storage offers various access tiers, which allows storing the blob object data in a very cost-effective manner. The available access tiers include:
- Hot– Augmented for storing frequently accessed data.
- Cool– Optimized for storing less frequently accessed data, and the storage period lasts for at least 30 days.
- Archive– Enhanced for storing rarely accessed data and the storage period lasts for at least 180 days with flexible latency requirements
Speed Performance:
A single blob supports up to 500 requests per second. If you have multiple clients that need to read the same blob and you might exceed this limit, then consider using a block blob storage account. A block blob storage account provides a higher request rate, or I/O operations per second (IOPS).
You can also use a content delivery network (CDN) such as Azure CDN to distribute operations on the blob. For more information about Azure CDN, see Azure CDN overview.
Limits for Blob storage:
|
Resource |
Type of Blob |
Limit |
|
Maximum size of single blob container |
5 PB |
|
|
Maximum number of blocks |
Block / Append |
50,000 |
|
Maximum block size |
Block |
100 MB |
|
Maximum total block size |
Append |
4 MiB |
|
Maximum total blob size |
Block |
Approx. 4.75 TB |
|
Maximum total blob size |
Append |
Approximately 195 GB |
|
Maximum total blob size |
Page |
8 TB |
|
Maximum stored access policies per container |
All |
5 |
|
Target request rate for blob |
All |
500/second |
|
Target throughput for blob |
Page |
60 MB/s |
|
Target throughput for blob |
Block |
Depends on storage account ingress/egress limits |
Azure Files Storage
Azure Files provide fully managed File shares in the cloud that are approachable via the industry-standard SMB. Azure File shares can be attached parallelly by cloud or on-premises deployments of Windows, Linux, and macOS. It can be cached on Windows servers with Azure File Sync for quicker access. It permits the user to set up highly obtainable network file shares that can be accessed by using the standard Server Message Block (SMB) protocol. Multiple VMs can share similar files with both read and write permissions.
The only contrast between Azure Files and files on a corporate file share is, user can access the files from anywhere by using a URL that points to the file and contains a shared access signature (SAS) token. SAS tokens can be generated by the user; they allow specified access to a private asset for a specific time period.
Storage tiers
Azure Files offers four different tiers of storage, premium, transaction optimized, hot, and cool to allow you to tailor your shares to the performance and price requirements of your scenario:
- Premium: Premium file shares are backed by solid-state drives (SSDs) and provide consistent high performance and low latency, within single-digit milliseconds for most IO operations, for IO-intensive workloads. Premium file shares are suitable for a wide variety of workloads like databases, web site hosting, and development environments. Premium file shares can be used with both Server Message Block (SMB) and Network File System (NFS) protocols.
- Transaction optimized: Transaction optimized file shares enable transaction heavy workloads that don’t need the latency offered by premium file shares. Transaction optimized file shares are offered on the standard storage hardware backed by hard disk drives (HDDs). Transaction optimized has historically been called “standard”, however this refers to the storage media type rather than the tier itself (the hot and cool are also “standard” tiers, because they are on standard storage hardware).
- Hot: Hot file shares offer storage optimized for general purpose file sharing scenarios such as team shares. Hot file shares are offered on the standard storage hardware backed by HDDs.
- Cool: Cool file shares offer cost-efficient storage optimized for online archive storage scenarios. Cool file shares are offered on the standard storage hardware backed by HDDs.
Limitations related to file shares
|
Resource |
Standard file shares |
Premium file shares |
|
Minimum share size |
N/A |
100 GB |
|
Maximum share size |
5 TB by default, can be increased up to 100TB |
100 TB |
|
Maximum file size |
1 TB |
4 TB |
|
Maximum IOPS |
1,000 IOPS* – Can be increased to 10000 IOPS |
100,000 IOPS |
|
Maximum stored access policies** |
5 |
5 |
|
Target throughput** |
60 MB/sec** – can be increased to 300MB/S |
Ingress 4,136 MB/s Egress 6,204 MB/s |
|
Maximum number of share snapshots |
200 |
200 |
|
Maximum directory/file name length (chars) |
2,048 |
2,048 |
|
Maximum hard links |
N/A |
178 |
Azure Blob Storage vs File Storage
Azure Blob Storage and File Storage, both services have their own defined properties and are implemented in different scenarios. Azure Files provides fully managed and organized cloud file shares that can be accessed from anywhere. Azure Blob Storage permits the storage of unstructured data and it can be accessed at a massive scale
Consider a development environment where every developer needs access to IDE and tools without using the internet to download it. In this situation, Azure Blob Storage would meet the need and using which the developer can only store development tools then give a link to the team to access the Blob location.
For implementing a File server in an organization, the user should choose the Azure Files option. A File server is used to share Files across departments in an organization. When it comes to File sharing, end user should not be allowed to access the copies of the file from its URI and need to be mapped locally in the computers. This is when Azure File Storage fits the organization’s need.
Azure Storage Queues
Azure Queue storage is an Azure service that implements cloud-based queues. Each queue maintains an inventory of messages. Application components access a queue employing a REST API or an Azure-supplied client library. Typically, you’ll have one or more sender components and one or more receiver components. Sender components add messages to the queue. Messages are retrieved from the front of the queue for processing by receiver components. The subsequent illustration shows multiple sender applications adding messages to the Azure Queue and one receiver application retrieving the messages. Storage Queues are part of the Azure Storage infrastructure, feature a simple REST-based GET/PUT/PEEK interface, providing reliable, persistent messaging within and between services.
Concepts of Queue service
User Case:
The middle tier provides plenty of capacity to handle normal loads. However, a look at the server logs revealed the system was overloaded when several journalists tried to upload larger breaking stories at the same time. Some writers complained the portal became unresponsive, and others said they lost their stories altogether. The user has spotted a direct correlation between the reported issues and the spike in demand on the middle tier servers.
Clearly, user needs a way to handle these unexpected peaks. At such a situation user doesn’t want to add more instances of the website and middle tier web service because they are expensive and, under normal conditions, redundant. They could dynamically spin up instances, but this takes time and they would have the issue waiting for new servers to come online.
This problem can be solved by using Azure Queue storage. A storage queue is a high-performance message buffer that can act as a broker between the front-end components and the middle tier. The front-end components place a message for each new alert into a queue. The middle tier then retrieves these messages one at a time from the queue for processing. At times of high demand, the queue may grow in length, but no stories will be lost, and the application will remain responsive. When demand drops back to normal levels, the web service will catch up by working through the queue backlog.
Azure Table Storage
Azure Table storage behaves as a service that stores structured NoSQL data inside the cloud, producing an attribute store with a schema less design. Because Table storage is schema less, it is easy to adapt your data because the needs of your application evolve. Access to Table storage data is fast and cost-effective for several sorts of applications and is usually lower in cost than traditional SQL for similar volumes of knowledge.
Table storage is employed to store flexible datasets like user data for web applications, address books, device information, or other metadata that the service requires. User can store any number of entities in a table, and a storage account may contain any number of tables, up to the capacity limit of the storage account Azure Table Storage Pricing. Azure tables are perfect for storing structured, non-relational data. Real time uses of Table storage include:
- Storing datasets that do not require complex joins, foreign keys, or stored procedures and may be denormalized for fast access
- Quickly querying data using a clustered index
- Accessing data using the OData protocol and LINQ queries with WCF Data Service .NET Libraries
Azure Disk Storage
Azure managed disks are block-level storage parts that are managed by Azure and used with Azure Virtual Machines. Managed disks are similar a physical disk in an on-premises server but virtualized. In managed disks, all you must do is specify the disk size, type, and provision the disk. Once the disk is provisioned, Azure handles the rest. Each disk can take one of three roles in a virtual machine:
- OS disk. One disk in each virtual machine contains the operating system files. When user creates a virtual machine, he/she selects a virtual machine image and that fixes the operating system and the OS disk that’s attached to the new machine. The OS disk has a maximum capacity of 2,048 GB.
- Data disk. User can add one or more data virtual disks to each virtual machine to store data. For example, database files, website static content, or custom application code should be stored on data disks. The number of data disks that can be added depends on the virtual machine size. Each data disk has a maximum capacity of 32,767 GB.
- Temporary disk. Each virtual machine contains a single temporary disk, which is used for short-term storage applications such as page files and swap files. The contents of temporary disks are lost during maintenance events, so do not use these disks for critical data. These disks are local to the server and are not stored in a storage account.
Azure Disk Storage Use case
The user manages a healthcare organization, and he is beginning a lift-and-shift migration to the cloud where many of their systems will be running on Azure virtual machines. These systems have a variety of usage and performance profiles which are highly confidential. The user is concerned about the storage and does not want to access that data outside the virtual machine.
To address these needs, the organization’s got to option is Azure Disk Storage. The Azure Disk Storage is capable of,
- “Lift and shift” of applications that use native file system APIs to read and write data to persistent disks.
- Preserve data that is not required to be accessed from outside the virtual machine to which the disk is attached.
References
- Azure Storage Account Overview
- Azure Blob Storage documentation
- AZ-104-MICROSOFTAZUREADMINISTRATOR – Lab 07 – Manage Azure Storage Student lab manual