Section 13 – Secure data and applications – Configure security for storage

DJ Blogs: What is Azure Storage Account

  • Configure access control for storage accounts
  • Configure storage account access keys
  • Configure Azure AD authentication for Azure Storage and Azure Files

Different Storage Accounts types

The Azure Storage platform includes the following data services:

Azure Blobs

A massively scalable object store for text and binary data. Also includes support for big data analytics through Data Lake Storage Gen2.

Blob storage is ideal for:

  • Serving images or documents directly to a browser.
  • Storing files for distributed access.
  • Streaming video and audio.
  • Storing data for backup and restore, disaster recovery, and archiving.
  • Storing data for analysis by an on-premises or Azure-hosted service.

Objects in Blob storage can be accessed from anywhere in the world via HTTP or HTTPS. Users or client applications can access blobs via URLs, the Azure Storage REST APIAzure PowerShellAzure CLI, or an Azure Storage client library.

Azure Files

Managed file shares for cloud or on-premises deployments.

One thing that distinguishes Azure Files from files on a corporate file share is that you can access the files from anywhere in the world using a URL that points to the file and includes a shared access signature (SAS) token. You can generate SAS tokens; they allow specific access to a private asset for a specific amount of time.

File shares can be used for many common scenarios:

  • Many on-premises applications use file shares. This feature makes it easier to migrate those applications that share data to Azure. If you mount the file share to the same drive letter that the on-premises application uses, the part of your application that accesses the file share should work with minimal, if any, changes.
  • Configuration files can be stored on a file share and accessed from multiple VMs. Tools and utilities used by multiple developers in a group can be stored on a file share, ensuring that everybody can find them, and that they use the same version.
  • Resource logs, metrics, and crash dumps are just three examples of data that can be written to a file share and processed or analyzed later.

Azure Queues

A messaging store for reliable messaging between application components.

The Azure Queue service is used to store and retrieve messages. Queue messages can be up to 64 KB in size, and a queue can contain millions of messages. Queues are generally used to store lists of messages to be processed asynchronously.

Azure Tables

A NoSQL store for schemaless storage of structured data.

Azure Table storage is now part of Azure Cosmos DB. To see Azure Table storage documentation, see the Azure Table Storage Overview. In addition to the existing Azure Table storage service, there is a new Azure Cosmos DB Table API offering that provides throughput-optimized tables, global distribution, and automatic secondary indexes.

Azure Disks

Block-level storage volumes for Azure VMs.

An Azure managed disk is a virtual hard disk (VHD). You can think of it like a physical disk in an on-premises server but virtualized. Azure-managed disks are stored as page blobs, which are a random IO storage object in Azure. We call a managed disk ‘managed’ because it is an abstraction over page blobs, blob containers, and Azure storage accounts. With managed disks, all you have to do is provision the disk, and Azure takes care of the rest.

And each service is accessed through a storage account.

What about different options?

Premium storage account types

When creating a new storage accounts and choosing Premium, you will these options

Type of storage accountSupported storage servicesRedundancy optionsUsage
Standard general-purpose v2Blob Storage (including Data Lake Storage1), Queue Storage, Table Storage, and Azure FilesLocally redundant storage (LRS) / geo-redundant storage (GRS) / read-access geo-redundant storage (RA-GRS)

Zone-redundant storage (ZRS) / geo-zone-redundant storage (GZRS) / read-access geo-zone-redundant storage (RA-GZRS)2
Standard storage account type for blobs, file shares, queues, and tables. Recommended for most scenarios using Azure Storage. If you want support for network file system (NFS) in Azure Files, use the premium file shares account type.
Premium block blobs3Blob Storage (including Data Lake Storage1)LRS

ZRS2
Premium storage account type for block blobs and append blobs. Recommended for scenarios with high transaction rates or that use smaller objects or require consistently low storage latency. Learn more about example workloads.
Premium file shares3Azure FilesLRS

ZRS2
Premium storage account type for file shares only. Recommended for enterprise or high-performance scale applications. Use this account type if you want a storage account that supports both Server Message Block (SMB) and NFS file shares.
Premium page blobs3Page blobs onlyLRSPremium storage account type for page blobs only. Learn more about page blobs and sample use cases.

1 Data Lake Storage is a set of capabilities dedicated to big data analytics, built on Azure Blob Storage. For more information, see Introduction to Data Lake Storage Gen2 and Create a storage account to use with Data Lake Storage Gen2.

2 ZRS, GZRS, and RA-GZRS are available only for standard general-purpose v2, premium block blobs, and premium file shares accounts in certain regions. For more information, see Azure Storage redundancy.

3 Premium performance storage accounts use solid-state drives (SSDs) for low latency and high throughput.

Redundancy

Locally-redundant storage (LRS)

Diagram showing how data is replicated in a single data center with LRS

A write request to a storage account that is using LRS happens synchronously. The write operation returns successfully only after the data is written to all three replicas.

Zone-redundant storage (ZRS)

Diagram showing how data is replicated in the primary region with ZRS

With ZRS, your data is still accessible for both read and write operations even if a zone becomes unavailable. If a zone becomes unavailable, Azure undertakes networking updates, such as DNS re-pointing. These updates may affect your application if you access data before the updates have completed.

ZRS is supported for all Azure Storage services through standard general-purpose v2 storage accounts, including:

  • Azure Blob storage (hot and cool block blobs, non-disk page blobs)
  • Azure Files (all standard tiers: transaction optimized, hot, and cool)
  • Azure Table storage
  • Azure Queue storage

Geo-redundant storage (GRS)

Diagram showing how data is replicated with GRS or RA-GRS

Geo-redundant storage (GRS) copies your data synchronously three times within a single physical location in the primary region using LRS.

A write operation is first committed to the primary location and replicated using LRS. The update is then replicated asynchronously to the secondary region. When data is written to the secondary location, it’s also replicated within that location using LRS.

Geo-zone-redundant storage (GZRS)

Diagram showing how data is replicated with GZRS or RA-GZRS

Only standard general-purpose v2 storage accounts support GZRS. GZRS is supported by all of the Azure Storage services, including:

  • Azure Blob storage (hot and cool block blobs, non-disk page blobs)
  • Azure Files (all standard tiers: transaction optimized, hot, and cool)
  • Azure Table storage
  • Azure Queue storage

Read-access geo-redundant storage (RA-GRS) or read-access geo-zone-redundant storage (RA-GZRS)

To determine which write operations have been replicated to the secondary region, your application can check the Last Sync Time property for your storage account.

Availability

ParameterLRSZRSGRS/RA-GRSGZRS/RA-GZRS
Percent durability of objects over a given yearat least 99.999999999% (11 9’s)at least 99.9999999999% (12 9’s)at least 99.99999999999999% (16 9’s)at least 99.99999999999999% (16 9’s)
Availability for read requestsAt least 99.9% (99% for Cool or Archive access tiers)At least 99.9% (99% for Cool or Archive access tiers)At least 99.9% (99% for Cool or Archive access tiers) for GRS

At least 99.99% (99.9% for Cool or Archive access tiers) for RA-GRS
At least 99.9% (99% for Cool or Archive access tiers) for GZRS

At least 99.99% (99.9% for Cool or Archive access tiers) for RA-GZRS
Availability for write requestsAt least 99.9% (99% for Cool or Archive access tiers)At least 99.9% (99% for Cool or Archive access tiers)At least 99.9% (99% for Cool or Archive access tiers)At least 99.9% (99% for Cool or Archive access tiers)
Number of copies of data maintained on separate nodesThree copies within a single regionThree copies across separate availability zones within a single regionSix copies total, including three in the primary region and three in the secondary regionSix copies total, including three across separate availability zones in the primary region and three locally redundant copies in the secondary region

Summary of options

Type of storage accountSupported storage servicesRedundancy optionsUsage
Standard general-purpose v2Blob (including Data Lake Storage1), Queue, and Table storage, Azure FilesLRS/GRS/RA-GRS

ZRS/GZRS/RA-GZRS2
Standard storage account type for blobs, file shares, queues, and tables. Recommended for most scenarios using Azure Storage. Note that if you want support for NFS file shares in Azure Files, use the premium file shares account type.
Premium block blobs3Blob storage (including Data Lake Storage1)LRS

ZRS2
Premium storage account type for block blobs and append blobs. Recommended for scenarios with high transactions rates, or scenarios that use smaller objects or require consistently low storage latency. Learn more about example workloads.
Premium file shares3Azure FilesLRS

ZRS2
Premium storage account type for file shares only. Recommended for enterprise or high-performance scale applications. Use this account type if you want a storage account that supports both SMB and NFS file shares.
Premium page blobs3Page blobs onlyLRSPremium storage account type for page blobs only. Learn more about page blobs and sample use cases.

Data Lake Storage is a set of capabilities dedicated to big data analytics, built on Azure Blob storage. For more information, see Introduction to Data Lake Storage Gen2 and Create a storage account to use with Data Lake Storage Gen2.

2 Zone-redundant storage (ZRS) and geo-zone-redundant storage (GZRS/RA-GZRS) are available only for standard general-purpose v2, premium block blobs, and premium file shares accounts in certain regions. For more information, see Azure Storage redundancy.

3 Premium performance storage accounts use solid-state drives (SSDs) for low latency and high throughput.

You cannot change a storage account to a different type after it is created. To move your data to a storage account of a different type, you must create a new account and copy the data to the new account.

Configure access control for storage accounts

RBAC and Data roles

Azure Active Directory (AAD) authorizes access rights to secured resources through Azure role-based access control (Azure RBAC). Azure Storage defines a set of Azure built-in roles that encompass common sets of permissions used to access blob data.

When an Azure role is assigned to an Azure AD security principal, Azure grants access to those resources for that security principal. An Azure AD security principal may be a user, a group, an application service principal, or a managed identity for Azure resources.

Azure built-in roles for blobs

Azure RBAC provides a number of built-in roles for authorizing access to blob data using Azure AD and OAuth. Some examples of roles that provide permissions to data resources in Azure Storage include:

Full list of Built-in Data roles

Here is the full list of Data based roles inside Azure RBAC

Built-in roleDescriptionID
Storage
Classic Storage Account ContributorLets you manage classic storage accounts, but not access to them.86e8f5dc-a6e9-4c67-9d15-de283e8eac25
Classic Storage Account Key Operator Service RoleClassic Storage Account Key Operators are allowed to list and regenerate keys on Classic Storage Accounts985d6b00-f706-48f5-a6fe-d0ca12fb668d
Data Box ContributorLets you manage everything under Data Box Service except giving access to others.add466c9-e687-43fc-8d98-dfcf8d720be5
Data Box ReaderLets you manage Data Box Service except creating order or editing order details and giving access to others.028f4ed7-e2a9-465e-a8f4-9c0ffdfdc027
Data Lake Analytics DeveloperLets you submit, monitor, and manage your own jobs but not create or delete Data Lake Analytics accounts.47b7735b-770e-4598-a7da-8b91488b4c88
Reader and Data AccessLets you view everything but will not let you delete or create a storage account or contained resource. It will also allow read/write access to all data contained in a storage account via access to storage account keys.c12c1c16-33a1-487b-954d-41c89c60f349
Storage Account ContributorPermits management of storage accounts. Provides access to the account key, which can be used to access data via Shared Key authorization.17d1049b-9a84-46fb-8f53-869881c3d3ab
Storage Account Key Operator Service RolePermits listing and regenerating storage account access keys.81a9662b-bebf-436f-a333-f67b29880f12
Storage Blob Data ContributorRead, write, and delete Azure Storage containers and blobs. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.ba92f5b4-2d11-453d-a403-e96b0029c9fe
Storage Blob Data OwnerProvides full access to Azure Storage blob containers and data, including assigning POSIX access control. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.b7e6dc6d-f1e8-4753-8033-0f276bb0955b
Storage Blob Data ReaderRead and list Azure Storage containers and blobs. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.2a2b9908-6ea1-4ae2-8e65-a410df84e7d1
Storage Blob DelegatorGet a user delegation key, which can then be used to create a shared access signature for a container or blob that is signed with Azure AD credentials. For more information, see Create a user delegation SAS.db58b8e5-c6ad-4a2a-8342-4190687cbf4a
Storage File Data SMB Share ContributorAllows for read, write, and delete access on files/directories in Azure file shares. This role has no built-in equivalent on Windows file servers.0c867c2a-1d8c-454a-a3db-ab2ea1bdc8bb
Storage File Data SMB Share Elevated ContributorAllows for read, write, delete, and modify ACLs on files/directories in Azure file shares. This role is equivalent to a file share ACL of change on Windows file servers.a7264617-510b-434b-a828-9731dc254ea7
Storage File Data SMB Share ReaderAllows for read access on files/directories in Azure file shares. This role is equivalent to a file share ACL of read on Windows file servers.aba4ae5f-2193-4029-9191-0cb91df5e314
Storage Queue Data ContributorRead, write, and delete Azure Storage queues and queue messages. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.974c5e8b-45b9-4653-ba55-5f855dd0fb88
Storage Queue Data Message ProcessorPeek, retrieve, and delete a message from an Azure Storage queue. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.8a0f0c08-91a1-4084-bc3d-661d67233fed
Storage Queue Data Message SenderAdd messages to an Azure Storage queue. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.c6a89b2d-59bc-44d0-9896-0f6e12d7b80a
Storage Queue Data ReaderRead and list Azure Storage queues and queue messages. To learn which actions are required for a given data operation, see Permissions for calling blob and queue data operations.19e7f393-937e-4f77-808e-94535e297925
Storage Table Data ContributorAllows for read, write and delete access to Azure Storage tables and entities0a9a7e1f-b9d0-4cc4-a60d-0319b160aaa3
Storage Table Data ReaderAllows for read access to Azure Storage tables and entities76199698-9eea-4c19-bc75-cec21354c6b6
Web
Azure Spring Cloud Data ReaderAllow read access to Azure Spring Cloud Datab5537268-8956-4941-a8f0-646150406f0c
Other
Azure Digital Twins Data OwnerFull access role for Digital Twins data-planebcd981a7-7f74-457b-83e1-cceb9e632ffe
Azure Digital Twins Data ReaderRead-only role for Digital Twins data-plane propertiesd57506d4-4c8d-48b1-8587-93c323f6a5a3

You can find the with search under RBAC assignements

As an example for Storage Blob Data Owner we can see the Data actions from the template. There are separate actions for services and the data itself

Configure storage account access keys

Shared access signatures (SAS)

There are two SAS keys in a storage account. This is for usability reasons, of you need to rotate the other key, the other one still works and your service don’t get downtime.

A shared access signature (SAS) is a URI that grants restricted access rights to Azure Storage resources. You can provide a shared access signature to clients who should not be trusted with your storage account key but whom you wish to delegate access to certain storage account resources. By distributing a shared access signature URI to these clients, you grant them access to a resource for a specified period of time.An account-level SAS can delegate access to multiple storage services (i.e. blob, file, queue, table). Note that stored access policies are currently not supported for an account-level SAS.

You have the following options to generate the connection string. You have to allow some of the three resource types to generate a connection string.

When you click Generate you will get a random string based on the settings you chose, one for each service type.

Let’s have a closer look at the File service SAS URL. Here you can see the ip-address and the start and the end times for the string generated.

And you can use this string to connect example with Azure Storage Explorer.

And choose either one.

Paste in the connection string that was generated.

And hit connect after reviewing the settings are correct.

And you can see the services under the connection we just made.

Access Keys

Access keys authenticate your applications’ requests to this storage account. Keep your keys in a secure location like Azure Key Vault, and replace them often with new keys. The two keys allow you to replace one while still using the other.

You have two keys.

And again you can connect to the Storage Account with the key and name of the account, like this.

Verify settings and connect.

Now you can see the connection we just made in the Explorer.

Shared Access Tokens

A shared access signature (SAS) is a URI that grants restricted access to an Azure Storage container. Use it when you want to grant access to storage account resources for a specific time range without sharing your storage account key.

You can find the Access tokens under the corresponding service but not Azure Files, there is an preview for a feature that allows you to add Azure AD users directly to Azure Files, yes without Azure Domain Services.

Generating the token is the same as with the Connection string so I won’t be going thru this one.

Access policy

A stored access policy provides an additional level of control over service-level shared access signatures (SAS) on the server side. Establishing a stored access policy serves to group shared access signatures and to provide additional restrictions for signatures that are bound by the policy. You can use a stored access policy to change the start time, expiry time, or permissions for a signature, or to revoke it after it has been issued.

The following storage resources support stored access policies:

  • Blob containers
  • File shares
  • Queues
  • Tables

When you click Add policy you will get almost the same options than with Access keys.

You can add permissions and time limit for the policy.

Lifecycle management

With the lifecycle management policy, you can:

  • Transition blobs from cool to hot immediately when they are accessed, to optimize for performance.
  • Transition blobs, blob versions, and blob snapshots to a cooler storage tier if these objects have not been accessed or modified for a period of time, to optimize for cost. In this scenario, the lifecycle management policy can move objects from hot to cool, from hot to archive, or from cool to archive.
  • Delete blobs, blob versions, and blob snapshots at the end of their lifecycles.
  • Define rules to be run once per day at the storage account level.
  • Apply rules to containers or to a subset of blobs, using name prefixes or blob index tags as filters.

Geo-replication

You can enable your geo-replication inside the configuration.

And now you can see two different Data centers inside the geo-replication.

Security for Storage

Defender for storage

Azure storage accounts are billed hourly. Applies to Blob containers, File shares and Data lakes Gen2.

Threat protection alerts – Advanced behavioral analytics and the Microsoft Intelligent Security Graph provide an edge over evolving cyber-attacks. Built-in behavioral analytics and machine learning can identify attacks and zero-day exploits. Monitor networks, machines, data stores (SQL servers hosted inside and outside Azure, Azure SQL databases, Azure SQL Managed Instance, and Azure Storage) and cloud services for incoming attacks and post-breach activity. Streamline investigation with interactive tools and contextual threat intelligence.

You can find the recommendations also from Storage accounts -> Security

Explore security anomalies

When storage activity anomalies occur, you receive an email notification with information about the suspicious security event. Details of the event include:

  • The nature of the anomaly
  • The storage account name
  • The event time
  • The storage type
  • The potential causes
  • The investigation steps
  • The remediation steps

Encryption

Storage service encryption protects your data at rest. Azure Storage encrypts your data as it’s written in our datacenters, and automatically decrypts it for you as you access it. Note that after enabling Storage Service Encryption, only new data will be encrypted, and any existing files in this storage account will retroactively get encrypted by a background encryption process.

You can encrypt your data with your own key. It can be stored in Key vault.

Networking

By default the access is allowed from all networks.

You can choose the network you have already defined or create new ones. It’s also possible to add exceptions to these rules.

Or you can use private endpoints.

The type of sub-resource for the resource selected above that your private endpoint will be able to access.

and Private Link is created.

Azure Private Link is a security feature that will flow the traffic inside Microsoft Backbone network.

Secure and public access

Disable public access if not needed and enable Secure access (https) if not already enabled. For new provisions it’s already enabled by default.

Data protection

Microsoft offers data protection controls for Storage accounts. You can find them under Data proctection.

Azure AD

Enable Azure AD authentication for Storage services. It the following I enabled it to Azure Files.

This feature is currently in Preview.

Managed identities

Use managed identities to authenticate your usage instead of normal users. You can use the below Managed identity types.

Configure Azure AD authentication for Azure Storage and Azure Files

Azure Files with Azure AD DS

Once either Azure AD DS or on-premises AD DS authentication is enabled, you can use Azure built-in roles or configure custom roles for Azure AD identities and assign access rights to any file shares in your storage accounts. The assigned permission allows the granted identity to get access to the share only, nothing else, not even the root directory. You still need to separately configure directory or file-level permissions for Azure file shares.

Microsoft recommends using ZRS for Azure Files workloads. If a zone becomes unavailable, no remounting of Azure file shares from the connected clients is required.

Storage account

Let’s say we have this kind of setup for the storage

Some explanation on the options

When storage account key access is disabled, any requests to the account that are authorized with Shared Key, including shared access signatures (SAS), will be denied. Client applications that currently access the storage account using shared key will no longer work

When blob public access is enabled, one is permitted to configure container ACLs to allow anonymous access to blobs within the storage account. When disabled, no anonymous access to blobs within the storage account is permitted, regardless of underlying ACL configurations

Microsoft network routing will direct your traffic to enter the Microsoft cloud as quickly as possible from its source. Internet routing will direct your traffic to enter the Microsoft cloud closer to the Azure endpoint

Customer-managed key (CMK) support can be limited to blob service and file service only, or to all service types. After the storage account is created, this support cannot be changed.

Azure encrypts storage account data at rest by default with 256-bit AES encryption. Infrastructure encryption adds a second layer of encryption to your storage account’s data

Infrastructure-level encryption relies on Microsoft-managed keys and always uses a separate key.

The infrastructure encryption setting for an encryption scope cannot be changed after the scope is created.

The following table compares key management options for Azure Storage encryption.

UsageMicrosoft-managed keysCustomer-managed keysCustomer-provided keys
Encryption/decryption operationsAzureAzureAzure
Azure Storage services supportedAllBlob storage, Azure FilesBlob storage
Key storageMicrosoft key storeAzure Key VaultAzure Key Vault or any other key store
Key rotation responsibilityMicrosoftCustomerCustomer
Key usageMicrosoftAzure portal, Storage Resource Provider REST API, Azure Storage management libraries, PowerShell, CLIAzure Storage REST API (Blob storage), Azure Storage client libraries
Key accessMicrosoft onlyMicrosoft, CustomerCustomer only

File share

When you setup Azure AD DS all these settings cannot be modified after provisioning

And done

You can also enable security settings under the file shares, it’s recommended to use AES-256 encryption instead of RC4-HMAC

Once done, you can use a Domain joined workstation to access the shares.

Azure files with Azure AD

Azure AD integration is a game changer, you will have native Kerberos directly from Azure AD. In the preview state it’s currently working only with synced users and groups.

Prerequisites

The Azure AD Kerberos functionality is only available on the following operating systems:

The user accounts must be hybrid user identities, which means you’ll also need Active Directory Domain Services (AD DS) and Azure AD Connect. You must create these accounts in Active Directory and sync them to Azure AD. The service doesn’t currently support environments where users are managed with Azure AD and optionally synced to Azure AD Directory Services.

If you already have a storage account integrated to Azure AD DS, remove it or create a new storage account.

Microsoft instructions for implementation

When it’s done you will see the following inside the storage account

And you will see an App registration that was create with the PowerShell commands. Check that you have the following permissions inside API permissions

And that the Admin consent is given for the API permissions

And that the client secret is populated

Assign share-level permissions

All users that need to have FSLogix profiles stored on the storage account you’re using must be assigned the Storage File Data SMB Share Contributor role.

Assign directory level access permissions

The system you use to configure the permissions must meet the following requirements:

  • The version of Windows meets the supported OS requirements defined in the Prerequisites section.
  • Is Azure AD-joined or Hybrid Azure AD-joined to the same Azure AD tenant as the storage account.
  • Has line-of-sight to the domain controller.
  • Is domain-joined to your Active Directory (Windows Explorer method only).

During the public preview, configuring permissions using Windows Explorer also requires storage account configuration. You can skip this configuration step when using icacls.

Azure AD join

And after credentials, check info and click join

For Directory level access you will need Azure AD Hybrid or Azure AD joined devices

And you will see the difference inside Azure AD under Devices

Data migration

Once done with the setup, you can in example use Data migration to assist you with choosing the right migration method

Things to remember

Different Storage Accounts types

Redundancy types for the different types and availability

Data based roles inside RBAC and the other access types available

Different security solutions for Storage

Microsoft recommends using ZRS for Azure Files workloads.

Azure Files available with Azure ADDS and Azure AD native Kerberos

Encryption for storage

UsageMicrosoft-managed keysCustomer-managed keysCustomer-provided keys
Encryption/decryption operationsAzureAzureAzure
Azure Storage services supportedAllBlob storage, Azure FilesBlob storage
Key storageMicrosoft key storeAzure Key VaultAzure Key Vault or any other key store
Key rotation responsibilityMicrosoftCustomerCustomer
Key usageMicrosoftAzure portal, Storage Resource Provider REST API, Azure Storage management libraries, PowerShell, CLIAzure Storage REST API (Blob storage), Azure Storage client libraries
Key accessMicrosoft onlyMicrosoft, CustomerCustomer only

Link to main post

Author: Harri Jaakkonen

Leave a Reply

Your email address will not be published. Required fields are marked *