none
Doubt about Blob Archive

    Întrebare

  • Hello friends,

    I need to do a save 50TB of data in the cloud with fair price and the blob was up to now the best.

    I have read a lot about the types of Blob options and type archives is the best and the cheapest.

    However I have some technical doubts and need the opt-in of someone with practical knowledge:

    1) What is the maximum data ingestion rate in Azure Blobs Archive Tier. We have a temporary link of 100 Mbps. Is this rate achievable?

    2) Can AZ Copy copy directly to the Azure Blobs Archive Tier?

    3) If AZ Copy is used and the VPN falls in the middle of a data send, is there a resumption control from the point where it was before the fall?

    4) Does Azure's volume mapping on the server as a Windows letter allow mapping directly to the Azure Blobs Archive Tier?

    5) If Azure volume mapping is used on the server, how would the copy be performed in order to have resume control after a crash?

    Thank you.
    29 mai 2018 03:28

Toate mesajele

  • Number of storage accounts per region     200

    Max storage account capacity         500 TiB

    Max number of blob containers, blobs, file shares, tables, queues, entities, or messages per storage account         No limit

    Max ingress3 per storage account (US Regions)    10 Gbps if RA-GRS/GRS enabled, 20 Gbps for LRS/ZRS4.

    Max egress3 per storage account (US Regions)    20 Gbps if RA-GRS/GRS enabled, 30 Gbps for LRS/ZRS4.

    Max ingress3 per storage account (Non-US regions)       5 Gbps if RA-GRS/GRS enabled, 10 Gbps for LRS/ZRS4.

    Max egress3 per storage account (Non-US regions)        10 Gbps if RA-GRS/GRS enabled, 15 Gbps for LRS/ZRS4.

    AzCopy copies data to Blob container.

    Once the upload is canceled, you can save the checkpoint, and resume your upload from the saved checkpoint. 

    Refer to this link for details on Naming and Referencing Containers, Blobs, and Metadata.

    ------------------------------------------------------------------------------------------
    If this answer was helpful, click “Mark as Answer” or “Up-Vote”. To provide additional feedback on your forum experience, click 
    here.

    29 mai 2018 04:34
    Moderator
  • Number of storage accounts per region     200

    Max storage account capacity         500 TiB

    Max number of blob containers, blobs, file shares, tables, queues, entities, or messages per storage account         No limit

    Max ingress3 per storage account (US Regions)    10 Gbps if RA-GRS/GRS enabled, 20 Gbps for LRS/ZRS4.

    Max egress3 per storage account (US Regions)    20 Gbps if RA-GRS/GRS enabled, 30 Gbps for LRS/ZRS4.

    Max ingress3 per storage account (Non-US regions)       5 Gbps if RA-GRS/GRS enabled, 10 Gbps for LRS/ZRS4.

    Max egress3 per storage account (Non-US regions)        10 Gbps if RA-GRS/GRS enabled, 15 Gbps for LRS/ZRS4.

    AzCopy copies data to Blob container.

    Once the upload is canceled, you can save the checkpoint, and resume your upload from the saved checkpoint. 

    Refer to this link for details on Naming and Referencing Containers, Blobs, and Metadata.

    ------------------------------------------------------------------------------------------
    If this answer was helpful, click “Mark as Answer” or “Up-Vote”. To provide additional feedback on your forum experience, click 
    here.

    Hello,

    thank you for the informations.

    Is this rate information for any type of Blob? (Hot, cold or file)

    After copying the file to the Blob File, can I map the volume / directory to restore need or will I only have access to the files after requesting the support team?

    Thank you.
    29 mai 2018 12:59
  • The information I have shared in my first reply is about Scalability targets for a storage account, Refer to this link for scalability details of Azure blob storage, Azure file, Azure file sync, Azure queue storage and Azure table storage.

    Once you have created a storage account you have full access/permission to your blob, table, queue storages.


    30 mai 2018 08:48
    Moderator
  • @Sandrix,

    Q. Is this rate information for any type of Blob? (Hot, cold or file)

    Ans: Yes, you are correct.

    Q. Can I map the volume / directory to restore need or will I only have access to the files after requesting the support team?

    Ans: you can map/mount Blob storage as a file system to restore data from the blob storage if you are using a Linux OS.

    You have to use the AzCopy tool/ Azure PowerShell/ Storage Explorerto  restore/copy data from the blob storageif you are using a windows OS.   


    30 mai 2018 14:28
    Moderator
  • @Sandrix,

    Q. Is this rate information for any type of Blob? (Hot, cold or file)

    Ans: Yes, you are correct.

    Q. Can I map the volume / directory to restore need or will I only have access to the files after requesting the support team?

    Ans: you can map/mount Blob storage as a file system to restore data from the blob storage if you are using a Linux OS.

    You have to use the AzCopy tool/ Azure PowerShell/ Storage Explorerto  restore/copy data from the blob storageif you are using a windows OS.   


    Hello,

    Thanks for the answers.

    That means that when you talk about drive mapping it's just using Linux, right?

    I have a question regarding the type of Blobs with regard to restoration.

    There are differences in cost for blob types, so there is different time at restore time.

    Whether using mapping with Linux or on Windows using AzCopy tool / Azure PowerShell / Storage Explorer for the type of blob file tier, the file will only be available after requesting the restore by tickect, right?

    This means that I will upload the files to the blob file tier and will only have access after opening a ticket, right?

    Thank you.
    31 mai 2018 05:06
  • Q. That means that when you talk about drive mapping it's just using Linux, right?

    Ans: Yes, you are correct.

    Q. Whether using mapping with Linux or on Windows using AzCopy tool / Azure PowerShell / Storage Explorer for the type of blob file tier, the file will only be available after requesting the restore by ticket, right?

    Ans: No need to create a support ticket to copy/restore data from the Azure blob to an on-premises server. You can copy/restore data at any time from the Azure blob storage.

    31 mai 2018 15:52
    Moderator
  • Q. That means that when you talk about drive mapping it's just using Linux, right?

    Ans: Yes, you are correct.

    Q. Whether using mapping with Linux or on Windows using AzCopy tool / Azure PowerShell / Storage Explorer for the type of blob file tier, the file will only be available after requesting the restore by ticket, right?

    Ans: No need to create a support ticket to copy/restore data from the Azure blob to an on-premises server. You can copy/restore data at any time from the Azure blob storage.

    Hello,

    Thanks for the answers.

    Good to talk to you, as it seems to me that you have practical knowledge. This is good ...

    Strange, because I read that the type of Blob Arquive Tier, being cheaper and used for those who do not access frequently is necessary to open a ticket and has a higher SLA to recover.

    So is not it?

    A friend explained to me the differences (hot, cold and arquive tier) where the files are stored on tape, other than the hot and cold ones that are stored on faster disks.

    This practical process is still not clear to me, I say in relation to the real difference between blob types in regards to sending and restoring.

    Thank you.


    1 iunie 2018 03:19
  • There are three tiers of blob storage in Azure:

    • Hot: The cheapest to access, but the most expensive to store.
    • Cool: Medium price storage, but expensive to access.
    • Archive: Extremely cheap per GB storage (~$2.05 per TB per month)

    Archive storage is unique because it does not offer read performance – you cannot download or directly access blobs (files) from archive storage. You can only send items from hot/cool storage to archive storage, and then “rehydrate” the blobs again by restoring them to hot/cool storage – then you can download or read the blobs. Hot and cool storage have a read latency of milliseconds but rehydrating a blob from archive storage can take up to 15 hours.

    Cool and archive storage both have minimum storage durations. For example, if you place a file into cool storage, Azure expects you to keep that file there for a minimum of 30 days. If you retrieve it after 5 days, then there’s a pro-rated minimum storage charge of 25 days (30-5). Archive storage expects you to keep files in that tier for at least 180 days. If I retrieve a file after 5 days, then there is a pro-rated charge of 175 days (180-5). In other words, only put things into cool or archive storage if they are either being used infrequently (cool) or not at all (archive)

    Refer : Azure Blob storage: Hot, cool, and archive storage tiers

    --------------------------------------------------------------------------------------------------

    If this answer was helpful, click “Mark as Answer” or Up-Vote. To provide additional feedback on  your forum experience, click here



    2 iunie 2018 20:22
    Moderator
  • There are three tiers of blob storage in Azure:

    • Hot: The cheapest to access, but the most expensive to store.
    • Cool: Medium price storage, but expensive to access.
    • Archive: Extremely cheap per GB storage (~$2.05 per TB per month)

    Archive storage is unique because it does not offer read performance – you cannot download or directly access blobs (files) from archive storage. You can only send items from hot/cool storage to archive storage, and then “rehydrate” the blobs again by restoring them to hot/cool storage – then you can download or read the blobs. Hot and cool storage have a read latency of milliseconds but rehydrating a blob from archive storage can take up to 15 hours.

    Cool and archive storage both have minimum storage durations. For example, if you place a file into cool storage, Azure expects you to keep that file there for a minimum of 30 days. If you retrieve it after 5 days, then there’s a pro-rated minimum storage charge of 25 days (30-5). Archive storage expects you to keep files in that tier for at least 180 days. If I retrieve a file after 5 days, then there is a pro-rated charge of 175 days (180-5). In other words, only put things into cool or archive storage if they are either being used infrequently (cool) or not at all (archive)

    Refer : Azure Blob storage: Hot, cool, and archive storage tiers

    --------------------------------------------------------------------------------------------------

    If this answer was helpful, click “Mark as Answer” or Up-Vote. To provide additional feedback on  your forum experience, click here



    Hi,

    Thanks for the info.

    I believe we are relying on technical concept, but in practice, it works like?

    It says "you can not download or directly access blobs (files) from archive storage". I do not understand this.

    If I hire the archived blob, I'll do the sending for it, this I understood, but the restore, how does it work?

    I will have access to view the files using AzCopy tool / Azure PowerShell / Storage Explorer, but I will not be able to download them?

    If I want to send more files? Is it infinite? Or do you have a ceiling?


    Thank you.

    3 iunie 2018 03:33
  • There are three tiers of blob storage in Azure:

    • Hot: The cheapest to access, but the most expensive to store.
    • Cool: Medium price storage, but expensive to access.
    • Archive: Extremely cheap per GB storage (~$2.05 per TB per month)

    Archive storage is unique because it does not offer read performance – you cannot download or directly access blobs (files) from archive storage. You can only send items from hot/cool storage to archive storage, and then “rehydrate” the blobs again by restoring them to hot/cool storage – then you can download or read the blobs. Hot and cool storage have a read latency of milliseconds but rehydrating a blob from archive storage can take up to 15 hours.

    Cool and archive storage both have minimum storage durations. For example, if you place a file into cool storage, Azure expects you to keep that file there for a minimum of 30 days. If you retrieve it after 5 days, then there’s a pro-rated minimum storage charge of 25 days (30-5). Archive storage expects you to keep files in that tier for at least 180 days. If I retrieve a file after 5 days, then there is a pro-rated charge of 175 days (180-5). In other words, only put things into cool or archive storage if they are either being used infrequently (cool) or not at all (archive)

    Refer : Azure Blob storage: Hot, cool, and archive storage tiers

    --------------------------------------------------------------------------------------------------

    If this answer was helpful, click “Mark as Answer” or Up-Vote. To provide additional feedback on  your forum experience, click here



    Hi,

    Thanks for the info.

    I believe we are relying on technical concept, but in practice, it works like?

    It says "you can not download or directly access blobs (files) from archive storage". I do not understand this.

    If I hire the archived blob, I'll do the sending for it, this I understood, but the restore, how does it work?

    I will have access to view the files using AzCopy tool / Azure PowerShell / Storage Explorer, but I will not be able to download them?

    If I want to send more files? Is it infinite? Or do you have a ceiling?


    Thank you.

    Hello,

    I just created a Blob on Azure.

    It says the type of layer is Hot.

    How do I change to arquive tier?

    Talk about hidrotation and rehydration. How does this work in practice?

    Do we do this via Azure Storage Explorer?

    Thank you.

    3 iunie 2018 05:15
  • Azure storage offers three storage tiers for Blob object storage so that you can store your data most cost-effectively depending on how you use it. The Azure hot storage tier is optimized for storing data that is accessed frequently. The Azure cool storage tier is optimized for storing data that is infrequently accessed and stored for at least 30 days. The Azure archive storage tier is optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours). 

    You may only tier your object storage data to hot, cool, or archive in Blob storage or General Purpose v2 (GPv2) accounts. General Purpose v1 (GPv1) accounts do not support tiering. 

    While a blob/data is in archive storage, it is offline and cannot be read (except the metadata, which is online and available), copied, overwritten, or modified. Nor can you take snapshots of a blob in archive storage. However, you may use existing operations to delete, list, get blob properties/metadata, or change the tier of your blob.

    To read data in archive storage, you must first change the tier of the blob to hot or cool. This process is known as rehydration and can take up to 15 hours to complete. Large blob sizes are strongly recommended for optimal performance. Rehydrating several small blobs concurrently may add additional time.

    During rehydration, you may check the Archive Status blob property to confirm if the tier has changed. The status reads "rehydrate-pending-to-hot" or "rehydrate-pending-to-cool" depending on the destination tier. Upon completion, the archive status property is removed, and the Access Tier blob property reflects the new hot or cool tier.

    For more information refer:

    https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-storage-tiers 


    3 iunie 2018 14:47
    Moderator
  • Azure storage offers three storage tiers for Blob object storage so that you can store your data most cost-effectively depending on how you use it. The Azure hot storage tier is optimized for storing data that is accessed frequently. The Azure cool storage tier is optimized for storing data that is infrequently accessed and stored for at least 30 days. The Azure archive storage tier is optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours). 

    You may only tier your object storage data to hot, cool, or archive in Blob storage or General Purpose v2 (GPv2) accounts. General Purpose v1 (GPv1) accounts do not support tiering. 

    While a blob/data is in archive storage, it is offline and cannot be read (except the metadata, which is online and available), copied, overwritten, or modified. Nor can you take snapshots of a blob in archive storage. However, you may use existing operations to delete, list, get blob properties/metadata, or change the tier of your blob.

    To read data in archive storage, you must first change the tier of the blob to hot or cool. This process is known as rehydration and can take up to 15 hours to complete. Large blob sizes are strongly recommended for optimal performance. Rehydrating several small blobs concurrently may add additional time.

    During rehydration, you may check the Archive Status blob property to confirm if the tier has changed. The status reads "rehydrate-pending-to-hot" or "rehydrate-pending-to-cool" depending on the destination tier. Upon completion, the archive status property is removed, and the Access Tier blob property reflects the new hot or cool tier.

    For more information refer:

    https://social.msdn.microsoft.com/Forums/en-US/1cc4fa66-8861-46d0-8422-9a258e7f0fb9/doubt-about-blob-archive?forum=windowsazuredata

    Hello,

    this more operational information got better now.

    Trust me if my understanding is now clear.

    When I create it it's a hot Blob.

    So, I'm going to copy the files to the Hot Blog and if I do not use these data for 30 days, it becomes cold and if it reaches 180 days it becomes archived. Automatically?

    Is this a level of blob or files? (I mean, the type of layer)

    If it is automatically, for me to have access to the data that has become tiered, I need to run the layer back to hot again, then a rehydration process starts.

    So if I create a single hot blob and send the 50TB which we will not access for 180 days or more, when I need to access one or more files, will I have to rehydrate the whole blob? (It will take a long time!)

    So he suggests having small blobs to speed up rehydration, right.


    Thank you.
    3 iunie 2018 16:25
  • Q. So, I'm going to copy the files to the Hot Blog and if I do not use these data for 30 days, it becomes cold and if it reaches 180 days it becomes archived. Automatically?

    Ans: No, you have to change/move the blob tier (cool, hot and archive) manually. The Azure cool storage tier is optimized for storing data that is infrequently accessed and stored for at least 30 days. The Azure archive storage tier is optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours). Any blob that is deleted or moved out of the cool (GPv2 accounts only) or archive tier before 30 days and 180 days respectively will incur a prorated early deletion charge. You can determine how long a blob has been in the cool or archive tier by checking the Access Tier Change Time blob property which provides a stamp of the last tier change. See Cool and archive early deletion section for more details.

    Q. Is this a level of blob or files?

    Ans : Blob

    5 iunie 2018 13:23
    Moderator
  • Q. So, I'm going to copy the files to the Hot Blog and if I do not use these data for 30 days, it becomes cold and if it reaches 180 days it becomes archived. Automatically?

    Ans: No, you have to change/move the blob tier (cool, hot and archive) manually. The Azure cool storage tier is optimized for storing data that is infrequently accessed and stored for at least 30 days. The Azure archive storage tier is optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours). Any blob that is deleted or moved out of the cool (GPv2 accounts only) or archive tier before 30 days and 180 days respectively will incur a prorated early deletion charge. You can determine how long a blob has been in the cool or archive tier by checking the Access Tier Change Time blob property which provides a stamp of the last tier change. See Cool and archive early deletion section for more details.

    Q. Is this a level of blob or files?

    Ans : Blob

    Hello,

    Do you have any image of where I do this configuration?

    I searched and did not find.

    I found it just to file.

    So I could create a layered Blob file and move all the data there, without having to move to a hot / cool and then make it into archives, right?

    Thank you.
    5 iunie 2018 19:00
  • Could you explain a bit more on your queries?

    6 iunie 2018 13:01
    Moderator
  • Just checking in if you have had a chance to see the previous response. 

    14 iunie 2018 21:14
    Moderator