none
Disk Compression on Azure Data Lake Store

    Question

  • Hi,

    Can we do disk compression on Azure Data Lake Store? Please let me know.

    Thanks,

    Soumitra

    Friday, November 3, 2017 2:52 PM

Answers

  • Hi,

    There are some optimizations done to the data when it is ingested into ADLS (as called out in this paper , but those are transparent to the user and cannot be changed by the user. 

    If you want to store your data in compressed form like ORC, you will have to do that compression/conversion yourself before you write the file to ADLS.

    Thanks,
    Sachin Sheth
    Program Manager, Azure Data Lake.

    Saturday, November 4, 2017 12:54 AM

All replies

  • Hi,

    There are some optimizations done to the data when it is ingested into ADLS (as called out in this paper , but those are transparent to the user and cannot be changed by the user. 

    If you want to store your data in compressed form like ORC, you will have to do that compression/conversion yourself before you write the file to ADLS.

    Thanks,
    Sachin Sheth
    Program Manager, Azure Data Lake.

    Saturday, November 4, 2017 12:54 AM
  • Thank you Sachin. Link is not working for me. Could you please share the link again.
    Saturday, November 4, 2017 2:22 AM