none
How to delete all files in Azure Data Lake Store folder? RRS feed

All replies

  • This feature is not built-in supported yet, and at our backlog. Today you can write your own .net codes and put with ADF custom activity to achieve the purpose.

    thx,

    -Oliver


    Oliver Yao - MSFT

    Thursday, January 25, 2018 8:13 PM
  • Even in SSIS with Azure feature pack installed - Azure Data Lake Store File System Task, there is only Copy to and Copy from supported.


    sathya - My blog ** Mark as answered if my post solved your problem and Vote as helpful if my post was useful **.

    Thursday, January 25, 2018 9:34 PM
  • As an alternative to implementing your own custom activity with .NET code, you can use ADF web activity calling ADLS REST API for deleting files in ADLS, which is easier to author and does not require deploying Azure Batch compute for running custom activity.

    ACOM doc for ADF web activity:

    https://docs.microsoft.com/en-us/azure/data-factory/control-flow-web-activity

    ADLS REST API for file deletion:

    https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-rest-api#delete-a-file

    Friday, January 26, 2018 7:30 AM
    Moderator
  • How to delete ALL 10 files in folder with Web activity and Rest API? Some kind of loop needed? 

    https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-rest-api#delete-a-file


    Kenny_I

    Friday, January 26, 2018 9:53 AM
  • How to delete ALL 10 files in folder with Web activity and Rest API? Some kind of loop needed? 

    https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-rest-api#delete-a-file


    Kenny_I

    Hi Kenny,

    One other option, if you have on-premise SQL Server. I am sure this will work but unfortunately I am not able to try this now:

    1) Create an Azure service principal with Azure PowerShell, so that we can login into Azure non interactively.

    2) Use the powershell command to delete the files

    3) Create SSIS package to invoke the powershell commands

    4) We can execute packages using Stored procedure

    5) Now execute the SP using Stored procedure activity in ADF

    Make sure Integration run time is set up in ADF connections


    sathya - My blog ** Mark as answered if my post solved your problem and Vote as helpful if my post was useful **.

    Thursday, February 8, 2018 8:27 PM
  • As an alternative to implementing your own custom activity with .NET code, you can use ADF web activity calling ADLS REST API for deleting files in ADLS, which is easier to author and does not require deploying Azure Batch compute for running custom activity.

    ACOM doc for ADF web activity:

    https://docs.microsoft.com/en-us/azure/data-factory/control-flow-web-activity

    ADLS REST API for file deletion:

    https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-data-operations-rest-api#delete-a-file

    Hi, I was able to use the Web activity to call the REST API to delete a directory in my ADLS, but my access token that was generated by the curl command is expiring after one day.  How do I generate a token with a longer expiration date?
    Thursday, January 24, 2019 4:31 PM