none
Azure Blob Storage: grep and concat different files together

    Question

  • I have a blob storage container full with small CSV files. What I would like to do is do a simple 'grep' on each of these CSVs and concat the results together in a single large CSV file that should be stored in the same container. In Linux terms this would look something like: grep KeyWord *.CSV > bigfile.out

    Obviously, I could mount the container and do these command but I would reckon this is will lead to having load of data-transfers from the VM to the blob storage. Other solutions would be using some custom python script that will loop over my files and concat within the python script. However, again, this would mean having to transfer each and every CSV file from blob storage towards the VM that does the filtering.

    Is there a way using Azure APIs to do this in a bit more native way? I'd assume that would perform much better than the scenarios described above.

    Monday, August 17, 2015 12:37 PM

Answers

All replies