locked
use Azure to build cloud file system solution RRS feed

  • Question

  • We want to use Azure to build file system.
    If I am wrong please correct me. I can use Azure storage to store file data, Table (or SQL) to store file location.
    I use C# Azure SDK.
    Entry for saving file info like this:
    FileEntry
    {
    id;
    parentId;
    name;
    type;//folder or file
    size;
    }

    I start message loop at work role to get message.
    After client uploade file to Azure storage then send message.
    Work role get message and record file info to FileEntry.

    directory structure may like below
    root (Entry info: type=folder, id=1, parentId=0)
     my.jpg (Entry info: type=file,id=2, parentId=1, size=1024)
     +sub1 (Entry info: type=folder, id=3, parentId=1)
      +sub2 (Entry info: type=folder, id=4, parentId=2)


    My questions:
    1) After work role get one message, I create one thread to record file info.
       Is using thread mode efficient? Work role get thousands of request, create thousands threads?
    2) Use asynchronous socket at work role instead of message will more efficient?
    3) How can I quickly delete folder entry including subfile? after delete folder, I need to know many file size are delete.
       So that I can update total file size.
       One folder create one blob container, one table, then delete container, table?
    4) I record file name not include file path in FileEntry. Becasue file/folder will be renamed. How can I fast get FileEntry file

    path?

    Thanks!

    Friday, January 17, 2014 5:17 AM

Answers

  • Dear Kevin,

    I would love to help you but to be frank i am not following what are you trying to do on Azure..

    all the questions you have can have many answers and all can be correct but if you give a little more detail of what you are trying to do it will be so much valuable to give my inputs.

    ----------------


    Vishal Narayan Saxena http://twitter.com/vishalishere http://www.ogleogle.com/vishal/

    Friday, January 17, 2014 6:08 AM
  • hi,

    >>(current our solution,
    1. we upload file to blob at client site;
    2. send queue message to notify that file was uploaded;
    3. work role get message;
    4. work role save entry at Azure Table to record file info. Including id, parent id, name,size)
    5. Client query table to get file list.

    Why didn't  you save data into azure table on client side? I suggest you could upload the file and get the file URL. And then you could save file info into azure table on client side. Like this:

                using (var fileStream = System.IO.File.OpenRead(filepat))
                {
                    //
                    blockBlob.UploadFromStream(fileStream);
                }
                insertTable(entity);// save file info into azure table

    >>I am worry about so many threads running at same time will cost many resource. Any improve?
    I save all file info entries in one table. So when I need to delete a "folder entry", I need to get all entries in the folder, then delete. But it take time to get all entries. Any improve?

    Did you try to use azure storage rest api? I suggest you could refer to use this in your client side (http://msdn.microsoft.com/en-us/library/windowsazure/dd179355.aspx ). Or you could create a WCF service as custom API for your project. you could set the insert / delete / update  and other operation.

    Please try it.

    Hope it helps.

    Regards,

    Will


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, February 3, 2014 10:13 AM
  • Hello Kevin,

    If you are looking for a file explorer then please take a look at 

    http://azurestorageexplorer.codeplex.com/

    as it may help you.

    Thanks,

    Vishal Supekar


    Regards, Vishal Supekar BE.CSE MCPDEA(Micorsoft Certified)

    Tuesday, February 4, 2014 6:29 AM

All replies

  • Dear Kevin,

    I would love to help you but to be frank i am not following what are you trying to do on Azure..

    all the questions you have can have many answers and all can be correct but if you give a little more detail of what you are trying to do it will be so much valuable to give my inputs.

    ----------------


    Vishal Narayan Saxena http://twitter.com/vishalishere http://www.ogleogle.com/vishal/

    Friday, January 17, 2014 6:08 AM

  • Thanks for reply.

    In short, we plan use Azure to store file.
    How can I fast save file, get file, delete file, create folder, delete folder?

    (current our solution,
    1. we upload file to blob at client site;
    2. send queue message to notify that file was uploaded;
    3. work role get message;
    4. work role save entry at Azure Table to record file info. Including id, parent id, name,size)
    5. Client query table to get file list.

    In our solution,  work role will create one thread when get one message. Because save/delete entries take time.
    I am worry about so many threads running at same time will cost many resource. Any improve?
    I save all file info entries in one table. So when I need to delete a "folder entry", I need to get all entries in the folder, then delete. But it take time to get all entries. Any improve?
    )


    Thank again.
    Friday, January 17, 2014 10:09 AM
  • hi,

    >>(current our solution,
    1. we upload file to blob at client site;
    2. send queue message to notify that file was uploaded;
    3. work role get message;
    4. work role save entry at Azure Table to record file info. Including id, parent id, name,size)
    5. Client query table to get file list.

    Why didn't  you save data into azure table on client side? I suggest you could upload the file and get the file URL. And then you could save file info into azure table on client side. Like this:

                using (var fileStream = System.IO.File.OpenRead(filepat))
                {
                    //
                    blockBlob.UploadFromStream(fileStream);
                }
                insertTable(entity);// save file info into azure table

    >>I am worry about so many threads running at same time will cost many resource. Any improve?
    I save all file info entries in one table. So when I need to delete a "folder entry", I need to get all entries in the folder, then delete. But it take time to get all entries. Any improve?

    Did you try to use azure storage rest api? I suggest you could refer to use this in your client side (http://msdn.microsoft.com/en-us/library/windowsazure/dd179355.aspx ). Or you could create a WCF service as custom API for your project. you could set the insert / delete / update  and other operation.

    Please try it.

    Hope it helps.

    Regards,

    Will


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, February 3, 2014 10:13 AM
  • Hello Kevin,

    If you are looking for a file explorer then please take a look at 

    http://azurestorageexplorer.codeplex.com/

    as it may help you.

    Thanks,

    Vishal Supekar


    Regards, Vishal Supekar BE.CSE MCPDEA(Micorsoft Certified)

    Tuesday, February 4, 2014 6:29 AM