none
How to copy multiple files using task RRS feed

  • Question

  • instead of copying file one by one how could i copy files simultaneously by using task library. looking for code example.
    Monday, April 25, 2016 9:26 AM

Answers

  • Here's a quick sample

                DateTime EndDate = DateTime.Now;
                var dest = @"c:\temp";
                var files = Enumerable.Range(0, 8).Select(value => $"Z:\report_{EndDate.Date.AddDays(value).ToString("dd-MM-yyyy")}.csv").ToList();
    
                var opts = new ParallelOptions() { MaxDegreeOfParallelism = 8 };
                Parallel.ForEach(files,opts, file =>
                {
                    System.IO.File.Copy(file, dest);
                });
    

    >give me few scenario of CPU bound task or job?

    For instance, if you were Encrypting, Encoding, or Compressing the files, instead of just copying them you might become CPU-bound.

    David


    David http://blogs.msdn.com/b/dbrowne/

    • Marked as answer by Mou_kolkata Monday, April 25, 2016 5:52 PM
    Monday, April 25, 2016 4:07 PM
  • List<string> files = new List<string>();
    
    //files = GetFiles();
    
    // approach 1
    files.AsParallel().ForAll(file =>
    {
    	//copy file to dest here;
    });
    
    // approach 2
    Parallel.ForEach(files, file =>
    {
    	//copy file to dest here;
    });
    you can use parallel task to do copy files simultaneously as multi task/thread.
    • Proposed as answer by YuGUANMVP Monday, April 25, 2016 4:10 PM
    • Marked as answer by Mou_kolkata Monday, April 25, 2016 5:52 PM
    Monday, April 25, 2016 4:10 PM
  • Race conditions require multiple pieces of code working on the same data. For there to be one, the same source file would have to be represented multiple times in the list.

    Modern filesystem are de-facto transactional Database engines. Mutual exclusion of access is among the first things those enforce.
    You need to go out of your way (write access with read sharing) to even have a chance to run into a race condition with those.

    If you are worried about race conditions the simple way is to not do multitasking in the first place. As I alrady said, disk operations will not benefit from Multithreading or -tasking. You can order the reads and writes however fast you want. The disk will still only execute them at it's speed.

    • Marked as answer by Mou_kolkata Tuesday, April 26, 2016 6:14 PM
    Monday, April 25, 2016 7:19 PM

All replies

  • Multitasking has purpose for preventing the GUI from locking up during a long opeartion.

    Multithreading however only makes sense if the operation is actually CPU bound. You first need to check if your problem is even multithreadable, otherwise you end up with code that is slower, more memory demanding and needlessly complex - all at the same time:
    Primal issues - Multithreading, lists in memory and checking for prime number

    This sounds like a Disk bound operation from first to last word.
    There might be a minor case to run the copy process for each target physical disk in paralell (if you got a multi target copy). But that is hardly worht the effort either.

    Monday, April 25, 2016 1:05 PM
  • suppose my files name with path store in list and i want to use task to copy all those file stored in list from destination folder to target folder.

    List<string> filename = new List<string>(Enumerable.Range(0,8).
    Select(value=>"Z:\report_"+EndDate.Date.AddDays(value).ToString("dd-MM-yyyy")+".csv"));

    so filename variable has all files info which i have to copy. now tell me how i can use task to copy all files and also there will be one call back function which tell me when all files copying done. looking for fresh code.

    why do u think if use task to copy multiple files simultaneously then process will be slower?

    are your trying to say Disk bound operation never handle with multi threading?

    give me few scenario of CPU bound task or job?

    thanks

    Monday, April 25, 2016 3:58 PM
  • Here's a quick sample

                DateTime EndDate = DateTime.Now;
                var dest = @"c:\temp";
                var files = Enumerable.Range(0, 8).Select(value => $"Z:\report_{EndDate.Date.AddDays(value).ToString("dd-MM-yyyy")}.csv").ToList();
    
                var opts = new ParallelOptions() { MaxDegreeOfParallelism = 8 };
                Parallel.ForEach(files,opts, file =>
                {
                    System.IO.File.Copy(file, dest);
                });
    

    >give me few scenario of CPU bound task or job?

    For instance, if you were Encrypting, Encoding, or Compressing the files, instead of just copying them you might become CPU-bound.

    David


    David http://blogs.msdn.com/b/dbrowne/

    • Marked as answer by Mou_kolkata Monday, April 25, 2016 5:52 PM
    Monday, April 25, 2016 4:07 PM
  • List<string> files = new List<string>();
    
    //files = GetFiles();
    
    // approach 1
    files.AsParallel().ForAll(file =>
    {
    	//copy file to dest here;
    });
    
    // approach 2
    Parallel.ForEach(files, file =>
    {
    	//copy file to dest here;
    });
    you can use parallel task to do copy files simultaneously as multi task/thread.
    • Proposed as answer by YuGUANMVP Monday, April 25, 2016 4:10 PM
    • Marked as answer by Mou_kolkata Monday, April 25, 2016 5:52 PM
    Monday, April 25, 2016 4:10 PM
  • @YuGUAN i am not sure but i have doubt that if i use Parallel.ForEach to copy multiple files then may be race condition may occur but nothing found in your code some thing which avoid race condition. please share your suggestion. thanks

    Monday, April 25, 2016 5:52 PM
  • @David
    i am not sure but i have doubt that if i use Parallel.ForEach to copy multiple files then may be race condition may occur but nothing found in your code some thing which avoid race condition. please share your suggestion. thanks
    Monday, April 25, 2016 5:52 PM
  • Race conditions require multiple pieces of code working on the same data. For there to be one, the same source file would have to be represented multiple times in the list.

    Modern filesystem are de-facto transactional Database engines. Mutual exclusion of access is among the first things those enforce.
    You need to go out of your way (write access with read sharing) to even have a chance to run into a race condition with those.

    If you are worried about race conditions the simple way is to not do multitasking in the first place. As I alrady said, disk operations will not benefit from Multithreading or -tasking. You can order the reads and writes however fast you want. The disk will still only execute them at it's speed.

    • Marked as answer by Mou_kolkata Tuesday, April 26, 2016 6:14 PM
    Monday, April 25, 2016 7:19 PM
  • there is no race condition, feel safe to use it, Parallel.ForEach will not select one item for more than once.
    Friday, April 29, 2016 4:48 AM