Copy a folder structure across the world over a WAN

Question

Hi all,

I need to replicate a folder structure on a different domain over a WAN. 

I currently have a .NET Windows Service that copies folders from one folder to another (in the same domain) that maintains the permissions on the files and folders in each.  This is done by using a Process to launch xcopy to do it.  Something as follows:

Dim params As String = """{0}"" /e /O /K /H ""{1}""" & rootFolder & """"
Dim copyParams As String = String.Format(params, masterFolder, newFolder)
Dim psi As New ProcessStartInfo("xcopy", copyParams)

psi.CreateNoWindow = True
psi.WindowStyle = ProcessWindowStyle.Hidden

Dim p As System.Diagnostics.Process
p = System.Diagnostics.Process.Start(psi)

p.WaitForExit()

The service runs on our domain here in the UK.  Now along come the Australia office to integrate...

When I look at the folders of the other domain (Australia) in Windows Explorer, it takes an age to view the folder list.  I've copied the folders to my local machine also in Windows Explorer and it can take up to 3 minutes to copy 1.28MB.  Yes 1.28MB.  This is on Windows XP SP2.  The file shares in oz have been created with Windows DFS I do believe.

Now if it takes 2 or 3 minutes to copy folders to my local machine, it will take longer to copy them back to another folder in Australia.  I am about to do some performance tests on this using the above method when it is available.

But if it is going to take this long to copy then it is going to kill performance in the application.  There will be a number of changes made to the system that will required the folder structure to be replicated for each item.

I don't know if there are any quicker or better ways to do this or any other alternatives?

Andez

---

Andez

Answer 1

Calling XCOPY forces the need for a mapped drive.  Mapped drives delve into the world of NETBIOS.  NETBIOS is ancient and Slow.....
Use FTP... It's fast and efficient.  If your network is secure within the DMZ you don't need to consider FTPs, but if you need secure file transfer you will need to look into another FTPS solution because MS doesn't provide it out of the box.

You could write your own File transfer utility but it would be a waste of time. 

Other options:  How about using open source file transfer engines like BIT Torrent.

Answer 2

Not done too much on Secure FTP, but I don't know if that would do what we need but I'll check it.  The main problem is to maintain the user permissions on the folders and files we copy.  Only thing I've used or seen that does it is using good old Windows API's and commands.

I've seen one or two bits on WMI but never tried it with that so gonna look into that.

There was a nice article in VSJ this month (Jul/Aug 2009) about network impact that got me thinking about this.  And on the same day we get a request to integrate the application with Australia.

---

Andez

Answer 3

Try this then:

static void Main(string[] args)
    {
        DirectoryInfo sourceDir = new DirectoryInfo("c:\\a");
        DirectoryInfo destinationDir = new DirectoryInfo("c:\\b");

        CopyDirectory(sourceDir, destinationDir);

    }

    static void CopyDirectory(DirectoryInfo source, DirectoryInfo destination)
    {
        if (!destination.Exists)
        {
            destination.Create();
        }

        // Copy all files.
        FileInfo[] files = source.GetFiles();
        foreach (FileInfo file in files)
        {
            file.CopyTo(Path.Combine(destination.FullName,
                file.Name));
        }

        // Process subdirectories.
        DirectoryInfo[] dirs = source.GetDirectories();
        foreach (DirectoryInfo dir in dirs)
        {
            // Get destination directory.
            string destinationDir = Path.Combine(destination.FullName, dir.Name);

            // Call CopyDirectory() recursively.
            CopyDirectory(dir, new DirectoryInfo(destinationDir));
        }
    }

But remember this creates the need for UNC paths, not the fastest....

Answer 4

This helps me a lot.

---

Samarendra Swain Team Sharepoint www.manuhsolutions.com