none
AZCopy to upload PST

    Question

  • We do not have (or want) an Azure subscription. We are testing migration from on-prem exchange to O365 using the Office 365 import service. I am using the AZCopy tool to upload a single PST file (1.3 GB). I downloaded and installed the tool got the key and URL and ran the command to upload the PST file as described at https://technet.microsoft.com/en-us/library/ms.o365.cc.ingestionhelp.aspx#BKMK_NetworkUpload

    How can I monitor progress of the upload. I can't tell if it is even working. All the log file says is:

    Start transfer: johnd.pst => johnd.pst

    How can I tell when (or if) it finishes successfully? It has been about 6 hours now.

    Tuesday, September 15, 2015 9:07 PM

Answers

  • We found the issue. If we performed the transfer from an Azure virtual machine, the transfer of a 1.3 GB PST file worked great and only took 2 minutes, but the transfer continually failed from any on-premise machine in any physical location. We discovered that the transfer was failing because there were too many concurrent threads. I used the /NC parameter in the AZCopy command line to limit the number of threads to 3 and that allowed the transfer to occur - albeit at a much slower rate. The same file took 1 hour and 7 minutes to transfer with the /NC parameter set to 3. I can't find any information on how many concurrent threads are started by default. That would be nice to know.
    Tuesday, September 22, 2015 2:31 PM

All replies

  • It may related to file size or number of files and your network, you could also try re-startable mod.

    Check this earlier discussed thread - Azcopy: The client could not finish the operation within specified timeout: https://social.technet.microsoft.com/Forums/en-US/d45f0af0-2d57-46a0-a4c2-617fdb71b9cd/azcopy-the-client-could-not-finish-the-operation-within-specified-timeout?forum=windowsazuredata

    You may also explore the new PST Capture Tool from Microsoft, makes it easier to search and discover PST files in the network and to import them back to an on-premise Exchange environment, to BPOS or to Office 365.

    If you wish you may also take help from this Import/Export solution which lets you to Import and Export email items between multiple source.

    Hope it helps you!

    Wednesday, September 16, 2015 6:36 AM
  • If you run AzCopy in an command window, and not redirect the output, you should see the transfer progress update in the command window like "Finished: *** file(s), transfer speed: ***".

    If the transder finish, you can see "Finished *** of total *** file(s)." and trasnfer summary.

    Could you share the following information, then we can see if any issue:

    1. AzCopy command line and output (please hide credential part)

    2. Test account name and time

    3. Running enviroment (CPU, network)

    BTW, it's user's responds to keep the source file unchanged when Azcopy is running. So please don't do any change to the PST file when uploading it with AzCopy.

    Thanks!



    Tuesday, September 22, 2015 2:00 AM
  • We found the issue. If we performed the transfer from an Azure virtual machine, the transfer of a 1.3 GB PST file worked great and only took 2 minutes, but the transfer continually failed from any on-premise machine in any physical location. We discovered that the transfer was failing because there were too many concurrent threads. I used the /NC parameter in the AZCopy command line to limit the number of threads to 3 and that allowed the transfer to occur - albeit at a much slower rate. The same file took 1 hour and 7 minutes to transfer with the /NC parameter set to 3. I can't find any information on how many concurrent threads are started by default. That would be nice to know.
    Tuesday, September 22, 2015 2:31 PM
  • The default thread count is 8*core count, such on a machine with 8 CPU cores, the thread count would be 8*8=64.

    BTW, on machine with slow network, but many cores, we would suggest to use a smaller thread count with /NC:. Anyway, this will make the transfer slower.

    Wednesday, September 23, 2015 1:18 AM