none
Convert to Parallel ForEach loop VB.Net RRS feed

  • Question

  • How do I convert the following loop into a Parallel.ForEach loop in VB.Net?

     Sub Main()
            For Each station In EnumToList(Of StationEnum)()
                Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & station.ToString() & ".dly"
                Dim ftpUri As New Uri(ftpFile)
                Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & station.ToString() & ".dly"

                DownloadFileFromServer(ftpUri, myfile)
            Next
        End Sub

    I'm using an Enum to list my weather stations.

    Appreciate any help.


    Marilyn Gambone

    Thursday, April 5, 2018 1:30 PM

Answers

  • Another approach using Async Tasks

        Sub Main()
            DoDownloads()
            Console.ReadLine()
        End Sub
    
        Private Async Sub DoDownloads()
            Dim dwnldTasks As New List(Of Task)
    
            Dim watch As Stopwatch = Stopwatch.StartNew()
            Dim stations() As String = {"path1", "path2", "path3", "path4"}
    
            For Each station As String In stations
                Dim t As Task
                t = Task.Factory.StartNew(Sub(aPath)
                                              Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & aPath.ToString & ".dly"
                                              Dim ftpUri As New Uri(ftpFile)
                                              Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & aPath.ToString & ".dly"
    
                                              DownloadFileFromServer(ftpUri, myfile)
    
                                          End Sub, station)
                dwnldTasks.Add(t)
            Next
    
            Await Task.WhenAll(dwnldTasks.ToArray)
    
            Console.WriteLine("Total time (ms): {0}", watch.ElapsedMilliseconds)
        End Sub
    
    


    "Those who use Application.DoEvents() have no idea what it does and those who know what it does never use it."

    - from former MSDN User JohnWein

    SerialPort Info

    Multics - An OS ahead of its time.

    • Marked as answer by deskcheck1 Friday, April 6, 2018 12:43 PM
    Thursday, April 5, 2018 3:28 PM
  • Hi y'all,

    Never mind.  Found out how to do it.  Changed my methods to "Shared":

    Shared Sub Main()
            Dim watch = Stopwatch.StartNew()
            Dim stations As New List(Of String)
            Dim i As Integer

            For Each station In EnumToList(Of StationEnum)()
                stations.Add(station.ToString())
            Next

            Parallel.ForEach(Of String)(stations, Sub(station)
                                                      Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & station.ToString() & ".dly"
                                                      Dim ftpUri As New Uri(ftpFile)
                                                      Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & station.ToString() & ".dly"

                                                      DownloadFile(ftpUri, myfile)
                                                      i = i + 1
                                                      Console.WriteLine("Downloaded {0}", i)
                                                  End Sub)

            watch.Stop()
            Console.WriteLine("Total time (ms): {0}", watch.ElapsedMilliseconds)   '15023, 6 files
            Console.ReadLine()
        End Sub


    Marilyn Gambone

    • Marked as answer by deskcheck1 Thursday, April 5, 2018 2:21 PM
    Thursday, April 5, 2018 2:20 PM

All replies

  • Marilyn,

    I don't know what a parallel.ForEach loop is. 

    But what I know is that an enum is not a collection. It is a kind of program extension to make your program more readable. 

    However, you can solve your problem with this kind of code. 

        Sub Main()
            Dim Stations As New List(Of String)
            Stations.Add("one")
            Stations.Add("two")
            For i = 0 To Stations.Count - 1
                Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & Stations(i).ToString() & ".dly"
                Dim ftpUri As New Uri(ftpFile)
                Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & Stations(i) & ".dly"
                DownloadFileFromServer(ftpUri, myfile)
            Next
        End Sub


    Success
    Cor

    Thursday, April 5, 2018 1:53 PM
  • Hi y'all,

    Never mind.  Found out how to do it.  Changed my methods to "Shared":

    Shared Sub Main()
            Dim watch = Stopwatch.StartNew()
            Dim stations As New List(Of String)
            Dim i As Integer

            For Each station In EnumToList(Of StationEnum)()
                stations.Add(station.ToString())
            Next

            Parallel.ForEach(Of String)(stations, Sub(station)
                                                      Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & station.ToString() & ".dly"
                                                      Dim ftpUri As New Uri(ftpFile)
                                                      Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & station.ToString() & ".dly"

                                                      DownloadFile(ftpUri, myfile)
                                                      i = i + 1
                                                      Console.WriteLine("Downloaded {0}", i)
                                                  End Sub)

            watch.Stop()
            Console.WriteLine("Total time (ms): {0}", watch.ElapsedMilliseconds)   '15023, 6 files
            Console.ReadLine()
        End Sub


    Marilyn Gambone

    • Marked as answer by deskcheck1 Thursday, April 5, 2018 2:21 PM
    Thursday, April 5, 2018 2:20 PM
  • Hi y'all,

    Never mind.  Found out how to do it.  Changed my methods to "Shared":

    Shared Sub Main()
            Dim watch = Stopwatch.StartNew()
            Dim stations As New List(Of String)
            Dim i As Integer

            For Each station In EnumToList(Of StationEnum)()
                stations.Add(station.ToString())
            Next

            Parallel.ForEach(Of String)(stations, Sub(station)
                                                      Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & station.ToString() & ".dly"
                                                      Dim ftpUri As New Uri(ftpFile)
                                                      Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & station.ToString() & ".dly"

                                                      DownloadFile(ftpUri, myfile)
                                                      i = i + 1
                                                      Console.WriteLine("Downloaded {0}", i)
                                                  End Sub)

            watch.Stop()
            Console.WriteLine("Total time (ms): {0}", watch.ElapsedMilliseconds)   '15023, 6 files
            Console.ReadLine()
        End Sub


    Marilyn Gambone

    Be aware that shared does nothing, a module is always shared. 

    But she copied a lot from my example and marked that as here answer. I don't know where she lives, but in my culture they call that plagiate. 


    Success
    Cor

    Thursday, April 5, 2018 3:11 PM
  • Another approach using Async Tasks

        Sub Main()
            DoDownloads()
            Console.ReadLine()
        End Sub
    
        Private Async Sub DoDownloads()
            Dim dwnldTasks As New List(Of Task)
    
            Dim watch As Stopwatch = Stopwatch.StartNew()
            Dim stations() As String = {"path1", "path2", "path3", "path4"}
    
            For Each station As String In stations
                Dim t As Task
                t = Task.Factory.StartNew(Sub(aPath)
                                              Dim ftpFile As String = "ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/" & aPath.ToString & ".dly"
                                              Dim ftpUri As New Uri(ftpFile)
                                              Dim myfile As String = "D:\ProjectsWorkspaces\Weather 2018\from web\" & aPath.ToString & ".dly"
    
                                              DownloadFileFromServer(ftpUri, myfile)
    
                                          End Sub, station)
                dwnldTasks.Add(t)
            Next
    
            Await Task.WhenAll(dwnldTasks.ToArray)
    
            Console.WriteLine("Total time (ms): {0}", watch.ElapsedMilliseconds)
        End Sub
    
    


    "Those who use Application.DoEvents() have no idea what it does and those who know what it does never use it."

    - from former MSDN User JohnWein

    SerialPort Info

    Multics - An OS ahead of its time.

    • Marked as answer by deskcheck1 Friday, April 6, 2018 12:43 PM
    Thursday, April 5, 2018 3:28 PM
  • To Cor:  Excuse Me?  What do you mean "plagiate".  I don't know who you are but my source is this:  https://msdn.microsoft.com/en-us/library/ff963552.asp

    You're saying one can't come up with codes based on MSDN website.  I'd be careful with what you say.  Very hurtful.

    Also, I get an error "Methods in a Module cannot be declared 'Shared'" if I use a Module.


    Marilyn Gambone



    • Edited by deskcheck1 Friday, April 6, 2018 12:25 PM
    Friday, April 6, 2018 12:10 PM
  • Hi,

    Thank you.  I compared the results between Asynchronous and Parallel methods and the difference is very minimal.  To download 6 files using Asynchronous method takes 3552 milliseconds.  Using Parallel method takes 3190 milliseconds.

    Thank you for this.  I'll be using this for some of my work.


    Marilyn Gambone

    Friday, April 6, 2018 12:43 PM