locked
Parallel.For RRS feed

  • Question

  • All,

    I'm trying to get my head around a Parallel.For and I'm confused about the exception that I'm getting. I get an exception indicating that the index was out of range in the following:

    Parallel.For(0, _ peopleList.Count, _ Sub(i As Integer) sb.AppendLine(String.Format("{0},{1},{2},{3},{4},{5},{6},{7}", _ peopleList(i).FirstName, _ peopleList(i).MiddleInitial, _ peopleList(i).LastName, _ peopleList(i).StreetAddress, _ peopleList(i).City, _ peopleList(i).State, _ peopleList(i).ZipCode, _ peopleList(i).EMail)) End Sub)


    Sorry for the horizontal scroll...

    I'm pretty sure that I've set it up right. The first parameter is the starting index, inclusive, the second parameter is the ending index, exclusive, and in this case, the third is a lambda expression.

    I am assuming that "index" its referring to is the iterator? If not then what else could it be?

    Just as a test, this one works fine:

    Dim number As Integer = 0
    
    Parallel.For(0, 101, Sub(i As Integer)
                               number = i
                             End Sub)

    Thanks :)


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Sunday, November 8, 2015 11:38 PM

Answers

  • I believe the issue is that StringBuilder.Append isn't thread-safe; the method refers to internal counter fields and may execute unsafe code.  The method can also cause the internal backing store of characters to expand in size so its possible for two threads to simultaneously begin trying to expand the collection.  You would need to capture a lock around the string builder to make it safe and then you'd loose all the threading performance.  Instead the results should be added to one of the concurrent collections.  When the overall process is complete you can then go through the collection and build up the string builder.

    Otherwise each parallel method needs its own string builder and then those all have to be combined in the end.


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"


    Monday, November 9, 2015 1:21 AM
  • I agree with Reed.

            Dim coll As New Concurrent.BlockingCollection(Of String)
    
            Parallel.For(0, peopleList.Count, Sub(i)
                                                  Dim s As String = String.Format("{0},{1},{2},{3},{4},{5},{6},{7}{8}", _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  Environment.NewLine)
                                                  coll.Add(s)
                                              End Sub)
    
            Dim sb As New System.Text.StringBuilder
    
            For Each ssb As String In coll
                sb.Append(ssb)
            Next

    Frank you do know that there is no guarantee that the list order will be maintained.  I suspect that the reason the others could not reproduce the problem is because of how Parallel partitions the work.


    "Those who use Application.DoEvents() have no idea what it does and those who know what it does never use it." - MSDN User JohnWein    Multics - An OS ahead of its time.




    • Edited by dbasnett Monday, November 9, 2015 12:37 PM
    • Marked as answer by Frank L. Smith Monday, November 9, 2015 12:41 PM
    Monday, November 9, 2015 12:20 PM

All replies

  • Try peopleList.Count - 1  ???
    Sunday, November 8, 2015 11:46 PM
  • Try peopleList.Count - 1  ???

    But that's not what it is.

    It's exclusive (sort of like the .Next method of the Random class).

    If you look at the second example, "number" ends up being 100, so I'm pretty sure that I'm right here.

    Thanks :)


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln


    Sunday, November 8, 2015 11:48 PM
  •  I don`t seem to be able to reproduce the error.  The code below seems to work fine on my end.  8)

    Imports System.Threading.Tasks
    
    Public Class Form1
        Private peopleList As New List(Of Person)
    
        Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
            For x As Integer = 0 To 5
                Dim p As New Person
                With p
                    .FirstName = "Name " & x.ToString
                    .MiddleInitial = "MI " & x.ToString
                    .LastName = "Last Name " & x.ToString
                    .StreetAddress = "SA " & x.ToString
                    .State = "State " & x.ToString
                    .City = "City " & x.ToString
                    .ZipCode = "zip " & x.ToString
                    .EMail = "email " & x.ToString
                End With
                peopleList.Add(p)
            Next
    
        End Sub
    
        Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
            Dim sb As New System.Text.StringBuilder
    
            Parallel.For(0, peopleList.Count, _
                         Sub(i As Integer)
                             sb.AppendLine(String.Format("{0},{1},{2},{3},{4},{5},{6},{7}", _
                                                         peopleList(i).FirstName, _
                                                         peopleList(i).MiddleInitial, _
                                                         peopleList(i).LastName, _
                                                         peopleList(i).StreetAddress, _
                                                         peopleList(i).City, _
                                                         peopleList(i).State, _
                                                         peopleList(i).ZipCode, _
                                                         peopleList(i).EMail))
                         End Sub)
    
            RichTextBox1.Text = sb.ToString
        End Sub
    End Class
    
    Public Class Person
        Public Property FirstName As String
        Public Property MiddleInitial As String
        Public Property LastName As String
        Public Property StreetAddress As String
        Public Property City As String
        Public Property State As String
        Public Property ZipCode As String
        Public Property EMail As String
    End Class
    


    If you say it can`t be done then i`ll try it

    Monday, November 9, 2015 12:24 AM
  • IR,

    That definitely makes me wonder now.

    I guess I'll look more into it tomorrow but that's essentially what I have, yea.

    Thanks


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Monday, November 9, 2015 12:28 AM
  • IR,

    That definitely makes me wonder now.

    I guess I'll look more into it tomorrow but that's essentially what I have, yea.

    Thanks


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln


     I`m playing around a little to see if i can reproduce the error but,  no luck so far.   8)

    If you say it can`t be done then i`ll try it

    Monday, November 9, 2015 12:32 AM
  • I`m playing around a little to see if i can reproduce the error but,  no luck so far.

    It's an enigma for sure.

    I just checked to see if my data itself might have been screwy (I set it to load up a different file), but not so apparently.


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Monday, November 9, 2015 12:35 AM
  • I even thought about this:

    "Maybe my collection is changing after the fact". That would do it, but no - it's using a method in a class that uses XElement.Load(url) ... which is a blocking call so once it returns, that data is there completely.

    I'll look more tomorrow but currently ... I'm stumped.


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Monday, November 9, 2015 1:04 AM
  • I even thought about this:

    "Maybe my collection is changing after the fact". That would do it, but no - it's using a method in a class that uses XElement.Load(url) ... which is a blocking call so once it returns, that data is there completely.

    I'll look more tomorrow but currently ... I'm stumped.


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln


     Yeah,  i was wondering about that.  I was wondering if it was being done in a separate thread or in some way that was not "Blocking" and the collection was still being modified when the parallel for was started.  I can`t imagine what it might be at this point.

    If you say it can`t be done then i`ll try it

    Monday, November 9, 2015 1:13 AM
  • I believe the issue is that StringBuilder.Append isn't thread-safe; the method refers to internal counter fields and may execute unsafe code.  The method can also cause the internal backing store of characters to expand in size so its possible for two threads to simultaneously begin trying to expand the collection.  You would need to capture a lock around the string builder to make it safe and then you'd loose all the threading performance.  Instead the results should be added to one of the concurrent collections.  When the overall process is complete you can then go through the collection and build up the string builder.

    Otherwise each parallel method needs its own string builder and then those all have to be combined in the end.


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"


    Monday, November 9, 2015 1:21 AM
  • I believe the issue is that StringBuilder.Append isn't thread-safe; the method refers to internal counter fields and may execute unsafe code.  The method can also cause the internal backing store of characters to expand in size so its possible for two threads to simultaneously begin trying to expand the collection.  You would need to capture a lock around the string builder to make it safe and then you'd loose all the threading performance.  Instead the results should be added to one of the concurrent collections.  When the overall process is complete you can then go through the collection and build up the string builder.

    Otherwise each parallel method needs its own string builder and then those all have to be combined in the end.


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"


    That definitely makes sense and I "sensed" that might be suspect, or maybe, but wasn't sure if I was looking for the right thing or not.

    I'll work on it tomorrow and report back.

    Thanks!


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Monday, November 9, 2015 1:24 AM
  • I agree with Reed.

            Dim coll As New Concurrent.BlockingCollection(Of String)
    
            Parallel.For(0, peopleList.Count, Sub(i)
                                                  Dim s As String = String.Format("{0},{1},{2},{3},{4},{5},{6},{7}{8}", _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  peopleList(i), _
                                                                                  Environment.NewLine)
                                                  coll.Add(s)
                                              End Sub)
    
            Dim sb As New System.Text.StringBuilder
    
            For Each ssb As String In coll
                sb.Append(ssb)
            Next

    Frank you do know that there is no guarantee that the list order will be maintained.  I suspect that the reason the others could not reproduce the problem is because of how Parallel partitions the work.


    "Those who use Application.DoEvents() have no idea what it does and those who know what it does never use it." - MSDN User JohnWein    Multics - An OS ahead of its time.




    • Edited by dbasnett Monday, November 9, 2015 12:37 PM
    • Marked as answer by Frank L. Smith Monday, November 9, 2015 12:41 PM
    Monday, November 9, 2015 12:20 PM
  • Frank you do know that there is no guarantee that the list order will be maintained.

    I didn't think about it, but I can see how you're probably right - and that would include PLINQ also.

    Thanks. :)


    If I had eight hours to chop down a tree, I'd spend six sharpening my axe. -- Abraham Lincoln

    Monday, November 9, 2015 12:41 PM