locked
speed difference - adding results to listview compared to datagridview - why? RRS feed

  • Question

  • Hi

    I have the two following bits of code, both query the event log using LINQ in exactly the same way, the first shows the results in a listview, the second in a datagridview - other than the output the code is the same

    So why does the listview version take 204 seconds to finish, when the datagridview only takes 3 seconds?

    What am I missing? Really confused as to the difference in time taken

    First the list view version:-

    Dim EventLog1 As New EventLog("Security")
    
            Dim query = From logEntry In EventLog1.Entries
                        Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                        Where logEntry.Message.Contains("Darren")
                        Order By logEntry.TimeWritten Descending
    
    
            Dim ListViewResults As ListViewItem
            Dim array(6) As String
    
            ListView1.View = View.Details
            ListView1.Columns.Add("Date/Time", 120)
            ListView1.Columns.Add("ID", 50)
            ListView1.Columns.Add("Source", 150)
            ListView1.Columns.Add("Type", 100)
            ListView1.Columns.Add("Machine", 100)
            ListView1.Columns.Add("Message", 1000)
    
            For x = 0 To query.Count - 1
                array(0) = (query(x).TimeGenerated.ToString)
                array(1) = (query(x).InstanceId And &HFFFF)
                array(2) = (query(x).Source.ToString)
                array(3) = (query(x).EntryType.ToString)
                array(4) = (query(x).MachineName.ToString)
                array(5) = (query(x).Message)
                ListViewResults = New ListViewItem(array)
                ListView1.Items.Add(ListViewResults)
                ListView1.Refresh()
            Next

    And then the datagridview version:-

     Dim EventLog1 As New EventLog("Security")
    
                   Dim query = From logEntry In EventLog1.Entries.AsParallel
                        Where logEntry.InstanceID = 4624
                        Where logEntry.Message.Contains("Darren")
                        Order By logEntry.TimeGenerated Descending
    
    
               With DataGridView1
                    .AutoGenerateColumns = True
                    .DataSource = query.ToList
                    .Columns.Remove("Data")
                    .Columns.Remove("Category")
                    .Columns.Remove("CategoryNumber")
                    .Columns.Remove("Index")
                End With
    
    


    Darren Rose

    Saturday, July 22, 2017 10:48 PM

Answers

  • LINQ uses deferred execution, so in your code for the ListrView:

    array(0) = (query(x).TimeGenerated.ToString)
    array(1) = (query(x).InstanceId And &HFFFF).ToString
    array(2) = (query(x).Source.ToString)
    array(3) = (query(x).EntryType.ToString)
    array(4) = (query(x).MachineName.ToString)
    array(5) = (query(x).Message)

    you are executing the query 6 times.... and then you do that in a loop 'query.count' times.

    Your DataGridView code, however, only executes the query the one time:

    With DataGridView1
        .AutoGenerateColumns = True
        .DataSource = query.ToList

    To speed up the ListView code, try changing the query to:

    Dim query = (From logEntry In EventLog1.Entries
                 Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                 Where logEntry.Message.Contains("Darren")
                 Order By logEntry.TimeWritten Descending).ToArray

    The call to '.ToArray' will force the execution of the query the one time, and return an array, not an IEnumerable.


    • Edited by S P C Sunday, July 23, 2017 12:18 AM formatting
    • Proposed as answer by Frank L. Smith Sunday, July 23, 2017 12:35 AM
    • Marked as answer by wingers Sunday, July 23, 2017 11:41 AM
    Sunday, July 23, 2017 12:17 AM

All replies

  • Underlying source code in a DataGridView is more efficient then adding items to a ListView one by one.

    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.
    VB Forums - moderator
    profile for Karen Payne on Stack Exchange, a network of free, community-driven Q&A sites

    Saturday, July 22, 2017 10:54 PM
  • Hi,

    use the BeginUpdate() and EndUpdate() methods of the ListView.

    Regards,

      Thorsten

    Saturday, July 22, 2017 10:54 PM
  • Hi,

    use the BeginUpdate() and EndUpdate() methods of the ListView.

    Regards,

      Thorsten

    Makes no difference, just as slow, just can't see anything in listview until completely updated

    Darren Rose

    Saturday, July 22, 2017 11:39 PM
  • Underlying source code in a DataGridView is more efficient then adding items to a ListView one by one.

    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.
    VB Forums - moderator
    profile for Karen Payne on Stack Exchange, a network of free, community-driven Q&A sites

    And it would cause that much difference? there are only 14 items to add in this case, so speed difference is ludicrous

    Have used listview for many other things and never noticed problem like this before

    Must be some other reason in my code?


    Darren Rose

    Saturday, July 22, 2017 11:40 PM
  • Hi

    I have the two following bits of code, both query the event log using LINQ in exactly the same way, the first shows the results in a listview, the second in a datagridview - other than the output the code is the same

    So why does the listview version take 204 seconds to finish, when the datagridview only takes 3 seconds?

    What am I missing? Really confused as to the difference in time taken

    First the list view version:-

    Dim EventLog1 As New EventLog("Security")
    
            Dim query = From logEntry In EventLog1.Entries
                        Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                        Where logEntry.Message.Contains("Darren")
                        Order By logEntry.TimeWritten Descending
    
    
            Dim ListViewResults As ListViewItem
            Dim array(6) As String
    
            ListView1.View = View.Details
            ListView1.Columns.Add("Date/Time", 120)
            ListView1.Columns.Add("ID", 50)
            ListView1.Columns.Add("Source", 150)
            ListView1.Columns.Add("Type", 100)
            ListView1.Columns.Add("Machine", 100)
            ListView1.Columns.Add("Message", 1000)
    
            For x = 0 To query.Count - 1
                array(0) = (query(x).TimeGenerated.ToString)
                array(1) = (query(x).InstanceId And &HFFFF)
                array(2) = (query(x).Source.ToString)
                array(3) = (query(x).EntryType.ToString)
                array(4) = (query(x).MachineName.ToString)
                array(5) = (query(x).Message)
                ListViewResults = New ListViewItem(array)
                ListView1.Items.Add(ListViewResults)
                ListView1.Refresh()
            Next

    And then the datagridview version:-

     Dim EventLog1 As New EventLog("Security")
    
                   Dim query = From logEntry In EventLog1.Entries.AsParallel
                        Where logEntry.InstanceID = 4624
                        Where logEntry.Message.Contains("Darren")
                        Order By logEntry.TimeGenerated Descending
    
    
               With DataGridView1
                    .AutoGenerateColumns = True
                    .DataSource = query.ToList
                    .Columns.Remove("Data")
                    .Columns.Remove("Category")
                    .Columns.Remove("CategoryNumber")
                    .Columns.Remove("Index")
                End With
    


    Darren Rose

    In the second case you're using parallelization - you can't then compare the apple to the orange.

    ...or is that not your question?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 12:09 AM
  • LINQ uses deferred execution, so in your code for the ListrView:

    array(0) = (query(x).TimeGenerated.ToString)
    array(1) = (query(x).InstanceId And &HFFFF).ToString
    array(2) = (query(x).Source.ToString)
    array(3) = (query(x).EntryType.ToString)
    array(4) = (query(x).MachineName.ToString)
    array(5) = (query(x).Message)

    you are executing the query 6 times.... and then you do that in a loop 'query.count' times.

    Your DataGridView code, however, only executes the query the one time:

    With DataGridView1
        .AutoGenerateColumns = True
        .DataSource = query.ToList

    To speed up the ListView code, try changing the query to:

    Dim query = (From logEntry In EventLog1.Entries
                 Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                 Where logEntry.Message.Contains("Darren")
                 Order By logEntry.TimeWritten Descending).ToArray

    The call to '.ToArray' will force the execution of the query the one time, and return an array, not an IEnumerable.


    • Edited by S P C Sunday, July 23, 2017 12:18 AM formatting
    • Proposed as answer by Frank L. Smith Sunday, July 23, 2017 12:35 AM
    • Marked as answer by wingers Sunday, July 23, 2017 11:41 AM
    Sunday, July 23, 2017 12:17 AM
  • In the second case you're using parallelization - you can't then compare the apple to the orange.

    ...or is that not your question?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Hi Frank

    Ignore the AsParallel bit , even if I remove that (or) include it in both there is still a huge difference in the time it takes to do listview version compared to datagridview version


    Darren Rose

    Sunday, July 23, 2017 12:26 AM
  • LINQ uses deferred execution, so in your code for the ListrView:

    array(0) = (query(x).TimeGenerated.ToString)
    array(1) = (query(x).InstanceId And &HFFFF).ToString
    array(2) = (query(x).Source.ToString)
    array(3) = (query(x).EntryType.ToString)
    array(4) = (query(x).MachineName.ToString)
    array(5) = (query(x).Message)

    you are executing the query 6 times.... and then you do that in a loop 'query.count' times.

    Your DataGridView code, however, only executes the query the one time:

    With DataGridView1
        .AutoGenerateColumns = True
        .DataSource = query.ToList

    To speed up the ListView code, try changing the query to:

    Dim query = (From logEntry In EventLog1.Entries
                 Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                 Where logEntry.Message.Contains("Darren")
                 Order By logEntry.TimeWritten Descending).ToArray

    The call to '.ToArray' will force the execution of the query the one time, and return an array, not an IEnumerable.



    Thank you S P C - that has made it just as quick for both versions - didn't realise that so appreciate the explanation :)

    Darren Rose

    Sunday, July 23, 2017 12:27 AM
  • Darren,

    Don't rely on on the compiler for ambiguous things when using LINQ.

    Option Strict On
    Option Explict On

    -- and yes this one too:

    Option Infer OFF


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 12:33 AM


  • Thank you S P C - that has made it just as quick for both versions - didn't realise that so appreciate the explanation :)

    Darren Rose

    Good. :)

    LINQ is really just an instruction; not a command.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 12:35 AM
  • Darren,

    Don't rely on on the compiler for ambiguous things when using LINQ.

    Option Strict On
    Option Explicit On

    -- and yes this one too:

    Option Infer OFF


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Frank, yes I am bad for not turning those on, I must remember to start doing it more

    So to help me understand if you will

    Once I turn all three of those on I get a lot of errors in my code, none that immediately would have alerted me to above problem/solution though that I can see

    But unsure how my code should read to avoid errors

    e.g.

     Dim query = (From logEntry In EventLog1.Entries.AsParallel
                         Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                         Where logEntry.Message.Contains("Darren")
                         Order By logEntry.TimeWritten Descending).ToArray

    Dim query  - needs to have As declaration - but As what? object? as that causes further errors

    logEntry.InstanceId and logEntry.Message and logEntry.TimeWritten - these all give late binding errors

     For x As Integer = 0 To query.Count - 1
    
                array(0) = (query(x).TimeGenerated.ToString)
                array(1) = (query(x).InstanceId And &HFFFF)
                array(2) = (query(x).Source.ToString)
                array(3) = (query(x).EntryType.ToString)
                array(4) = (query(x).MachineName.ToString)
                array(5) = (query(x).Message)
                ListViewResults = New ListViewItem(array)
                Update_ListView1(ListViewResults)
    
    
            Next

    query.count and query(x) - again all give late binding errors

    How should this be done?


    Darren Rose

    Sunday, July 23, 2017 11:53 AM
  • Darren,

    Bear with me about an hour or so please.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 12:27 PM
  • If you're stuck with "As Object" then there's more work to do.

    Do you remember around the end of last year we talked a fair amount about LINQ (specifically LINQ-To-Object)? It will return some form of a generic Enumerable, remember?

    *****

    Let's use an example where we'll both have the same data. How about have a look at this please:

    http://www.fls-online.net/VBNet_Forum/SampleData/

    There's a .csv file that I'd like to get you to download to your desktop. I know it's large, but this will make the point about deferred execution once you run this example:

    Option Strict On
    Option Explicit On
    Option Infer Off
    
    Public Class Form1
        Private _desktop As String = _
            Environment.GetFolderPath(Environment.SpecialFolder.Desktop)
    
        Private _people As List(Of PersonData)
    
        Private Sub _
            Form1_Load(sender As System.Object, _
                       e As System.EventArgs) _
                       Handles MyBase.Load
    
            _people = New List(Of PersonData)
    
            Dim filePath As String = _
                IO.Path.Combine(_desktop, "Sample_Data_1000000.csv")
    
            GetCSVData(filePath)
    
            Stop
    
            Dim qry As IEnumerable(Of PersonData) = _
                From pd As PersonData In _people _
                    Where CInt(pd.ZipCode) = 11510
    
            Stop
    
            Dim i As Integer = qry.Count
    
            Stop
    
        End Sub
    
    
    
        Private Sub GetCSVData(ByVal filePath As String)
    
            If Not String.IsNullOrWhiteSpace(filePath) Then
    
                Try
                    Dim fi As New IO.FileInfo(filePath)
    
                    If fi.Exists Then
                        _people.Clear()
    
                        Using tfp As New Microsoft.VisualBasic.FileIO.TextFieldParser(fi.FullName)
                            With tfp
                                .TextFieldType = FileIO.FieldType.Delimited
                                .Delimiters = New String() {","}
    
                                ' Set the following to true if you have fields
                                ' which are enclosed in quotation marks:
                                .HasFieldsEnclosedInQuotes = True
                            End With
    
                            Dim currentLineOfText() As String
    
                            While Not tfp.EndOfData
                                currentLineOfText = tfp.ReadFields()
                                _people.Add(New PersonData _
                                            With {.FirstName = currentLineOfText(0), _
                                                  .MiddleInitial = currentLineOfText(1), _
                                                  .LastName = currentLineOfText(2), _
                                                  .StreetAddress = currentLineOfText(3), _
                                                  .CityName = currentLineOfText(4), _
                                                  .StateName = currentLineOfText(5), _
                                                  .ZipCode = currentLineOfText(6), _
                                                  .Email = currentLineOfText(7)})
                            End While
                        End Using
                    End If
    
                Catch ex As Exception
                    MessageBox.Show(String.Format("An error occurred:{0}{0}{1}", _
                                                  vbCrLf, ex.Message), _
                                                  "Error Reading Text File", _
                                                  MessageBoxButtons.OK, _
                                                  MessageBoxIcon.Warning)
                End Try
            End If
    
        End Sub
    
    
    
    
        Private Class PersonData
            Public Property FirstName As String
            Public Property MiddleInitial As String
            Public Property LastName As String
            Public Property StreetAddress As String
            Public Property CityName As String
            Public Property StateName As String
            Public Property ZipCode As String
            Public Property Email As String
        End Class
    End Class

    I have "Stop" in there three times, intentionally.

    The first one is after it reads the file and populates a List(Of Class). That will take about a minute so be patient. Once it's done though, continue and watch how quickly it runs that query!

    ... wait ... it can't do it that fast..

    Now continue again and it's noticeably slower. Why?

    Deferred execution.

    *****

    Late binding will kill performance. The difference is whether the 'binding' is done during compile time or during run time. When you force it to only know it's an "object", it has no choice but to do it during run time:

    https://docs.microsoft.com/en-us/dotnet/visual-basic/programming-guide/language-features/early-late-binding/

    From that article, also note this:

    "Early bound objects allow the compiler to allocate memory and perform other optimizations before an application executes."

    Does that help some?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 12:52 PM
  • As an addendum - from what I said last night - once you've run that example, replace the .Load with this and uncomment out each section one at a time and try it please:

    Private Sub _ Form1_Load(sender As System.Object, _ e As System.EventArgs) _ Handles MyBase.Load _people = New List(Of PersonData) Dim filePath As String = _ IO.Path.Combine(_desktop, "Sample_Data_1000000.csv") GetCSVData(filePath) Stop 'Dim qry As System.Linq.ParallelQuery(Of PersonData) = _ ' From pd As PersonData In _people.AsParallel _ ' Where CInt(pd.ZipCode) = 11510 'Dim qry As IEnumerable(Of PersonData) = _ ' From pd As PersonData In _people.AsParallel _ ' Where CInt(pd.ZipCode) = 11510 Stop Dim i As Integer = qry.Count Stop End Sub


    Do both work?

    Assuming so, are they the same thing then?

    How can it be two different things?


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    Sunday, July 23, 2017 1:01 PM
  • If you're stuck with "As Object" then there's more work to do.

    Do you remember around the end of last year we talked a fair amount about LINQ (specifically LINQ-To-Object)? It will return some form of a generic Enumerable, remember?

    *****

    Let's use an example where we'll both have the same data. How about have a look at this please:

    .....code removed...

    I have "Stop" in there three times, intentionally.

    The first one is after it reads the file and populates a List(Of Class). That will take about a minute so be patient. Once it's done though, continue and watch how quickly it runs that query!

    ... wait ... it can't do it that fast..

    Now continue again and it's noticeably slower. Why?

    Deferred execution.

    *****

    Late binding will kill performance. The difference is whether the 'binding' is done during compile time or during run time. When you force it to only know it's an "object", it has no choice but to do it during run time:

    https://docs.microsoft.com/en-us/dotnet/visual-basic/programming-guide/language-features/early-late-binding/

    From that article, also note this:

    "Early bound objects allow the compiler to allocate memory and perform other optimizations before an application executes."

    Does that help some?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Yes I do vaguely remember that conversation and have since started reading a bit more on LINQ as I thought it may be useful for this next project I am working on

    ---

    Example run and yes I noticed slight differences in speed and can I think now understand why

    ---

    Will read that article in more depth later, but initial scan of it makes sense


    Darren Rose

    Sunday, July 23, 2017 1:43 PM


  • Will read that article in more depth later, but initial scan of it makes sense


    Darren Rose

    Ok good.

    I don't want to get things too confused here but back to what you asked originally:

    From logEntry In EventLog1.Entries.AsParallel
        Where CInt(logEntry.InstanceId And &HFFFF) = 4624
        Where logEntry.Message.Contains("Darren")
        Order By logEntry.TimeWritten Descending

    The "OrderBy" made it an IOrderedEnumerable but there's a catch:

    The .AsParallel means that it's PLINQ and I don't know the ordered equivalent of it in PLINQ but there's bound to be one. That's not to say that IOrderedEnumerable won't work and not cause Late Binding errors - the second example shows that.

    The second example "worked" both ways because of inheritance.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 1:52 PM

  • Do both work?

    Assuming so, are they the same thing then?

    How can it be two different things?


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    yes both worked and both returned same number of results

    Darren Rose

    Sunday, July 23, 2017 1:57 PM

  • yes both worked and both returned same number of results

    Darren Rose

    The first time that I heard the expression "IsA" was from Deborah Kurata who used to frequent here years ago.

    If you're up for a short read, a pretty good article about inheritance is here:

    http://www.codemag.com/article/0505031

    I can't help but laugh about one part of what she's exemplifying there:

    "As an example of the last point, I spoke with a developer from Australia that implemented a deep hierarchy of classes for an army simulation program that he was writing. The simulation had many types of classes in the hierarchy to define different types of aircraft, vehicles, artillery, and individual troops. After the army reviewed the simulation, being Australia they thought it would be more realistic to add kangaroos to field. So one of the developers simply added another subclass on the deep hierarchy under the soldier class because it contained the functionality for initialization and movement. Wasn't he surprised when he ran the simulation and the kangaroos shot back!"


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 2:06 PM

  • yes both worked and both returned same number of results

    Darren Rose

    The first time that I heard the expression "IsA" was from Deborah Kurata who used to frequent here years ago.

    If you're up for a short read, a pretty good article about inheritance is here:

    http://www.codemag.com/article/0505031

    Brilliant, I like that :)

    Thanks - will add the site to my "reading list"


    Darren Rose

    Sunday, July 23, 2017 2:08 PM
  • Darren,

    Off-topic, but that thing we worked on a few months ago to copy files ... did you have to run that as administrator or do you remember?

    I want to use [the core part of it] in something else that I'm working on.

    Thanks


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 2:46 PM
  • Darren,

    Off-topic, but that thing we worked on a few months ago to copy files ... did you have to run that as administrator or do you remember?

    I want to use [the core part of it] in something else that I'm working on.

    Thanks


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    If you mean the LocalFileCopier class - then yes I did have to run it as administrator

    Darren Rose

    Sunday, July 23, 2017 2:50 PM
  • Darren,

    Off-topic, but that thing we worked on a few months ago to copy files ... did you have to run that as administrator or do you remember?

    I want to use [the core part of it] in something else that I'm working on.

    Thanks


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    If you mean the LocalFileCopier class - then yes I did have to run it as administrator

    Darren Rose


    Thanks

    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 2:54 PM
  • Ok good.

    I don't want to get things too confused here but back to what you asked originally:

    From logEntry In EventLog1.Entries.AsParallel
        Where CInt(logEntry.InstanceId And &HFFFF) = 4624
        Where logEntry.Message.Contains("Darren")
        Order By logEntry.TimeWritten Descending

    The "OrderBy" made it an IOrderedEnumerable but there's a catch:

    The .AsParallel means that it's PLINQ and I don't know the ordered equivalent of it in PLINQ but there's bound to be one. That's not to say that IOrderedEnumerable won't work and not cause Late Binding errors - the second example shows that.

    The second example "worked" both ways because of inheritance.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Even after reading articles and some other pages I am still at a loss as to how to fix my specific issues re above code

    I have done the following but still getting an error I don't understand

    What am I missing?

    Private Class EventLogData
            Public Property InstanceId As Long
            Public Property Message As String
            Public Property TimeWritten As String
            Public Property TimeGenerated As String
            Public Property Source As String
            Public Property EntryType As String
            Public Property MachineName As String
    End Class
    
     Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    
                   Dim EventLog1 As New EventLog("Security")
    
            Dim query As IEnumerable(Of EventLogData) = (From logEntry As EventLogData In EventLog1.Entries
                                                         Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                                                         Where logEntry.Message.Contains(StringToFind)
                                                         Order By logEntry.TimeWritten Descending).ToArray
    
    ....
    
    
    End Sub


    Darren Rose

    Sunday, July 23, 2017 3:06 PM

  • Even after reading articles and some other pages I am still at a loss as to how to fix my specific issues re above code

    I have done the following but still getting an error I don't understand

    What am I missing?

    Private Class EventLogData
            Public Property InstanceId As Long
            Public Property Message As String
            Public Property TimeWritten As String
            Public Property TimeGenerated As String
            Public Property Source As String
            Public Property EntryType As String
            Public Property MachineName As String
    End Class
    
     Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    
                   Dim EventLog1 As New EventLog("Security")
    
            Dim query As IEnumerable(Of EventLogData) = (From logEntry As EventLogData In EventLog1.Entries
                                                         Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                                                         Where logEntry.Message.Contains(StringToFind)
                                                         Order By logEntry.TimeWritten Descending).ToArray
    
    ....
    
    
    End Sub


    Darren Rose

    How about confirm something please: Option Infer changes how Option Strict works.

    Set Infer ON. Does that make the compile error go away?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 3:15 PM
  • How about confirm something please: Option Infer changes how Option Strict works.

    Set Infer ON. Does that make the compile error go away?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    No it doesn't

    Anything I can do or answer to help just let me know, really want to get this area nailed in my mind as lots of upcoming projects will be using this sort of idea


    Darren Rose

    Sunday, July 23, 2017 3:17 PM
  • How about confirm something please: Option Infer changes how Option Strict works.

    Set Infer ON. Does that make the compile error go away?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    No it doesn't

    Anything I can do or answer to help just let me know, really want to get this area nailed in my mind as lots of upcoming projects will be using this sort of idea


    Darren Rose


    I'll see what I can come up with and report back.

    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 3:31 PM
  • Darren,

    I don't know event logs and it's showing.

    Set up a For Each and see if it gives you the same problem. LINQ will come in after you get the simple one done.

    Let me know please.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 3:45 PM
  • If I do For Each loop for same thing I get following error when I run the code


    Darren Rose

    Sunday, July 23, 2017 3:59 PM
  • If I do For Each loop for same thing I get following error when I run the code


    Darren Rose


    There's an enumerator somewhere that I'm missing. I have a sense that you need to use a "In Get.." something - where that will 'get' the instances which is what you're after.

    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 4:02 PM
  • I have no idea, tried everything I can think of and spent ages googling, but most comments I read say if using LINQ they turn off Option Strict as can be an real pain to get it working if not

    But I would prefer to do it properly and find way round if I can


    Darren Rose

    Sunday, July 23, 2017 4:10 PM
  • I have no idea, tried everything I can think of and spent ages googling, but most comments I read say if using LINQ they turn off Option Strict as can be an real pain to get it working if not

    But I would prefer to do it properly and find way round if I can


    Darren Rose

    Just to see if this might set things in the right direction, run this and assuming it works on your end, write your LINQ query based on "eventLogCollection":

    Option Strict On Option Explicit On Option Infer Off Public Class Form1 Private Sub Form1_Load(sender As System.Object, _ e As System.EventArgs) _ Handles MyBase.Load Dim eventLogCollection As New List(Of DarrenEventLogData) For Each el As EventLog In EventLog.GetEventLogs For Each ele As EventLogEntry In el.Entries eventLogCollection.Add(New DarrenEventLogData _ With {.EntryType = ele.EntryType, .InstanceId = ele.InstanceId, _ .MachineName = ele.MachineName, .Message = ele.MachineName, _ .Source = ele.Source, .TimeGenerated = ele.TimeGenerated, _ .TimeWritten = ele.TimeWritten}) Next Next Stop 'Dim query As IEnumerable(Of EventLogData) = _ ' (From logEntry As EventLogData In EventLog1.Entries Where CInt(logEntry.InstanceId And &HFFFF) = 4624 Where logEntry.Message.Contains(StringToFind) Order By logEntry.TimeWritten Descending).ToArray End Sub Private Class DarrenEventLogData Public Property InstanceId As Long Public Property Message As String Public Property TimeWritten As DateTime Public Property TimeGenerated As DateTime Public Property Source As String Public Property EntryType As EventLogEntryType Public Property MachineName As String End Class End Class


    I changed the types of three of your class' properties and I changed the name just to keep me a little less confused. ;-)

    If I'm even close to right here, I see why PLINQ was being used...


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 4:18 PM
  • Hi

    Just as Frank found, I needed to make the same type changes. This code seems similar to Franks code as well, but I hadn't seen Franks post until now. Anyway, my code collects all Objects before querying them (I couldn't use your query parameters for obvious reasons)

     Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            Dim EventLog1 As New EventLog("Security")
            EventLog1.Log = "System"
    
            For Each entry As EventLogEntry In EventLog1.Entries
                Dim el As New EventLogData
                With entry
                    el.InstanceId = .InstanceId
                    el.Message = .Message
                    el.TimeWritten = .TimeWritten
                    el.TimeGenerated = .TimeGenerated
                    el.Source = .Source
                    el.EntryType = .EntryType
                    el.InstanceId = .InstanceId
                    el.MachineName = .MachineName
                End With
                evList.Add(el)
            Next
    
            ' base query on a list of EventLogData
            Dim query As IEnumerable(Of EventLogData) = (From logEntry In evList Where logEntry.InstanceId > id Where logEntry.Message.Contains(StringToFind) Order By logEntry.TimeWritten Descending).ToArray
    
        End Sub
    
        Dim evList As New List(Of EventLogData)
    
        ' need something for testing
        Dim StringToFind As String = "event"
    
        ' yours was .instanceid and &HFFFF = 4624
        Dim id As Long = 100000
    
        Private Class EventLogData
            Public Property InstanceId As Long
            Public Property Message As String
            Public Property TimeWritten As DateTime
            Public Property TimeGenerated As DateTime
            Public Property Source As String
            Public Property EntryType As EventLogEntryType
            Public Property MachineName As String
        End Class


    Regards Les, Livingston, Scotland


    • Edited by leshay Sunday, July 23, 2017 4:43 PM
    Sunday, July 23, 2017 4:42 PM
  • Okay that worked to get all events from all logs

    I have amended it slightly below so I can specify which log to use - Security in this case and it still works okay

    So now we have a For Each working, can we get LINQ/PLINQ query working in same sort of way?

    Yes - some of the logs I am searching though have close to a million entries, hence trying to find best way to do this, as there seems to be many ways to search/query event logs, so was trying each way and timing them on same query to work out best route to follow

    Option Strict On
    Option Explicit On
    Option Infer Off
    
    Public Class Form1
        Private Sub Form1_Load(sender As System.Object,
                               e As System.EventArgs) _
                               Handles MyBase.Load
    
            Dim eventLogCollection As New List(Of DarrenEventLogData)
    
            Dim EventLog1 As New EventLog("Security")
    
            For Each logentry As EventLogEntry In EventLog1.Entries
                eventLogCollection.Add(New DarrenEventLogData _
                        With {.EntryType = logentry.EntryType, .InstanceId = CInt(logentry.InstanceId And &HFFFF),
                              .MachineName = logentry.MachineName, .Message = logentry.Message,
                              .Source = logentry.Source, .TimeGenerated = logentry.TimeGenerated,
                              .TimeWritten = logentry.TimeWritten})
            Next
    
            Stop
    
            'Dim query As IEnumerable(Of EventLogData) = _
            '    (From logEntry As EventLogData In EventLog1.Entries Where CInt(logEntry.InstanceId And &HFFFF) = 4624 Where logEntry.Message.Contains(StringToFind) Order By logEntry.TimeWritten Descending).ToArray
    
    
        End Sub
    
        Private Class DarrenEventLogData
            Public Property InstanceId As Long
            Public Property Message As String
            Public Property TimeWritten As DateTime
            Public Property TimeGenerated As DateTime
            Public Property Source As String
            Public Property EntryType As EventLogEntryType
            Public Property MachineName As String
        End Class
    End Class
    


    Darren Rose

    Sunday, July 23, 2017 4:44 PM

  • So now we have a For Each working, can we get LINQ/PLINQ query working in same sort of way?


    If you want speed - and you do - stay away from LINQ for the first part; let it get all of the data.

    With that done though, now you want use some form or LINQ so that you can apply the conditions. PLINQ or LINQ is a good question and the only way to know is try it (and time it) both ways. PLINQ has an overhead and sometimes it's slower than LINQ.

    Forget parallel, forget ordering ... write the query as simply as you can to get all entries back then go from there. After that, change it a little and see where that takes you.

    Agree?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 4:55 PM
  • ... my code collects all Objects before querying them (I couldn't use your query parameters for obvious reasons)...

    I agree. :)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 4:55 PM
  • Darren,

    For Option Strict On and Option Infer Off, I'd rewrite your original ListView code as follows:

    Dim EventLog1 As New EventLog("Security")
    
    
    Dim filteredResults() As EventLogEntry = (From logEntry As EventLogEntry In EventLog1.Entries.Cast(Of EventLogEntry)
                                              Where CInt(logEntry.InstanceId And &HFFFF) = 4624
                                              Where logEntry.Message.Contains("Darren")
                                              Order By logEntry.TimeWritten Descending).ToArray
    
    
    Dim ListViewResults As ListViewItem
    Dim array(6) As String
    
    ListView1.View = View.Details
    ListView1.Columns.Add("Date/Time", 120)
    ListView1.Columns.Add("ID", 50)
    ListView1.Columns.Add("Source", 150)
    ListView1.Columns.Add("Type", 100)
    ListView1.Columns.Add("Machine", 100)
    ListView1.Columns.Add("Message", 1000)
    
    ListView1.BeginUpdate()
    For x As Integer = 0 To filteredResults.Count - 1
        array(0) = filteredResults(x).TimeGenerated.ToString
        array(1) = (filteredResults(x).InstanceId And &HFFFF).ToString
        array(2) = filteredResults(x).Source
        array(3) = filteredResults(x).EntryType.ToString
        array(4) = filteredResults(x).MachineName
        array(5) = filteredResults(x).Message
    
        ListViewResults = New ListViewItem(array)
        ListView1.Items.Add(ListViewResults)
        ListView1.Refresh()
    Next
    ListView1.EndUpdate()
    

    For the above to work, you do not need to create your own EventLogEntry Class.

    Notice I changed the name from 'query' to filteredResults() which is an array of EventLogEntry instances. That's because calling .ToArray on the original query executes it immediately and returns the actual result of the query as an array (of EventLogEntry's in this case).

    Sunday, July 23, 2017 4:58 PM

  • So now we have a For Each working, can we get LINQ/PLINQ query working in same sort of way?


    If you want speed - and you do - stay away from LINQ for the first part; let it get all of the data.

    With that done though, now you want use some form or LINQ so that you can apply the conditions. PLINQ or LINQ is a good question and the only way to know is try it (and time it) both ways. PLINQ has an overhead and sometimes it's slower than LINQ.

    Forget parallel, forget ordering ... write the query as simply as you can to get all entries back then go from there. After that, change it a little and see where that takes you.

    Agree?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Okay fair enough

    Just with the query I was only storing those that matched the criteria which I assumed was a quicker way of doing it? I suppose I can do same with the For Each loop? as I though if we get all entries which on a particular server currently stands at 963,154 entries it would be quite slow, but if just getting those which met would be quicker - perhaps I am looking at it wrong way round


    Darren Rose

    Sunday, July 23, 2017 5:03 PM


  • Okay fair enough

    Just with the query I was only storing those that matched the criteria which I assumed was a quicker way of doing it? I suppose I can do same with the For Each loop? as I though if we get all entries which on a particular server currently stands at 963,154 entries it would be quite slow, but if just getting those which met would be quicker - perhaps I am looking at it wrong way round


    Darren Rose

    Good point and I don't know the answer here.

    LINQ is great but "fast" is not its hallmark.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 5:06 PM

  • yes both worked and both returned same number of results

    Darren Rose

    Answering my own question: The parallel equivalent is OrderedParallelQuery<T>.

    I had a feeling there was one. ;-)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 5:10 PM
  • Thank you SPC and Leshay for your response as well - much appreciated

    I will continue testing tonight/tomorrow and probably start another discussion on best way to get the events initially as from my research/testing there seems to be so many ways to get event log entries, and since for this project I am talking millions of events across dozens of servers (or perhaps If I use event subscriptions it will be many millions all on one server - yet to decide)

    But it seems I can use any of the following to get the data, all having pros / cons:-

    1. EventLogEntry class as per this forum discussion
    2. EvengLogEntryCollection class - discounted as examples I tested very slow as it seems to create a local collection of the events and then query them, so doubling the work?
    3. EventLogQuery and EventLogReader - this seems quite interesting as if looking at remote server it sends the query to the server and returns results I believe - so quite quick, just need to spend more time with the query side to be able to get information I would need (which is usually specific ID containing a specific username in a certain date range).  Other pro to this is you can search an event log file as well as actual event logs, so could easily search old/archived event logs in a .evtx file
    4. WMI - Win32_NTLogEvent - benefit is can do local or remote, but might not matter if using subscriptions so all events on one server
    5. Powershell  - Get-EventLog or Get-WinEvent - although this moves me away from having a nice VB app/gui - but I like to look at all options

    Darren Rose

    Sunday, July 23, 2017 5:13 PM
  • Darren,

    Look into all of those - I bet that there's a good way to do this and it likely doesn't use LINQ.

    *****

    Your point about how much is held in memory though, that can be done without LINQ; the 3.0 way does still work after all:

    Option Strict On
    Option Explicit On
    Option Infer Off
    
    Public Class Form1
        Private Sub Form1_Load(sender As System.Object, _
                               e As System.EventArgs) _
                               Handles MyBase.Load
    
            Dim eventLogCollection As New List(Of DarrenEventLogData)
    
            For Each el As EventLog In EventLog.GetEventLogs
                For Each ele As EventLogEntry In el.Entries
                    If ele.InstanceId = 999 Then
                        eventLogCollection.Add(New  _
                                               DarrenEventLogData With {.EntryType = ele.EntryType, _
                                                                        .InstanceId = 999, _
                                                                        .MachineName = ele.MachineName, _
                                                                        .Message = ele.Message, _
                                                                        .Source = ele.Source, _
                                                                        .TimeGenerated = ele.TimeGenerated, _
                                                                        .TimeWritten = ele.TimeWritten})
                    End If
                Next
            Next
    
            Stop
    
        End Sub
    
        Private Class DarrenEventLogData
            Public Property InstanceId As Long
            Public Property Message As String
            Public Property TimeWritten As DateTime
            Public Property TimeGenerated As DateTime
            Public Property Source As String
            Public Property EntryType As EventLogEntryType
            Public Property MachineName As String
        End Class
    End Class

    In that one, I'm looking for an instance ID of 999 (35 of them because I wrote a select query to find out) and you can add to that "... AndAlso ..." and keep adding them.

    Once you have a subset, now you might want to use LINQ to order them if you want.

    Look at what's built-in though. I bet there's a better way.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Sunday, July 23, 2017 5:25 PM

  • yes both worked and both returned same number of results

    Darren Rose

    Answering my own question: The parallel equivalent is OrderedParallelQuery<T>.

    I had a feeling there was one. ;-)


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    brilliant - thanks Frank

    Darren Rose

    Sunday, July 23, 2017 6:52 PM
  • Thanks Frank, very continue playing tonight and tomorrow and see what I come up with and then report back

    As usual thank you for your help

    Yet more beer I owe you :)


    Darren Rose

    Sunday, July 23, 2017 6:53 PM
  • Okay after lots of testing using different methods of reading/querying event logs I have found the quickest and best way is using EventLogQuery and EventLogReader

    Just for informational purposes below shows the results of using different methods to query a server for the same information each time e.g. security log, a specific user name and event ID 4624 - the query returning 898 records each time:-

    • EventLogEntry using LINQ = 30.85 seconds
    • EventLogEntry using For..Each loop = 60.61 seconds
    • EventLogEntryCollection using LINQ = 11 minutes 54 seconds!
    • EventLogQuery using Xpath query = 2.71 seconds
    • WMI = 98.74 seconds
    • Powershell using Get-EventLog = 36 seconds
    • Powershell using Get-WinEvent = 236 seconds

    As you can see the clear winner for speed is EventLogQuery - and this has the benefit of also working quickly remotely and being able to use XML XPath queries so quite a powerful way to search/query a log (or logs)


    Darren Rose


    • Edited by wingers Tuesday, July 25, 2017 1:25 PM typo
    Tuesday, July 25, 2017 1:25 PM
  • Okay after lots of testing using different methods of reading/querying event logs I have found the quickest and best way is using EventLogQuery and EventLogReader

    Just for informational purposes below shows the results of using different methods to query a server for the same information each time e.g. security log, a specific user name and event ID 4624 - the query returning 898 records each time:-

    • EventLogEntry using LINQ = 30.85 seconds
    • EventLogEntry using For..Each loop = 60.61 seconds
    • EventLogEntryCollection using LINQ = 11 minutes 54 seconds!
    • EventLogQuery using Xpath query = 2.71 seconds
    • WMI = 98.74 seconds
    • Powershell using Get-EventLog = 36 seconds
    • Powershell using Get-WinEvent = 236 seconds

    As you can see the clear winner for speed is EventLogQuery - and this has the benefit of also working quickly remotely and being able to use XML XPath queries so quite a powerful way to search/query a log (or logs)


    Darren Rose


    It paid off to do the research then.

    Thanks for the update. :)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 1:43 PM
  • It paid off to do the research then.

    Thanks for the update. :)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Yes definitely

    ***

    I do have a further query that you may well be able to help me with?

    Whilst doing the testing and getting timings I noticed some where really high initially - so I separately timed getting the query results and then timed showing them on form (in this instance a listview) and found that the high timings I got were down to the listview NOT the query

    For example using best method above:-

    Query time = 0.21 seconds          

    Total time including updating listview with 3080 items = 25.3 seconds

    and second example

    Query time = 3.24 seconds

    Total time including updating listview this time with 29633 items = 149.86 seconds

    ***

    I tried all the "usual" suggestions made on forum about using .BeginUpdate, .EndUpdate etc but this only knocks about 2 seconds off so no better

    ***

    So is it that ListView is just slow and I should use something else? or perhaps how I am doing it?

    I can show code if needed, but basically doing query and adding it to a list/collection I have created to hold results - this query and adding to list/collection works really quickly

    DimeventqueryResultsCollection AsNewList(OfEventRecord)

    I then loop though list/collection using For..Next and add each one to a ListView

    ***

    Thoughts? ideas?

    Probably a much better way to do it, especially since in future we could be talking hundreds of thousands of results in some cases

    I want to store results somewhere - as eventually for some queries there will be an ability to create a report (probably using DevExpress reporting bits) so will need to access them to show some values in the report


    Darren Rose

    Tuesday, July 25, 2017 1:55 PM

  • Thoughts? ideas?

    Probably a much better way to do it, especially since in future we could be talking hundreds of thousands of results in some cases

    I want to store results somewhere - as eventually for some queries there will be an ability to create a report (probably using DevExpress reporting bits) so will need to access them to show some values in the report


    Darren Rose

    Dev's datagrid (along with your data collection) will run about as fast anything out there.

    I don't know if you're aware of this or not, but their DataGrid is their flagship product (and it really is good). They pride themselves on, among other things, how fast it can handle large volumes of data.

    How you come up with your collection is up to you but it's very flexible (even just XML as the data source will work).

    https://www.youtube.com/watch?v=8_3kRf0Uisc

    A ListView isn't really the animal to use here - in my opinion.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 2:07 PM
  • I had a feeling you might suggest that and just gave it a quick try simply by setting the Grid Control DataSource to my list e.g.

    GridControl1.DataSource = eventqueryResultsCollection

    And it is blazingly fast!!!

    Perhaps you can advise me on how I can control what fields from my datasource are shown, as the collection contains a lot more information than I need to show in the grid


    Darren Rose

    Tuesday, July 25, 2017 2:27 PM
  • I had a feeling you might suggest that and just gave it a quick try simply by setting the Grid Control DataSource to my list e.g.

    GridControl1.DataSource = eventqueryResultsCollection

    And it is blazingly fast!!!

    Perhaps you can advise me on how I can control what fields from my datasource are shown, as the collection contains a lot more information than I need to show in the grid


    Darren Rose

    It really is and if memory isn't an issue, that's the way to go.

    I would probably do it differently than you but here's a start:

    Run the designer and one of the selection areas is "Columns" (or something like that). You can select what's to be shown, what's to be ignored, the order to be shown in, how to format this and that ... you'll get the idea once you practice with it some.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 2:36 PM
  • I would probably do it differently than you but here's a start:

    Run the designer and one of the selection areas is "Columns" (or something like that). You can select what's to be shown, what's to be ignored, the order to be shown in, how to format this and that ... you'll get the idea once you practice with it some.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Always happy to hear your way of doing it - as probably much better than mine!

    ***

    Okay so have added some columns and set Fieldname to name of field in my collection and it adds them fine, with one exception and I am confused how this works to be honest

    Let me explain:

    When adding data in my collection to ListView there is a field called FormatDescription that hold the main text of an event log message - and using line below it adds it fine to my listview

    array(5) = (eventqueryResultsCollection(x).FormatDescription())

    Using same name for the gridcontrol doesn't seem to work, but I think I know why and I would like to understand it

    If I look at properties of one of the items in my collection I can see all fields except the one called FormatDescription - so where is it getting it from?

    screenshot below shows what I mean - can see all fields except FormatDescription - so where is it?


    Darren Rose

    Tuesday, July 25, 2017 2:54 PM
  • I'm confused about what I'm seeing there.

    Is "FormatDescription" a property in your class? You are still using a class, right?


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    Tuesday, July 25, 2017 3:03 PM
  • Below is the line used to create my collection - it is using the built in type/object used for storing Event log records e.g. EventRecord

    DimeventqueryResultsCollection AsNewList(OfEventRecord)

    Perhaps that is where I am going wrong, but thought as per other forum posts that no need to create my own if one exists already that holds that information

    But even if I put a breakpoint in during the query and look at a record I can see all values except FormatDescription? see image below

     


    Darren Rose

    Tuesday, July 25, 2017 3:15 PM
  • Below is the line used to create my collection - it is using the built in type/object used for storing Event log records e.g. EventRecord

    DimeventqueryResultsCollection AsNewList(OfEventRecord)

    Perhaps that is where I am going wrong, but thought as per other forum posts that no need to create my own if one exists already that holds that information

    But even if I put a breakpoint in during the query and look at a record I can see all values except FormatDescription? see image below

     


    Darren Rose

    It looks like whatever an "EventRecord" instance is -- doesn't have everything you want then?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 3:22 PM
  • It looks like whatever an "EventRecord" instance is -- doesn't have everything you want then?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    But it does have it somewhere, as adding that field as I said above to ListView works and shows data

    Also if I just use below to show first entry from collection in a msgbox then it also shows the information

     MsgBox(eventqueryResultsCollection(0).FormatDescription())

    So even though not clearly shown when you look at properties - it must exist somewhere as it is showing it everywhere else just not in gridcontrol?


    Darren Rose

    Tuesday, July 25, 2017 3:31 PM
  • It looks like whatever an "EventRecord" instance is -- doesn't have everything you want then?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    But it does have it somewhere, as adding that field as I said above to ListView works and shows data

    Also if I just use below to show first entry from collection in a msgbox then it also shows the information

     MsgBox(eventqueryResultsCollection(0).FormatDescription())

    So even though not clearly shown when you look at properties - it must exist somewhere as it is showing it everywhere else just not in gridcontrol?


    Darren Rose

    I'd be guessing because I don't know EventLogs.

    Does [whatever you're using to get the logs] have a way to export to XML? Export, save, write ... keywords like that?

    If I could see the data it might make more sense, but I'm just guessing here.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 3:38 PM
  • Probably easier if you can see what I am seeing - so mocked up simple version of code below

    Just a form, no controls etc

    So if you change the EventID number on line 8 to an ID you have in your System Event log and then run it you will get message box showing you the FormatDescription field of the first entry it finds

    If you then put breakpoint on line 20 and look at eventInstance or line  38 and look at eventqueryResultsCollection you will see all fields except FormatDescription - even though we can still display it on screen!!!

    Imports System.Diagnostics.Eventing.Reader
    
    Public Class Form1
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    
            Dim eventqueryResultsCollection As New List(Of EventRecord)
    
            Dim query As String = "*[System/EventID=10010]"
    
            Dim evntquery As New EventLogQuery("System", PathType.LogName, query)
    
            Try
    
                Dim logReader As New EventLogReader(evntquery)
    
                Dim eventInstance As EventRecord = logReader.ReadEvent()
                While eventInstance IsNot Nothing
    
                                    eventqueryResultsCollection.Add(eventInstance)
    
                                   eventInstance = logReader.ReadEvent()
    
                End While
    
            Catch ex As EventLogNotFoundException
    
                Console.WriteLine("Error While reading the Event logs")
                Return
    
            Catch ex As Exception
    
                MessageBox.Show("An exception occured: " + ex.Message)
    
            End Try
    
            MsgBox(eventqueryResultsCollection(0).FormatDescription())
    
        End Sub
    End Class
    


    Darren Rose

    Tuesday, July 25, 2017 3:40 PM
  • Darren,

    I'll do what I can (and line numbers are meaningless out of context), but I'll be at least tomorrow getting to it.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 3:45 PM
  • Darren,

    I'll do what I can (and line numbers are meaningless out of context), but I'll be at least tomorrow getting to it.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thanks Frank

    Once you paste the code into a new form overwriting existing code then line numbers will hopefully become more meaningful

    Also for info the field I mention is shown here in documentation for EventRecord - it appears to be a "method" rather than a "property" so perhaps that is why I cannot see it but can access it.  If so how can I get it in datagrid

    https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.eventing.reader.eventrecord?view=netframework-4.5


    Darren Rose

    Tuesday, July 25, 2017 3:51 PM
  • Darren,

    I'll do what I can (and line numbers are meaningless out of context), but I'll be at least tomorrow getting to it.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thanks Frank

    Once you paste the code into a new form overwriting existing code then line numbers will hopefully become more meaningful

    Also for info the field I mention is shown here in documentation for EventRecord - it appears to be a "method" rather than a "property" so perhaps that is why I cannot see it but can access it.  If so how can I get it in datagrid

    https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.eventing.reader.eventrecord?view=netframework-4.5


    Darren Rose

    I didn't get far; it threw an exception that the operation isn't supported.

    *****

    Declare a local variable and use it with the function to have it return the information. I assume it's a string?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 4:01 PM
  • That's odd - perhaps down to version of .NET you are using

    What line/where did exception occur?

    ***

    Yes it is a formatted string it appears as screenshot below shows one of the FormatDescription() values

    I can also do below so definitely a string

     Dim test As String = eventqueryResultsCollection(0).FormatDescription()
            MsgBox(test)


    Darren Rose

    Tuesday, July 25, 2017 4:08 PM
  • That's odd - perhaps down to version of .NET you are using

    What line/where did exception occur?

    ***

    Yes it is a formatted string it appears as screenshot below shows one of the FormatDescription() values

    I can also do below so definitely a string

     Dim test As String = eventqueryResultsCollection(0).FormatDescription()
            MsgBox(test)


    Darren Rose

    You surely don't want to try to be showing all of that stuff - per entry - in a control?

    Looking at what I'm seeing though, if someone (not me) knows more about event logs they would probably explain where that data came from and how you could more easily get to just what you're interested in.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 4:15 PM
  • Well for that large message possibly not - but I do need to be storing it in my collection, so perhaps can double click on a grid entry and then show it instead, but I was coming to that later as if I can't even show it in grid then I can't then show it when you click the grid

    ***

    I am thinking I need to create my own class to store the information then I can retrieve information from the FormatDescription() method and store it as a string in my own message property


    Darren Rose

    Tuesday, July 25, 2017 4:20 PM
  • Well for that large message possibly not - but I do need to be storing it in my collection, so perhaps can double click on a grid entry and then show it instead, but I was coming to that later as if I can't even show it in grid then I can't then show it when you click the grid

    ***

    I am thinking I need to create my own class to store the information then I can retrieve information from the FormatDescription() method and store it as a string in my own message property


    Darren Rose

    With Dev's grid control you *can* do that but I don't know that you want to. ;-)

    You can set up any column to use any (or their) controls to display the data in and one of their controls "expands when clicked". MemoEx or something like that.

    *****

    I know you do you need your own class, but don't hold on to the data twice.

    The code that you showed earlier - that List<T> goes away. Get the data directly into instances of your class and store those instances in a List(Of YourClassTypeHere).

    Now it'll be whatever you want it to be. If it needs to be persisted to file, that can be done too.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 4:30 PM
  • Okay so I have created my own class as below to hold some of the data I need to show (will add more fields later but just for testing)

       Private Class MyEventLogData
            Public Property TimeGenerated As DateTime
            Public Property EventId As Integer
            Public Property Source As String
            Public Property EntryType As String
            Public Property MachineName As String
            Public Property Message As String
        End Class

    I then declare a list of MyEventLogData

    Dim myresults As New List(Of MyEventLogData)


    Then changed the original method which was adding it to the list of EventRecord (first line below) to add it to my list of MyEventLogData - assume this is correct way to do it?

    ' add results to list / collection - OLD WAY
                    eventqueryResultsCollection.Add(eventInstance)
    
    ' manually add to my own class - NEW WAY
     myresults.Add(New MyEventLogData With {
                          .TimeGenerated = eventInstance.TimeCreated,
                          .EventId = eventInstance.Id,
                          .Source = eventInstance.ProviderName,
                          .EntryType = eventInstance.LevelDisplayName,
                          .MachineName = eventInstance.MachineName,
                          .Message = eventInstance.FormatDescription()})

    Then I can set it as datasource for my grid as below and it seems to work!!

    GridControl1.DataSource = myresults

    Please comment if better way of doing this as I like to learn

    ***

    One thing I notice is that if I hover over the message column of the GridControl then it automatically shows a pop-up preview of the large message which is enough for me - ONLY problem is that it doesn't show it all, so assume some sort of character limit or similar - is this something you have come across or know about?


    Darren Rose


    • Edited by wingers Tuesday, July 25, 2017 4:40 PM
    Tuesday, July 25, 2017 4:39 PM
  • One thing I immediately notice though is that this has slowed the query down considerably

    Adding to the original List Of EventRecord takes 0.21 seconds for 3029 results

    Adding the same 3029 results to my List Of MyEventLogData takes 19.5 seconds

    Surely must be some way of doing this without adding all that time to what in the scheme of things is not many records at all


    Darren Rose

    Tuesday, July 25, 2017 4:45 PM
  • Okay so I have created my own class as below to hold some of the data I need to show (will add more fields later but just for testing)

       Private Class MyEventLogData
            Public Property TimeGenerated As DateTime
            Public Property EventId As Integer
            Public Property Source As String
            Public Property EntryType As String
            Public Property MachineName As String
            Public Property Message As String
        End Class

    I then declare a list of MyEventLogData

    Dim myresults As New List(Of MyEventLogData)


    Then changed the original method which was adding it to the list of EventRecord (first line below) to add it to my list of MyEventLogData - assume this is correct way to do it?

    ' add results to list / collection - OLD WAY
                    eventqueryResultsCollection.Add(eventInstance)
    
    ' manually add to my own class - NEW WAY
     myresults.Add(New MyEventLogData With {
                          .TimeGenerated = eventInstance.TimeCreated,
                          .EventId = eventInstance.Id,
                          .Source = eventInstance.ProviderName,
                          .EntryType = eventInstance.LevelDisplayName,
                          .MachineName = eventInstance.MachineName,
                          .Message = eventInstance.FormatDescription()})

    Then I can set it as datasource for my grid as below and it seems to work!!

    GridControl1.DataSource = myresults

    Please comment if better way of doing this as I like to learn

    ***

    One thing I notice is that if I hover over the message column of the GridControl then it automatically shows a pop-up preview of the large message which is enough for me - ONLY problem is that it doesn't show it all, so assume some sort of character limit or similar - is this something you have come across or know about?


    Darren Rose


    Great then. :)

    The only thing that I'll mention is that you might want to check the information coming in - at least that it's not null (if a reference type), but that will slow it down so it might be worth taking a chance on not validating it.

    With that much data - and a lot of it being strings - if you're going to persist it then use this one to compress the data along the way:

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/8222cb3e-99e7-48a7-9bfa-aa1cd913d9fa/binary-serializationdeserialization-a-library-to-compress-and-encrypt-the-data?forum=vbgeneral


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 4:47 PM
  • One thing I immediately notice though is that this has slowed the query down considerably

    Adding to the original List Of EventRecord takes 0.21 seconds for 3029 results

    Adding the same 3029 results to my List Of MyEventLogData takes 19.5 seconds

    Surely must be some way of doing this without adding all that time to what in the scheme of things is not many records at all


    Darren Rose

    Your original way wasn't getting everything so how can you compare the two?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 4:48 PM
  • One thing I immediately notice though is that this has slowed the query down considerably

    Adding to the original List Of EventRecord takes 0.21 seconds for 3029 results

    Adding the same 3029 results to my List Of MyEventLogData takes 19.5 seconds

    Surely must be some way of doing this without adding all that time to what in the scheme of things is not many records at all


    Darren Rose

    Your original way wasn't getting everything so how can you compare the two?


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    The original way WAS getting it all and a lot more as I was showing it in the ListView before we discussed using GridControl?

    My class is only holding 6 fields of information, originally (existing) class was holding 26 fields of information and I could access them all - only issue was as discussion that one of them was a method so couldn't easily add it to gridcontrol, but could add it to listview


    Darren Rose

    Tuesday, July 25, 2017 4:55 PM

  • The original way WAS getting it all and a lot more as I was showing it in the ListView before we discussed using GridControl?

    My class is only holding 6 fields of information, originally (existing) class was holding 26 fields of information and I could access them all - only issue was as discussion that one of them was a method so couldn't easily add it to gridcontrol, but could add it to listview


    Darren Rose

    Obviously they've optimized theirs.

    You might look at trying to inherit from it and add the long message thing to it and you might also look at parallelization but that can get to be involved and you definitely need to know the ins and outs of the methods you're using to get the data (a parallel.foreach uses lambda expressions which is confusing if you don't know the methods inside and out).

    I don't have a solution here...


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Tuesday, July 25, 2017 5:00 PM
  • Yes appears so

    Okay, let me try a different way which you might be able to advise on as is related to DevExpress GridControl rather than my data

    If I go back to using existing "optimized" class then how can I add results to GridControl manually like I did for ListView rather than setting DataSource - is that possible

    e.g. for listview I looped through each item in class and added it - that way I could access the mysterious "FormatDescription" field and add it okay

    So can I do equivalent of below - but add to GridControl rather than ListView perhaps?

            For x As Integer = 0 To eventqueryResultsCollection.Count - 1
                array(0) = (eventqueryResultsCollection(x).TimeCreated.ToString)
                array(1) = (eventqueryResultsCollection(x).Id.ToString)
                array(2) = (eventqueryResultsCollection(x).ProviderName.ToString)
                array(3) = (eventqueryResultsCollection(x).LevelDisplayName.ToString)
                array(4) = (eventqueryResultsCollection(x).MachineName.ToString)
                array(5) = (eventqueryResultsCollection(x).FormatDescription())
                ListViewResults = New ListViewItem(array)
                Update_ListView1(ListViewResults)
            Next


    Darren Rose

    Tuesday, July 25, 2017 5:06 PM
  • Yes appears so

    Okay, let me try a different way which you might be able to advise on as is related to DevExpress GridControl rather than my data

    If I go back to using existing "optimized" class then how can I add results to GridControl manually like I did for ListView rather than setting DataSource - is that possible

    e.g. for listview I looped through each item in class and added it - that way I could access the mysterious "FormatDescription" field and add it okay

    So can I do equivalent of below - but add to GridControl rather than ListView perhaps?

            For x As Integer = 0 To eventqueryResultsCollection.Count - 1
                array(0) = (eventqueryResultsCollection(x).TimeCreated.ToString)
                array(1) = (eventqueryResultsCollection(x).Id.ToString)
                array(2) = (eventqueryResultsCollection(x).ProviderName.ToString)
                array(3) = (eventqueryResultsCollection(x).LevelDisplayName.ToString)
                array(4) = (eventqueryResultsCollection(x).MachineName.ToString)
                array(5) = (eventqueryResultsCollection(x).FormatDescription())
                ListViewResults = New ListViewItem(array)
                Update_ListView1(ListViewResults)
            Next


    Darren Rose

    That would be worth asking Dev.

    I can tell you now that they'll suggest a way that will involve using a TableAdapter, or probably.

    Nothing wrong with that though; it plays well with their stuff, but nobody knows their grid better than they do so ask them what they'd suggest.


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    Tuesday, July 25, 2017 5:12 PM
  • Hi Frank

    Spoke to Dev about this and as expected they came back with a perfect solution

    I can still bind my data to the GridControl's DataSource but I can also add an extra "unbound" column for the column I was having issues with

    The unbound column then raises a CustomUnboundColumnData event which I can handle and then run the EventDescription() method to populate the event message

    Works seamlessly

    Reply from Dev copied below for your reference as certainly useful

    "You can use your list as data source of the GridControl. GridControl will automatically generate columns for all properties. Then, you can use an UnboundColumn to display data returned by the method. The main idea of this approach is to add the UnboundColumn  to the target GridView and then handle the ColumnView.CustomUnboundColumnData event. In this event, you can access the target object in the xxxx list using the ListSourceRowIndex  property of the e parameter, and then invoke the target method and assign the value returned by that method to the e.Value property."


    Darren Rose

    Wednesday, July 26, 2017 12:05 PM
  • Hi Frank

    Spoke to Dev about this and as expected they came back with a perfect solution

    I can still bind my data to the GridControl's DataSource but I can also add an extra "unbound" column for the column I was having issues with

    The unbound column then raises a CustomUnboundColumnData event which I can handle and then run the EventDescription() method to populate the event message

    Works seamlessly

    Reply from Dev copied below for your reference as certainly useful

    "You can use your list as data source of the GridControl. GridControl will automatically generate columns for all properties. Then, you can use an UnboundColumn to display data returned by the method. The main idea of this approach is to add the UnboundColumn  to the target GridView and then handle the ColumnView.CustomUnboundColumnData event. In this event, you can access the target object in the xxxx list using the ListSourceRowIndex  property of the e parameter, and then invoke the target method and assign the value returned by that method to the e.Value property."


    Darren Rose

    They definitely know their products well.

    I'm glad you have that working now. :)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Wednesday, July 26, 2017 12:53 PM
  • Hi Frank

    Something relating to this project you may be able to help me further with (hopefully!)

    I have the project working and can now query event logs and show relevant results in dev grid control, along with creating reports, exporting results etc

    As you gathered earlier this project does involve quite a large amount of data and large number of events in each log

    What I have to do on this network is setup event log forwarding so that events from all main servers got to just one server, which will then be used for analysis, archiving etc, but this presents challenges due to the amount of information and the fact we need easy access to at least a month (if not more) to perform queries on

    My question for you doesn't really relate to event logs etc so doesn't matter if you don't know that area

    One particular server (a domain controller) created 1 million events in just 10 days and the resulting size of the event log file (security.evtx) was 488mb - so you can imagine 30 (or 90 days) worth when up to 10 different servers are all sending events to one server!

    Event log data is in fact just XML it appears - as that is how I am querying it using Xpath

    So I tried exporting all 1 million events from event viewer as XML and the resulting file was 2.27Gb so ruled that method out, I also tried CSV and it was 1.09Gb

    Finally I simply zipped the 488mb file and it was only 45.3mb!!!

    Any suggestions on how best I can store months worth of data of this size in a better way? I had thought perhaps using SQL but not something I have ever used from a VB point of view.

    Then I thought perhaps Frank would have an idea where I can use something like you linked above to serialize? it to allow me to easily archive off old data and store it - but still be able to access it and search it later

    Below is a sample event record which you can see is just XML n.b. the field names vary from event to event in the EventData section

    Excuse the long explanation, but I do as ever really value your input on areas like this

    <?xml version="1.0"?>
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    	<System>
    		<Provider Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Name="Microsoft-Windows-Security-Auditing"/>
    		<EventID>4624</EventID>
    		<Version>2</Version>
    		<Level>0</Level>
    		<Task>12544</Task>
    		<Opcode>0</Opcode>
    		<Keywords>0x8020000000000000</Keywords>
    		<TimeCreated SystemTime="2017-07-30T23:56:27.504721200Z"/>
    		<EventRecordID>41581</EventRecordID>
    		<Correlation ActivityID="{EFF6CD65-009B-0002-80CD-F6EF9B00D301}"/>
    		<Execution ThreadID="16740" ProcessID="1672"/>
    		<Channel>Security</Channel>
    		<Computer>XPSDESKTOP</Computer>
    		<Security/>
    	</System>
    
    	<EventData>
    		<Data Name="SubjectUserSid">S-1-5-18</Data>
    		<Data Name="SubjectUserName">XPSDESKTOP$</Data>
    		<Data Name="SubjectDomainName">WORKGROUP</Data>
    		<Data Name="SubjectLogonId">0x3e7</Data>
    		<Data Name="TargetUserSid">S-1-5-18</Data>
    		<Data Name="TargetUserName">SYSTEM</Data>
    		<Data Name="TargetDomainName">NT AUTHORITY</Data>
    		<Data Name="TargetLogonId">0x3e7</Data>
    		<Data Name="LogonType">5</Data>
    		<Data Name="LogonProcessName">Advapi </Data>
    		<Data Name="AuthenticationPackageName">Negotiate</Data>
    		<Data Name="WorkstationName">-</Data>
    		<Data Name="LogonGuid">{00000000-0000-0000-0000-000000000000}</Data>
    		<Data Name="TransmittedServices">-</Data>
    		<Data Name="LmPackageName">-</Data>
    		<Data Name="KeyLength">0</Data>
    		<Data Name="ProcessId">0x680</Data>
    		<Data Name="ProcessName">C:\Windows\System32\services.exe</Data>
    		<Data Name="IpAddress">-</Data>
    		<Data Name="IpPort">-</Data>
    		<Data Name="ImpersonationLevel">%%1833</Data>
    		<Data Name="RestrictedAdminMode">-</Data>
    		<Data Name="TargetOutboundUserName">-</Data>
    		<Data Name="TargetOutboundDomainName">-</Data>
    		<Data Name="VirtualAccount">%%1843</Data>
    		<Data Name="TargetLinkedLogonId">0x0</Data>
    		<Data Name="ElevatedToken">%%1842</Data>
    	</EventData>
    </Event>


    Darren Rose

    Thursday, August 3, 2017 1:52 PM
  • Darren,

    I'll be a few hours before I can put my head into this.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thursday, August 3, 2017 2:00 PM
  • Darren,

    I'll be a few hours before I can put my head into this.


    "A problem well stated is a problem half solved.” - Charles F. Kettering


    no hurry at all - appreciate you taking the time :)

    Darren Rose

    Thursday, August 3, 2017 2:00 PM
  • Darren,

    If you just want "unlimited size" zipping, I have something else that I put together a while back:

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/0b4eb8f4-122d-4a06-92ad-e03514d62467/zippingunzipping-files?forum=vbgeneral

    I won't be sure that all the links work but if you find something that doesn't, just let me know and I'll update it.

    If you want to create a serializable class and use the one we talked about earlier this morning, that would automate it some (it also uses DotNetZip), but you'd have to transfer the data in the XML to instances of the class then back at some point and I think then the issue would be time. That's a lot of stuff!


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thursday, August 3, 2017 2:44 PM
  • Thanks Frank, I will take a look at that.

    Yes it is a huge amount of data and problem is that it has to be kept/stored somewhere in case ever need it to query something from months ago.

    I have it working now so it can query the live logs - which we can set so they are kept for a month at least and then archive.

    I also have ability to browse for an archived .evtx file and query that which also works well but does consume a huge amount of RAM whilst doing so as files can be so big. Originally got memory exceptions so had to set app to run 64-bit to work and then it works okay, but uses 3gb of RAM whilst querying the file.

    With archived logs I don't think speed is such a problem, it is more than ability to be able to reliably archive them off and be able to access them later if needed.

    I could try just zipping the .evtx files to save storage space and then unzip with your assembly as needed

    BUT I am thinking that some sort of database to store them all in so I can then query database may be way forward - just another area to learn about I suppose!


    Darren Rose

    Thursday, August 3, 2017 3:17 PM

  • With archived logs I don't think speed is such a problem, it is more than ability to be able to reliably archive them off and be able to access them later if needed.

    I could try just zipping the .evtx files to save storage space and then unzip with your assembly as needed

    BUT I am thinking that some sort of database to store them all in so I can then query database may be way forward - just another area to learn about I suppose!


    Darren Rose

    You didn't ask - and I'm not volunteering here!

    - BUT -

    I wouldn't do it that way. For this, using one or more classes has an advantage. For example:

    <Provider Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Name="Microsoft-Windows-Security-Auditing"/>
    <EventID>4624</EventID>
    <Version>2</Version>
    <Level>0</Level>
    <Task>12544</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8020000000000000</Keywords>
    <TimeCreated SystemTime="2017-07-30T23:56:27.504721200Z"/>
    <EventRecordID>41581</EventRecordID>
    <Correlation ActivityID="{EFF6CD65-009B-0002-80CD-F6EF9B00D301}"/>
    <Execution ThreadID="16740" ProcessID="1672"/>
    <Channel>Security</Channel>
    <Computer>XPSDESKTOP</Computer>
    <Security/>

    Everything there is a string (obviously so - it's XML). How are you going to store that using a database? As strings?

    If you think it through, you could set up a class which would have a field for (as an example) ProviderGUID as an instance of the GUID structure. Now that field just got a lot smaller.

    Go down the list. Only you know if something can better be represented by value types like Integer. I see what looks like that would fit the bill but I don't know all of the possibilities here - back to me not knowing event logs.

    Now the data should be a good bit smaller but I still wouldn't keep all of them in any one place.

    Devise a plan that will archive the data based on some scheme (maybe by date range, per server, or however works best).

    Along with it, devise a class that will know exactly where "xyz.bin" is so it becomes the manager for all of this stuff.

    It would be a heck of an undertaking but ... for what it's worth.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thursday, August 3, 2017 3:57 PM
  • Thanks Frank I will give it some thoughts and let you know how I get on

    A lot of the information is not actually really relevant or needed, so I am thinking I could get it down to just required fields then it would be smaller anyway, wherever I chose to store it

    Yes most of them would be strings, eventID could be integer, timecreated stored as date/time, but mostly strings

    The main issue wherever I store it is as you say a way to easily remember where it is to then be able to search a specific date range at a later date - that was why I thought perhaps some custom way of storing it like you have done in past for storing my IP testing results etc, some sort of binary or similar

    As always, I appreciate your thoughts


    Darren Rose

    Thursday, August 3, 2017 6:09 PM
  • Thanks Frank I will give it some thoughts and let you know how I get on

    A lot of the information is not actually really relevant or needed, so I am thinking I could get it down to just required fields then it would be smaller anyway, wherever I chose to store it

    Yes most of them would be strings, eventID could be integer, timecreated stored as date/time, but mostly strings

    The main issue wherever I store it is as you say a way to easily remember where it is to then be able to search a specific date range at a later date - that was why I thought perhaps some custom way of storing it like you have done in past for storing my IP testing results etc, some sort of binary or similar

    As always, I appreciate your thoughts


    Darren Rose

    It depends. ;-)

    If you can take the information handed to you from the event logs and put them into one or more classes and get the data from that - that's definitely a good idea.

    To get there will take a lot of doing but I think it would be worthwhile. I'm sure there's a lot of what's returned that's of no value [to your purpose] so your data would be a good bit more compact and also faster to get you the information that is of import to you.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thursday, August 3, 2017 6:31 PM
  • It depends. ;-)

    If you can take the information handed to you from the event logs and put them into one or more classes and get the data from that - that's definitely a good idea.

    To get there will take a lot of doing but I think it would be worthwhile. I'm sure there's a lot of what's returned that's of no value [to your purpose] so your data would be a good bit more compact and also faster to get you the information that is of import to you.


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Thanks Frank - apologies for delay in replying but girlfriends father died from a massive stroke yesterday so been side tracked

    Will get back to this and give it some more thought and may ask your advice as really need to find best way of storing all this information in best way to the query it later


    Darren Rose

    Saturday, August 5, 2017 9:26 PM

  • Thanks Frank - apologies for delay in replying but girlfriends father died from a massive stroke yesterday so been side tracked

    Will get back to this and give it some more thought and may ask your advice as really need to find best way of storing all this information in best way to the query it later


    Darren Rose

    I'm sorry to know that.

    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Saturday, August 5, 2017 11:12 PM