none
Best route to read thru PSV file and call FileSystemWatcher to notify the user of conditions being met

    Question

  • Have a winforms application, which currently has the FileSystemWatcher setup and works when the psv file is updated or deleted.. 
    So current setup, application flashes when the file(psv) is updated or deleted.(very general)

    New request is to ONLY flash when rows within the psv meet certain criteria
    The file contains the following:
    user|datetime|system|notes

    I need to loop thru the file for all records from today and based on configured thresholds determine if the application needs to flash.

    Example:  Number of Reported issues = 3   Threshold = 5 minutes

    So if there are 3 rows that have occurred within 5 minutes of each other, then flash the application.. 

    Things already in place and working:

    Flashing is working, but very general(no help needed here)

    Looping thru the PSV already working.. (done it in the past)

    The portion i need help with is the most efficient way to open/read the file and only look at the records within the threshold and they meet the 3 within 5 minutes then execute the flashing logic


    Thursday, April 27, 2017 3:36 PM

Answers

  • Well, here's a start. It's all I have time for tonight, and maybe it'll get you going to come up with more yourself. Read the whole file in, create a class to store the data and use LINQ to put it into a List<>. Something like this:

    private void TestIt()
    {
        string delimiter = "|";
        string FileName = "TestSubmissionsForumQuestion.psv";
        var Submissions = (File.ReadAllLines(FileName)
                            .Select(line => line.Split('|'))
                            .Where(values => values[0] != "User") // excludes header
                            .Where(values => Convert.ToDateTime(values[1]) > DateTime.Today)
                            //.OrderByDescending(values => Convert.ToDateTime(values[1])) //if you want to change order
                            .Select(values => new SubmissionsForAlerts()
                                { UserName = values[0],
                                    Date = Convert.ToDateTime(values[1]),
                                    System = values[4]
                                })
                            .ToList<SubmissionsForAlerts>() 
            );
    }
    private class SubmissionsForAlerts
    {
        public string UserName { get; set;}
        public DateTime Date { get; set; }
        public string System { get; set; }
    }

    Then, from there you could loop through Submissions, which is a List<SubmissionsForAlerts>, to check for the criteria you're looking for. I'm not sure if that part could be done with LINQ (probably not, but a foreach loop should be fine anyway). Putting only today's submissions into the List will reduce the number of records you loop through.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    • Marked as answer by Cubangt Thursday, May 4, 2017 1:03 PM
    Tuesday, May 2, 2017 5:22 AM
    Moderator
  • And you may also need:

    using System.Linq;


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    • Marked as answer by Cubangt Thursday, May 4, 2017 1:04 PM
    Wednesday, May 3, 2017 4:56 AM
    Moderator

All replies

  • Since you haven't shown your looping code, I can only guess as to how you're actually reading the file and processing it. (And I don't know what a "psv" file is, but perhaps that doesn't matter. Maybe it's similar to a "csv" file?) Are you reading line-by-line or reading in the whole file into a string[] array? Are the rows sorted by date? Ascending or Descending? Do you have a class defined with the properties (such as the Date property) and are you reading the data into a List<MyClass>? (In which case, you might be able to use a LINQ query.) Or are you simply parsing each piece of data (or maybe just the date) in each row?

    I think some more information as to the format of the data and how you are currently looping through it would make it easier for us to help you refine your process.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Sunday, April 30, 2017 4:28 PM
    Moderator
  • PSV is Pipe Separated File, instead of comma.

    I mentioned I had the looping from another project, but haven't been able to find that.. so at this moment, no looping logic to post.. but still looking.. I have plenty of projects where I loop thru excel files and parse out data so may be able to repurpose that if needed..

    I would think that reading it all in at once and then running the logic against that is best option, I cant lock the file to run this routine, it needs to run every time a new submissions is added so that the notifier is executed and admins are aware of the issues. As for the sorting, they are inserted in datetime order(Ascending)

    No class defined or parseing the data yet..

    Below is sample of a few records, if I can attached the actual psv with these I will, but below is perfect example of what I need to do.. Say my occurrences were 3 and the threshold was every 5 minutes, the 3rd entry would trigger the alerting to occur, 1 minute later someone else submits and that 4th also triggers because the min is 3 and the 4th is still within the 5 minute window of when it was submitted, so if anymore come in before it becomes 7:42 it would alert again when it is submitted.

    User|Date|Week Day|Time|Issue System|Issue Type|Issue Details|Screenshot Path|Productivity Lost
    UserName|4/29/2017 7:37:29 PM|Saturday|19:37:29|Email|Kickout||C:\Users\UserName\Documents\Issues\Email\04-29-2017\20170429193729_UserName-Kickout.jpg|None
    UserName|4/29/2017 7:37:57 PM|Saturday|19:37:57|Email|Error||C:\Users\UserName\Documents\Issues\Email\04-29-2017\20170429193757_UserName-Error.jpg|Less than 30 minutes
    UserName|4/29/2017 7:39:42 PM|Saturday|19:39:42|Email|Error||C:\Users\UserName\Documents\Issues\Email\04-29-2017\20170429193942_UserName-Error.jpg|1 hour to 2 hours
    UserName|4/29/2017 7:40:27 PM|Saturday|19:40:27|Email|Error||C:\Users\UserName\Documents\Issues\Email\04-29-2017\20170429194027_UserName-Error.jpg|1 hour to 2 hours

    Any help or suggestions would be great.. just need to understand what is the most optimal way to approach this and perform the task requested.

    Monday, May 1, 2017 3:38 AM
  • Sorry that I couldn't get to this sooner (been a busy day at work). I have a few questions for you before I can give you any relevant feedback:

    1. Is that header included in the file? If so, I assume that it's always first?
    2. Does the file only contain today's submissions? Or will it have previous day's activity also?
    3. Do you need to trigger the alert by UserName, or are all submissions by all users just lumped together? (Your example just shows one user, but it looks like the PSV file might be by user anyway?)

    That's all I've got, just off the top of my head (there may be more once I play around with it).


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Tuesday, May 2, 2017 12:37 AM
    Moderator
  • Header will always be included and always first.

    It will be a running total, meaning that it will always have past and present entries, Until they decide to archive the file.

    The 2 conditions(besides the occurrences and intervals) are date/time and system..

    So if I'm the admin over email, and my intervals is every 10 minutes and 5 occurrences within that 10 minutes, I don't care about any other systems only if my email system met those conditions..

    I really appreciate any help and suggestions..

    Tuesday, May 2, 2017 3:45 AM
  • Well, here's a start. It's all I have time for tonight, and maybe it'll get you going to come up with more yourself. Read the whole file in, create a class to store the data and use LINQ to put it into a List<>. Something like this:

    private void TestIt()
    {
        string delimiter = "|";
        string FileName = "TestSubmissionsForumQuestion.psv";
        var Submissions = (File.ReadAllLines(FileName)
                            .Select(line => line.Split('|'))
                            .Where(values => values[0] != "User") // excludes header
                            .Where(values => Convert.ToDateTime(values[1]) > DateTime.Today)
                            //.OrderByDescending(values => Convert.ToDateTime(values[1])) //if you want to change order
                            .Select(values => new SubmissionsForAlerts()
                                { UserName = values[0],
                                    Date = Convert.ToDateTime(values[1]),
                                    System = values[4]
                                })
                            .ToList<SubmissionsForAlerts>() 
            );
    }
    private class SubmissionsForAlerts
    {
        public string UserName { get; set;}
        public DateTime Date { get; set; }
        public string System { get; set; }
    }

    Then, from there you could loop through Submissions, which is a List<SubmissionsForAlerts>, to check for the criteria you're looking for. I'm not sure if that part could be done with LINQ (probably not, but a foreach loop should be fine anyway). Putting only today's submissions into the List will reduce the number of records you loop through.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    • Marked as answer by Cubangt Thursday, May 4, 2017 1:03 PM
    Tuesday, May 2, 2017 5:22 AM
    Moderator
  • I really appreciate the sample code.. Ill test out tonight when im back in front of my pc. Just sucks having to wait to work on it at home, it is what it is..

    thank you, ill post back my results and full implementation once its working for others to use.

    Tuesday, May 2, 2017 3:39 PM
  • You're welcome! Glad to help! =0)

    I work from home, so I don't have that problem!  ;0)   But still, sometimes work is so crazy busy (deadlines, etc.), that I don't have time to do other stuff, even though I'm at home!

    Yes, I'm curious to see what you come up with, so definitely post it when you're done!! But also, let me know if you get stuck ...


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Tuesday, May 2, 2017 4:30 PM
    Moderator
  • So I was playing around with the provided example, and received an error:

    Error 1 'System.Array' does not contain a definition for 'Select' and no extension method 'Select' accepting a first argument of type 'System.Array' could be found (are you missing a using directive or an assembly reference?)

    Its references this line

    .Select(line => line.Split('|'))

    Did I miss something?

    Wednesday, May 3, 2017 2:43 AM
  • You're probably missing:

    using System.Collections.Generic;


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Wednesday, May 3, 2017 4:53 AM
    Moderator
  • And you may also need:

    using System.Linq;


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    • Marked as answer by Cubangt Thursday, May 4, 2017 1:04 PM
    Wednesday, May 3, 2017 4:56 AM
    Moderator
  • Well, i believe i have generic already, but Linq, probably not.. 

    why wouldnt it give me the option to resolve this like other missing "using" statements?

    Typically first thing i do is right click, and select "resolve", but this time around that was not an option.

    once again, ill have to wait til tonight to correct and test :( 

    Bonnie, i really appreciate the help. 

    Wednesday, May 3, 2017 1:37 PM
  • I just verified that it *is* System.Linq that's missing. But, I do find it odd that you didn't already have that in your usings, because when adding a new Windows Form or a new Class to your project, System.Linq is one of the usings it includes for you. Although, maybe it depends on what version of Visual Studio you've got (I just verified that's the case for VS2010 and VS2015).


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Wednesday, May 3, 2017 4:58 PM
    Moderator
  • Well i must confess, typically when im towards the end of my development, i like to go thru and clean up code and remove unused code, so i typically go thru each of my code files, right-click and select to remove unused "usings" im sure that is why its not there. Easy enough fix for tonight.
    Wednesday, May 3, 2017 6:12 PM
  • Bonnie, you helped out so much.. attached is a screen shot stepping thru the test and as you can see, the data was pulled out of the file, will need to check now how I can actually call this routine within the filesystemwatcher to properly alert users that they have something to look for.

    I believe that I could call this routine(after adjusting to fit true conditions) from the changed event, have the above method return bool val indicating whether it met the conditions or not.

    basically if true, call FlashWindowEX(this)

            private void fsw_Changed(object sender, FileSystemEventArgs e)
            {
                FlashWindowEx(this);
    
            }

    Thursday, May 4, 2017 3:17 AM
  • >>I believe that I could call this routine(after adjusting to fit true conditions) from the changed event, have the above method return bool val indicating whether it met the conditions or not.<<

    Yep, that sounds about right ... that's how I'd do it. Have you figured out the code to use test for the conditions? I haven't thought about it at all, but I imagine it wouldn't be too difficult.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Thursday, May 4, 2017 4:42 AM
    Moderator
  • Bonnie,

    I really appreciate the help and suggestions. I have mark your responses as "Answers" and look forward to getting back to coding this in tonight and this weekend.

    As for testing, i have a test file that has about 50+ entries that i can use, ill need to update the file to have records with current dates for the logic to work, then i can move on to the next step.

    But in case i missed something, the test you are referring to for the conditions, was there something else i need to consider?

    I may change around your logic to pull in the "time" and "date"(they are separate fields in the file), first conditions pull in all current date records like your sample, then while looping thru the submissions, only look at the "time" value if within the given occurences and interval.

    Thursday, May 4, 2017 1:12 PM
  • >>But in case i missed something, the test you are referring to for the conditions, was there something else i need to consider?

    I may change around your logic to pull in the "time" and "date"(they are separate fields in the file), first conditions pull in all current date records like your sample, then while looping thru the submissions, only look at the "time" value if within the given occurences and interval.<<

    Nothing else to consider really ... just the bit of logic to determine the occurrences within the interval, as you were going to do.

    However, I would *not* use the separate time field. The date field that you have includes the time anyway, and doing time "manipulations" with a DateTime property is the preferable way to check the intervals, in my opinion.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Thursday, May 4, 2017 3:17 PM
    Moderator
  • Bonnie, I haven't forgotten about your examples and suggestions. I have a new build that will be demo'd for upper management and users, this week, if all goes well, I will begin to implement the logic you assisted with into that build for future release.

    So I will be working on it this week I think and hope to have a functional build soon. Will post back once I have something working or if I run into issues..

    thank you

    Monday, May 15, 2017 2:46 AM
  • Hope the demo goes well (or went well)!  =0)

    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Wednesday, May 17, 2017 3:06 PM
    Moderator
  • Thanks Bonnie,

    Demo is Friday Morning..

    Was working on the logic you provided to search thru and make sure if the conditions are met, and I believe I have first attempt that seems to work.. currently hardcoded with values, but will be updated soon to pull actual user settings into the variable..

    Do you see any issues I may run into with this logic now that ive modified yours.

                int subOccurances = 0;
                // Update with user interval selection
                TimeSpan tsInterval = new TimeSpan(0, 5, 0);
                // Update with user occurances selection
                int userOccurances = 3;
                
                char[] delimiter = new char[1];
                delimiter[0] = '|';
                var Submissions = (File.ReadAllLines(ppath)
                                    .Select(line => line.Split(delimiter))
                                    .Where(values => values[4] != "Issue System") // excludes header
                                    .Where(values => Convert.ToDateTime(values[1]) > DateTime.Today)
                                    //.OrderByDescending(values => Convert.ToDateTime(values[1])) //if you want to change order
                                    .Select(values => new SubmissionsForAlerts()
                                    {                                    
                                        //UserName = values[0],
                                        Date = Convert.ToDateTime(values[1]),
                                        System = values[4]
                                    })
                                    .ToList<SubmissionsForAlerts>()
                    );
                
                TimeSpan tsNow = DateTime.Now.TimeOfDay;
                TimeSpan tsStart = tsNow - tsInterval;
    
                foreach (var rec in Submissions)
                {
                    TimeSpan tsOcc = rec.Date.TimeOfDay;
                    foreach (var item in Properties.Settings.Default.AppListSel)
                    {
                        if (rec.System == item)
                        {
                            if ((tsOcc > tsStart) && (tsOcc < tsNow))
                            {
                                subOccurances++;
                            }
                        }
                    }
                }
    
                if (subOccurances > userOccurances)
                {
                    Messages ms = new Messages("FOUND", "ScreenShot");
                    ms.Show();
                }

    Since only a hand full of users(5) have the previous version of the application, if I can implement this logic before the demo or at least before providing to the masses it would def help a lot of the users.

    Thursday, May 18, 2017 4:07 AM
  • Just two suggestions:

    You'd have less Submission records to iterate through if you ordered by date descending (the line I originally commented out, because I hadn't thought about how you might do the counting).

    Also, you can break out of the loop as soon as you hit the userOccurances, so you don't have to iterate through the entire List of Submissions. Either like this:

    foreach (var rec in Submissions)
    {
    	TimeSpan tsOcc = rec.Date.TimeOfDay;
    	foreach (var item in Properties.Settings.Default.AppListSel)
    	{
    		if (rec.System == item)
    		{
    			if ((tsOcc > tsStart) && (tsOcc < tsNow))
    			{
    				subOccurances++;
    			}
    		}
    	}
    	if (subOccurances > userOccurances)
    	{
    		break;
    	}
    }
    if (subOccurances > userOccurances)
    {
    	Messages ms = new Messages("FOUND", "ScreenShot");
    	ms.Show();
    }
    

    Or like this:

    foreach (var rec in Submissions)
    {
    	TimeSpan tsOcc = rec.Date.TimeOfDay;
    	foreach (var item in Properties.Settings.Default.AppListSel)
    	{
    		if (rec.System == item)
    		{
    			if ((tsOcc > tsStart) && (tsOcc < tsNow))
    			{
    				subOccurances++;
    			}
    		}
    	}
    	if (subOccurances > userOccurances)
    	{
    		Messages ms = new Messages("FOUND", "ScreenShot");
    		ms.Show();
    		break;
    	}
    }
    


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Thursday, May 18, 2017 5:10 AM
    Moderator
  • Thank you, Ill play around with those suggestions and see how things work out..

    The Message line in my example was merely for me to see that the logic was working when I ran it outside of VS. That will not be in the final build.

    Now to post my question on the filesystemwatcher, that is how I'm currently monitoring the file that is being updated. With all the logic above the end goal is to only notify the user that has enabled notifications AND where their conditions(Occurrences and Intervals) are met.

    Would anyone know if I can post that in this forum? or under another forum section? This is just a brief portion of the current logic that is doing the notifications

            public static bool FlashWindowEx(Form form)
            {
                IntPtr hWnd = form.Handle;
                FLASHWINFO fInfo = new FLASHWINFO();
    
                fInfo.cbSize = Convert.ToUInt32(Marshal.SizeOf(fInfo));
                fInfo.hwnd = hWnd;
                fInfo.dwFlags = FLASHW_ALL | FLASHW_TIMERNOFG;
                fInfo.uCount = UInt32.MaxValue;
                fInfo.dwTimeout = 0;
    
                return FlashWindowEx(ref fInfo);
            }
    
            private void fsw_Changed(object sender, FileSystemEventArgs e)
            {
                FlashWindowEx(this);
            }
    
            private void fsw_Renamed(object sender, RenamedEventArgs e)
            {
                FlashWindowEx(this);
            }
    I'm thinking that with Bonnies suggested changes, I would write my conditions to call that method within the changed and renamed events and if met, then call the FlashWindowEx

    Friday, May 19, 2017 2:41 AM
  • Normally questions that are specific to WinForms would go in the WinForm forum:

    https://social.msdn.microsoft.com/Forums/windows/en-US/home?forum=winforms

    So, you could re-post your question about FlashWindowEx there if you want to, although it isn't a specific WinForm question and so this C# forum is probably fine for this question (although you should start a new thread).

    So, I Googled and I found a thread on StackOverflow about this that might help:

    http://stackoverflow.com/questions/4889181/get-user-attention-without-stealing-focus

    I haven't actually tried your FlashWindowEx code to see if I could make it work, but it's similar enough to that StackOverflow answer, so it's probably fine. Bottom line is that if you're having problems with that particular piece of code, then yeah, just post a new question, and here in the C# forum is probably OK. Even though I'm a Moderator, I don't know what all of the MSDN forums are, so there may be a more appropriate one. If so, someone will move your new question there.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Saturday, May 20, 2017 4:24 PM
    Moderator
  • Bonnie you are awesome.. after posting my question about the logic around the flashing code, I set it up and works.. I created my method to return bool and wrapped the flashing logic around the call to the flash like so.. works like a charm.. and the link you provided above was one of the links I had found and used as reference when I starting working on the notifications.

            private void fsw_Changed(object sender, FileSystemEventArgs e)
            {
                if (pcl.SubmissionNotifications())
                {
                    FlashWindowEx(this);
                }
            }

    Thank you so much for all the assistance.. Demo went great Friday, a lot more were very impressed with how the over all application worked.. and also got approvals from few managers and VP's, With the above flash logic working(at home), ill be able to start testing at work Monday and see how things work out..

    I truly appreciate the help with all my questions..

    Sunday, May 21, 2017 1:47 AM
  • You're welcome! I'm happy to help! I'm glad your demo was approved by the "higher-ups". I know how stressful that can be!!! Especially if they *don't* like it!!  But yours did ... Yay!  =0)

    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Sunday, May 21, 2017 2:29 PM
    Moderator
  • Well Bonnie im back, figured id start here before i start another post.. 

    So application has been running and working great, rolled out to about 30 people so far, was approved to roll out to about 80 more.. 

    BUT one thing that was noticed and we just tested and had mixed results..

    So there are about 5 of us that have the notification functionality enabled.. all of which are set to 3 occurrences within a 5 minute window. As far as i can tell it seems that only 1 or 2 of use will get notified when those thresholds are met and some never get notified.

    Are you familiar or anyone else that sees this familiar with the filesystemwatcher that would know if the reason that only 1 or 2 get notified is that its a first come first serve? I thought that the filesystemwatcher is unique to the users pc in that its different instances running so each would do their own monitoring.. I need help to identify if its just the way the call works and they just need to designate one person or if it can be adjusted/updated to allow any number of users to be notified.

    thank you in advance.

    Wednesday, July 12, 2017 9:17 PM
  • im just wondering if by chance when the filewatcher pulls in the psv values to loop thru if that is causing the other not to be notified since the file is not available to check against or if its something else.. 
    Wednesday, July 12, 2017 9:18 PM
  • Ive posted a new thread for some modifications I need for this logic..

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/51d49718-ff42-4e67-b26e-7170a657cb80/cant-seem-to-capture-the-correct-information-to-notify-user?forum=csharpgeneral

    If anyone can help, please let me know.. I'm trying to make some improvements this weekend before the new work week.

    Sunday, July 16, 2017 7:48 AM
  • Sorry I missed your questions from a few days ago. But, I saw your new question in your new thread and responded there today. I hope it helps ...

    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Sunday, July 16, 2017 10:58 PM
    Moderator