none
How to parse logfile based on common reference RRS feed

  • Question

  • I'm trying to sort a lot of statistics output from many firewalls, ideally producing a list by firewall name, then statistic count for that firewall.  For reference I'm using C#.

    However, I'm stumbling at the starting block, and can't figure out how to copy all lines until newline, starting with a known common reference.  This would then be stored to a list and I'd imagine key referenced by firewall name.

    I must admit, rather lost in how to approach this.  Some details on the type of data however:
    ---------------------------------------------------------------------------------------------------
    SW02FWDNS1
    /system/ha/ifaces/if1/hbeat_fail_out  0                heartbeats
    /system/ha/ifaces/if1/hbeat_in          0                heartbeats
    /system/ha/ifaces/if1/hbeat_miss_in  0                heartbeats
    /system/ha/ifaces/if1/hbeat_out        13016036  heartbeats

    SW02FWDNS2
    /system/ha/ifaces/if1/hbeat_fail_out   0                 heartbeats
    /system/ha/ifaces/if1/hbeat_in           0                 heartbeats
    /system/ha/ifaces/if1/hbeat_miss_in   0                 heartbeats
    /system/ha/ifaces/if1/hbeat_out         29102390   heartbeats
    ---------------------------------------------------------------------------------------------------

    Of note, "SW02FWDNS1" is the firewall name I'm using to contain each set of statistics in a list.  "FW" is always used in all firewall names, so I'm thinking to use "FW" as a reference.  After the final "heartbeats" text, there is then a newline, which I figure can be used as a stop point.  From this, I'd then like to store all statistic output for a given firewall into its own object, which can then be referenced later (I guess by key index?)

    I'm hoping to create a list of all firewall names, each containing values for hbeat_in and hbeat_out.  For example:

    SW01FWDNS1
    hbeat_in = 0
    hbeat_out = 13016036


    • Edited by Techbart Wednesday, June 19, 2019 8:47 AM
    Wednesday, June 19, 2019 8:46 AM

Answers

  • Hello,

    The following may or may not work in regards to if the lines are tabbed or space delimited as I did it spaced delimited yet if they are tabbed then there would be minor adjustments.

    I placed a text file in the app folder and read the file using the following.

    using System.Collections.Generic;
    using System.IO;
    
    namespace WindowsFormsApp1
    {
        public class FileLog 
        {
            public List<Firewall> Read(string pFileName)
            {
                var walls = new List<Firewall>();
    
                var lines = File.ReadAllLines(pFileName);
    
                for (int index = 0; index < lines.Length; index++)
                {
                    if (lines[index].Contains("FW"))
                    {
                        var wall = new Firewall() {Name = lines[index],Items = new List<string>()};
    
                        if (lines[index].Contains("FW"))
                        {
    
                            walls.Add(wall);
    
                            for (int innerIndex = index + 1; innerIndex < lines.Length; innerIndex++)
                            {
                                if (lines[innerIndex].Contains("FW"))
                                {
                                    break;                               
                                }
                                if (!string.IsNullOrWhiteSpace(lines[innerIndex].Trim()))
                                {
                                    wall.Items.Add(lines[innerIndex]);
                                }
                            }
    
                        }
    
    
                    }
                }
    
                return walls;
            }
    
        }
    }

    The container class

    using System;
    using System.Collections.Generic;
    using System.Linq;
    
    namespace WindowsFormsApp1
    {
        public class Firewall
        {
            public string Name { get; set; }
            public List<string> Items { get; set; }
    
            public int HeartBeatIn()
            {
                if (Items.Count > 0)
                {
                    var temp = Items.FirstOrDefault(data => data.Contains("hbeat_in"));
                    var index = temp.IndexOf("heartbeats");
                    return Convert.ToInt32(temp.Substring(40, 8).TrimEnd());                
                }
                else
                {
                    return -1;
                }
            }
            public int HeartBeatOut()
            {
                if (Items.Count > 0)
                {
                    var temp = Items.FirstOrDefault(data => data.Contains("hbeat_out"));
                    var index = temp.IndexOf("heartbeats");
                    return Convert.ToInt32(temp.Substring(41, temp.Length - index).TrimEnd());
                }
                else
                {
                    return -1;
                }
            }
        }
    }
    

    In a form with a button

    using System;
    using System.ComponentModel;
    using System.Data;
    using System.Drawing;
    using System.IO;
    using System.Text;
    using System.Threading.Tasks;
    using System.Windows.Forms;
    
    namespace WindowsFormsApp1
    {
        public partial class Form1 : Form
        {
            public Form1()
            {
                InitializeComponent();
            }
    
            private void button1_Click(object sender, EventArgs e)
            {
                var fileName = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "TextFile1.txt");
                var ops = new FileLog();
                var wallFirewalls = ops.Read(fileName);
                foreach (var wall in wallFirewalls)
                {
                    Console.WriteLine($"Name: {wall.Name} In: {wall.HeartBeatIn()} Out: {wall.HeartBeatOut()}");
                }
            }
        }
    }
    

    Results

    Name: SW02FWDNS1 In: 0 Out: 16036
    Name: SW02FWDNS2 In: 0 Out: 9102390


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Wednesday, June 19, 2019 10:37 AM
    Moderator

All replies

  • Hello,

    The following may or may not work in regards to if the lines are tabbed or space delimited as I did it spaced delimited yet if they are tabbed then there would be minor adjustments.

    I placed a text file in the app folder and read the file using the following.

    using System.Collections.Generic;
    using System.IO;
    
    namespace WindowsFormsApp1
    {
        public class FileLog 
        {
            public List<Firewall> Read(string pFileName)
            {
                var walls = new List<Firewall>();
    
                var lines = File.ReadAllLines(pFileName);
    
                for (int index = 0; index < lines.Length; index++)
                {
                    if (lines[index].Contains("FW"))
                    {
                        var wall = new Firewall() {Name = lines[index],Items = new List<string>()};
    
                        if (lines[index].Contains("FW"))
                        {
    
                            walls.Add(wall);
    
                            for (int innerIndex = index + 1; innerIndex < lines.Length; innerIndex++)
                            {
                                if (lines[innerIndex].Contains("FW"))
                                {
                                    break;                               
                                }
                                if (!string.IsNullOrWhiteSpace(lines[innerIndex].Trim()))
                                {
                                    wall.Items.Add(lines[innerIndex]);
                                }
                            }
    
                        }
    
    
                    }
                }
    
                return walls;
            }
    
        }
    }

    The container class

    using System;
    using System.Collections.Generic;
    using System.Linq;
    
    namespace WindowsFormsApp1
    {
        public class Firewall
        {
            public string Name { get; set; }
            public List<string> Items { get; set; }
    
            public int HeartBeatIn()
            {
                if (Items.Count > 0)
                {
                    var temp = Items.FirstOrDefault(data => data.Contains("hbeat_in"));
                    var index = temp.IndexOf("heartbeats");
                    return Convert.ToInt32(temp.Substring(40, 8).TrimEnd());                
                }
                else
                {
                    return -1;
                }
            }
            public int HeartBeatOut()
            {
                if (Items.Count > 0)
                {
                    var temp = Items.FirstOrDefault(data => data.Contains("hbeat_out"));
                    var index = temp.IndexOf("heartbeats");
                    return Convert.ToInt32(temp.Substring(41, temp.Length - index).TrimEnd());
                }
                else
                {
                    return -1;
                }
            }
        }
    }
    

    In a form with a button

    using System;
    using System.ComponentModel;
    using System.Data;
    using System.Drawing;
    using System.IO;
    using System.Text;
    using System.Threading.Tasks;
    using System.Windows.Forms;
    
    namespace WindowsFormsApp1
    {
        public partial class Form1 : Form
        {
            public Form1()
            {
                InitializeComponent();
            }
    
            private void button1_Click(object sender, EventArgs e)
            {
                var fileName = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "TextFile1.txt");
                var ops = new FileLog();
                var wallFirewalls = ops.Read(fileName);
                foreach (var wall in wallFirewalls)
                {
                    Console.WriteLine($"Name: {wall.Name} In: {wall.HeartBeatIn()} Out: {wall.HeartBeatOut()}");
                }
            }
        }
    }
    

    Results

    Name: SW02FWDNS1 In: 0 Out: 16036
    Name: SW02FWDNS2 In: 0 Out: 9102390


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Wednesday, June 19, 2019 10:37 AM
    Moderator
  • Hi Karen,

    Thanks very much for taking the time to look into this, much appreciated :).  Just wondering, could I ask for some tips on using web forms?  I should've mentioned in my initial post, but the intention was to use a console app, and as I haven't gotten into web forms yet, I'm a little lost on where to place code for these form elements.

    If I'm starting a new project, I guess I'd put the first section of code into Program.cs for this project, and for the second block of code to be put into a new class.  For the form with a button though, can I ask if this is done by adding a new web form to the existing project, or if I populate the default Form1.cs generated with the project?

    Cheers again! :)

    Wednesday, June 19, 2019 2:39 PM
  • Hi Techbart,

    You can do all of this in a Console app, you don't need to use a Windows Form as Karen posted (it wasn't a web form).

    The "form with a button" part of what she posted is simply a way to get the whole thing started in a Form. Since you're using a Console app, you'd just put the code that she posted in the button Click into your own app, probably in program.cs.


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Wednesday, June 19, 2019 3:35 PM
    Moderator
  • Well in say ASP.NET you basically subscribe to button click event back to the controller which in turn creates an instance of FileLog and calls Read. From there do what you wish e.g. analize results or do a post back to a web page.

    Bottom line is I used a Windows form project which is easy to show off the mechanics while a console project, WPF or any other project type that can read a text file (and dependent on physical location) can run the code presented.


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Wednesday, June 19, 2019 7:44 PM
    Moderator
  • Hi Techbart,

    Thank you for posting here.

    Based on your description, you want to parse logfile to get the information about the firewall.

    I think you could refer to karen's advice and if you want to do it in the console application, you could try the following code.

    Besides, I modified the bold code to make it correct.

    Code:

    namespace _1.Test1
    {
        public class FileLog
        {
            public List<Firewall> Read(string pFileName)
            {
                var walls = new List<Firewall>();
    
                var lines = File.ReadAllLines(pFileName);
    
                for (int index = 0; index < lines.Length; index++)
                {
                    if (lines[index].Contains("FW"))
                    {
                        var wall = new Firewall() { Name = lines[index], Items = new List<string>() };
    
                        if (lines[index].Contains("FW"))
                        {
    
                            walls.Add(wall);
    
                            for (int innerIndex = index + 1; innerIndex < lines.Length; innerIndex++)
                            {
                                if (lines[innerIndex].Contains("FW"))
                                {
                                    break;
                                }
                                if (!string.IsNullOrWhiteSpace(lines[innerIndex].Trim()))
                                {
                                    wall.Items.Add(lines[innerIndex]);
                                }
                            }
    
                        }
    
    
                    }
                }
    
                return walls;
            }
            public class Firewall
            {
                public string Name { get; set; }
                public List<string> Items { get; set; }
    
                public int HeartBeatIn()
                {
                    if (Items.Count > 0)
                    {
                        var temp = Items.FirstOrDefault(data => data.Contains("hbeat_in"));
                        var index = temp.IndexOf("heartbeats");
                        return Convert.ToInt32(temp.Substring(40, 8).TrimEnd());
                    }
                    else
                    {
                        return -1;
                    }
                }
                public int HeartBeatOut()
                {
                    if (Items.Count > 0)
                    {
                        var temp = Items.FirstOrDefault(data => data.Contains("hbeat_out"));
                        var index = temp.IndexOf("heartbeats");
                        return Convert.ToInt32(temp.Substring(39, temp.Length - index).TrimEnd());
                    }
                    else
                    {
                        return -1;
                    }
                }
            }
        }
        class Program
        {
            static void Main(string[] args)
            {
            var fileName = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"D:\1.txt");
            var ops = new FileLog();
            var wallFirewalls = ops.Read(fileName);
            foreach (var wall in wallFirewalls)
            {
                Console.WriteLine($"Name: {wall.Name} In: {wall.HeartBeatIn()} Out: {wall.HeartBeatOut()}");
            }
                Console.ReadKey();
            }
        }
    }
    

    Result:

    Best Regards,

    Jack


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, June 20, 2019 1:36 AM
    Moderator
  • Jack, as a Microsoft employee/contractor, you should know better: you should *NOT* propose your own reply as an answer. Especially since you used the code that Karen posted in her reply and tweaked it a bit.   This is not proper Forum etiquette!! 

    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com

    Tuesday, June 25, 2019 2:48 PM
    Moderator