none
Reading values from a configuration file for use in an F# script

    Question

  • Hi All,
    I recently wrote my first useful F# script (a SQL error log parser) and at its core it does exactly what I want but there is an issue.  In it I hard coded some values that I would like to read from a configuration file instead of hard coding.  Being a complete novice at F# I am stuck.  Any help would be appreciated.  First off, the hard coded portion that I want to replace looks like this.
    let Input_Stream = File.OpenText(@"C:\Temp\ERRORLOG")
    let Output_Stream = File.CreateText(@"C:\Temp\Stack_Dump_Times.txt")
    let From_Date = Convert.ToDateTime("2011-05-16 00:00:00.00")
    let Thru_Date = Convert.ToDateTime("2011-08-18 00:00:00.00")
    let includes = "Using 'dbghelp.dll' version '4.0.5'"
    let excludes = "Log was backed up"

    What I've tried is below.  Please try not to laugh too hard.
    let input_File = File.OpenText(@"C:\A_Scripts\F#\SQL_Error_Log_Parser\SQL_Error_Log_Parser\bin\Input_Parameters.lst");;
    
    let split_Pattern = new Regex(",");;
    let my_Params (some_Text : string) = split_Pattern.Split(some_Text)
    let nvp (some_Text : string ) = my_Params some_Text
    
    let Intput_Steam (the_Value : string) = File.OpenText(the_Value)
    let Output_Stream (the_Value : string) = File.CreateText(the_Value)
    let From_Date (the_Value : string) = Convert.ToDateTime(the_Value)
    let Thru_Date (the_Value : string) = Convert.ToDateTime(the_Value)
    let includes (the_Value : string) = the_Value
    let excludes (the_Value : string) = the_Value
    
    let param_Match (the_name:string, the_value:string ) = 
     match the_name with
     | "Input_Stream" -> Intput_Steam the_value
     | "Output_Stream" -> Output_Stream the_value
     | "From_Date" -> From_Date the_value
     | "Thru_Date" -> Thru_Date the_value
     | "includes" -> includes the_value
     | "excludes" -> excludes the_value
     | ()
    
    while not input_File.EndOfStream do
     let line_Of_Text = input_File.ReadLine()
     let nvp = my_Params line_Of_Text
     param_Match nvp.[0] nvp.[1]
    input_File.Close();;
    With the code like this I have one error.  The while is highlighted and I get the message, "Incomplete structured construct at or before this point in pattern matching. Expected '->' or other token."  Obviously the () at the end of my match is wrong.
     
    If I try something like
    nm -> failwith (sprintf "skipping unexpected name %s" nm)

    instead of the () I have many errors. 
    All of the match expressions except the first are in error like "This expression was expected to have type     StreamReader     but here has type     StreamWriter"
    The evaluation of param_Match is now highlighed with this error "This value is not a function and cannot be applied"
    Is anyone on this forum willing to guide me on something that should be pretty simple?  Reading through all of your post mine seems so trivial that I am embarrased to even post it but I'm really stuck and have no clue about this.  I'm just a DBA who would like to replace some PowerShell scripts with some F# scripts where the PowerShell scripts don't really work well such as parsing a 36 GB SQL error log.

    Joe Moyle

    Wednesday, August 17, 2011 8:36 PM

Answers

  • Joe and I disussed this offline, here was my suggestion (I haven't actually run the code :-))

     

    open System
    open System.IO
    
    let Start = DateTime.Now
    let input_File = File.OpenText(@"C:\A_Scripts\F#\SQL_Error_Log_Parser\SQL_Error_Log_Parser\bin\Input_Parameters.lst")
    
    // Build a dictionary mapping parameter names (strings) to parameter values (also strings)
    let parameters = 
      dict [ while not input_File.EndOfStream do
           let line_Of_Text = input_File.ReadLine()
           let nvp = line_Of_Text.Split(',')
           yield (nvp.[0], nvp.[1]) ]
    
    // Extract the expected parameters. If any are missing an exception will be raised
    let Input_Stream = File.OpenText(parameters.["Input_Stream"])
    let Output_Stream = File.CreateText(parameters.["Output_Stream"])
    let From_Date = Convert.ToDateTime(parameters.["From_Date"])
    let Thru_Date = Convert.ToDateTime(parameters.["Thru_Date"])
    let includes = parameters.["includes"]
    let excludes = parameters.["excludes"]
    
    

    • Marked as answer by Joe Moyle Thursday, August 18, 2011 1:27 AM
    Wednesday, August 17, 2011 11:05 PM

All replies

  • Hi Joe,
    Welcome!  The basic problem that the compiler has identified is that a match statement should consist of a bunch of cases which each evaluate to values of the same type.  However, in your code, the different cases evaluate to different types.  For example, given "Input_Stream", you are calling the Input_Stream function on the_value, which returns a StreamReader, but given "Output_Stream" you are calling the Output_Stream function on the_value, which returns a StreamWriter, and these types aren't compatible.
     
    However, I think what's really going wrong is that you want Input_Stream, Output_Stream, etc. to be values which get set during your loop, but the way they are defined now, they are functions.  Instead, I'd recommend structuring your code more like this:
    let mutable Intput_Steam = Unchecked.defaultof<_>
    let mutable Output_Stream = Unchecked.defaultof<_>
    let mutable From_Date = Unchecked.defaultof<_>
    let mutable Thru_Date = Unchecked.defaultof<_>
    let mutable includes = Unchecked.defaultof<_>
    let mutable excludes = Unchecked.defaultof<_>
    
    let param_Match (the_name:string, the_value:string ) = 
      match the_name with
      | "Input_Stream" -> Intput_Steam <- File.OpenText(the_value)
      | "Output_Stream" -> Output_Stream <- File.CreateText(the_value)
      | "From_Date" -> From_Date <- Convert.ToDateTime(the_value)
      | "Thru_Date" -> Thru_Date <- Convert.ToDateTime(the_value)
      | "includes" -> includes <- the_value
      | "excludes" -> excludes <- the_value
      | nm -> failwith (sprintf "skipping unexpected name %s" nm)

    Here I've defined variables (and initialized them to the default values for their respective types), and now the param_Match function updates the corresponding variable when it is found.  Now each branch has the same type (unit, which basically means that a statement has been executed for a side effect and no meaningful value is returned).
     
    Hope that helps,
    Keith


    Wednesday, August 17, 2011 9:18 PM
    Moderator
  • Joe and I disussed this offline, here was my suggestion (I haven't actually run the code :-))

     

    open System
    open System.IO
    
    let Start = DateTime.Now
    let input_File = File.OpenText(@"C:\A_Scripts\F#\SQL_Error_Log_Parser\SQL_Error_Log_Parser\bin\Input_Parameters.lst")
    
    // Build a dictionary mapping parameter names (strings) to parameter values (also strings)
    let parameters = 
      dict [ while not input_File.EndOfStream do
           let line_Of_Text = input_File.ReadLine()
           let nvp = line_Of_Text.Split(',')
           yield (nvp.[0], nvp.[1]) ]
    
    // Extract the expected parameters. If any are missing an exception will be raised
    let Input_Stream = File.OpenText(parameters.["Input_Stream"])
    let Output_Stream = File.CreateText(parameters.["Output_Stream"])
    let From_Date = Convert.ToDateTime(parameters.["From_Date"])
    let Thru_Date = Convert.ToDateTime(parameters.["Thru_Date"])
    let includes = parameters.["includes"]
    let excludes = parameters.["excludes"]
    
    

    • Marked as answer by Joe Moyle Thursday, August 18, 2011 1:27 AM
    Wednesday, August 17, 2011 11:05 PM
  • Hi Keith,

     

    Thank you for taking the time to reply.  I haven't had a chance to try your idea yet.  As you can see in Don's update he also assited me in an email and I'm going to try Don's suggested route first simply because I'm trying to avoid using mutable items.  Not that they are bad or anything.  I'm just trying to learn how to do it without them having always scripted in Perl and PowerShell it is forcing me to think differently.

     

    Hi Don,

    I did try your approach but I am getting an error.  I'm going to experiment with what you have shown me in an attempt to learn some more and solve as much as possible for myself.  So, I'm holding off on giving more details about the error I'm getting on purpose for now.

     

    To All Forum Participants,

    I'll update this thread with either success including exactly how it was done or more cries for help.  Please give me a few days to experiment in my spare time unless you know the solution for sure and are dying to share another way to skin this cat.


    Joe Moyle
    Wednesday, August 17, 2011 11:41 PM
  • Hi Joe,

    I wanted to give you an answer which addressed why you were seeing the error that you were and which made only minimal changes to the structure of your code to fix it.  However, I think Don's answer is much better - having so many mutable variables is definitely a "code smell" in F#, and his code is easier to understand as well.

     

    -Keith

    Thursday, August 18, 2011 1:23 AM
    Moderator
  • Hi All,

    Don's answer did work.  The errors I ran into after attempting to implement it revolved around escaping double quotes.  Addressing the value by the key seems to need double quotes.  My entries in my config file had double quotes around them.  So it was failing because Input_Steam does not equal "Input_Stream".  To same myself some time I simply removed the double quotes around the keys and values in my config file and all worked.

     


    Joe Moyle
    Friday, August 19, 2011 1:56 AM