none
Streamwriter in VB 2005

    Question

  • This SHOULD be easy.  I must be missing something.  I'm assembling a long string and trying to write it out to a file.  If I display it in a message box or textbox, it looks fine.  But when I write it out to a file, it gets chopped off.  I thought it might be my data, because it is supposed to be HTML, but it seems to be related to the amount of data, because it chops off at about the same number of lines no matter what the data.  I tried breaking it into smaller strings and doing two writelines, but it still does the same thing.

    lname = names(1)
    outFile =
    "D:\Class of 57\updatehtm\" & lname & ".htm"
    output = New FileStream(outFile, FileMode.OpenOrCreate, FileAccess.Write)
    fileWriter =
    New StreamWriter(output)
    outstring =
    "<!DOCTYPE HTML PUBLIC ""-//W3C//DTD HTML 4.01 Transitional//EN"" > " & vbCrLf
    outstring &=
    "<html>" & vbCrLf
    outstring &=
    "<head>" & vbCrLf

    ' more additions to string

    fileWriter.WriteLine(outstring)

    I have the same problem, even if I change the file extension to ".txt" rather than ".htm"  It writes about 28 lines of text. If I replace the "filewriter.writeline" with "msgbox" it shows the whole string.

     

    Ron DelPorto

    Monday, August 28, 2006 9:20 PM

Answers

  • Try wrapping the file stream and stream writer implementations inside using blocks.

    The FileStream object uses "buffered IO" to make its writes to disk efficent. As a result, it doesn't write out to disk with each you make a  call to write, but only writes out to disk once its buffer fills up. If you finish writing to a file stream and have not completly filled up the buffer, its contents will not be flushed to the disk untill the file stream is closed. If you do not explicitly close it, then this will not happen untill the garbage collector reclaims the object. If you wrap the file stream and stream writer implementations in using blocks then they will be disposed, and henced closed, once the code in the using block has completed.

    Try something like this:

    using fs as new FileStream(...), sw as new streamWriter(fs)
        'Put your code here
    end using

    That may do the trick.

    -Scott Wisniewski

    Tuesday, August 29, 2006 12:45 AM
    Moderator

All replies

  • interesting. what does it show when it "chops" off?

    I would also consider using the StringBuilder to see if this helps, and is best practice to use StringBuilder for any string jobs:

     

    Dim theStringBuilder as new System.Text.StringBuilder()

    theStringBuilder.Append(yourStringHere)

     

    fileWriter.WriteLine(theStringBuilder.ToString())

     

    what happens when you use this?

    Monday, August 28, 2006 11:17 PM
    Moderator
  • I would reccomend against using an intermediate StringBuilder if you are using a StreamWriter, as it will produce worse performance than if you just made Write and WriteLine calls on the StreamWriter directory.

    However, I would also reccomend that you not use a string concatination with a temporary string object.

    For example, rather than doing:

    outputstring = outputstring & "<head>"
    outputstring = outputstring & "</head>"
    '....
    sw.Write(outputstring)

    I would reccomend that you do this:

    sw.Write("<head>")
    sw.Write("</head>")

    With the stream writer, particularly because you are outputing to a file, you do not need to "buffer" the contents in memory inside a string builder. Instead its better to just write the contents out to the file directly. The OS and the file stream implementation will take care of buffering the file writes for you so that they are efficent.

    -Scott

    Tuesday, August 29, 2006 12:36 AM
    Moderator
  • Try wrapping the file stream and stream writer implementations inside using blocks.

    The FileStream object uses "buffered IO" to make its writes to disk efficent. As a result, it doesn't write out to disk with each you make a  call to write, but only writes out to disk once its buffer fills up. If you finish writing to a file stream and have not completly filled up the buffer, its contents will not be flushed to the disk untill the file stream is closed. If you do not explicitly close it, then this will not happen untill the garbage collector reclaims the object. If you wrap the file stream and stream writer implementations in using blocks then they will be disposed, and henced closed, once the code in the using block has completed.

    Try something like this:

    using fs as new FileStream(...), sw as new streamWriter(fs)
        'Put your code here
    end using

    That may do the trick.

    -Scott Wisniewski

    Tuesday, August 29, 2006 12:45 AM
    Moderator
  • Thanks Scott for that, you learn something new everyday :-) Could I ask, how can the stringbuilder in this case perhaps cause poor performance? What's the reason for it?
    Tuesday, August 29, 2006 12:47 AM
    Moderator
  • Scott,

      Thanks for the prompt reply and for the suggestions.  When I try:

    Using output As New FileStream(outFile, FileMode.OpenOrCreate, FileAccess.Write), _
      fileWriter =
    New StreamWriter(output)
            fileWriter.WriteLine(outstring)

    End Using

    I get the errors:

    Error 1 Variable 'output' hides a variable in an enclosing block. 
    Error 2 Variable 'fileWriter' hides a variable in an enclosing block. 

    Did I code that incorrectly?  Meantime I will try the direct write instead of accumulating everything in a string.

    Ron

    Tuesday, August 29, 2006 1:05 AM
  • You should replace the existing variables you declared in your function with the using declarations.

    For example if you had:

    dim output = new FileStream(...)
    dim fileWriter = new StreamWriter(output)

    '<insert coe here>

    you should replace it with the following

    using output = new FileStream(...), fileWriter = new StreamWriter(output)

       '<insert code here>
    end using

    Your probably are getting the compile errors because you forgot to remove the existing declarations.

    -Scott

    Tuesday, August 29, 2006 1:17 AM
    Moderator
  • Scott,

       THANK YOU!  That worked.  I'm still kind of astonished that I can't just write to the file the way I did in the first place, but it DID work this way and I learned something. BTW, the direct write, eg:

    filewriter.write("<html>")

    etc

    did NOT work.  It still stopped after about 28 lines.

    Thank you for being so willing to help.  Everytime I think I really understand VB.Net, I get some kind of comeuppance.

    Ron

    Tuesday, August 29, 2006 1:40 AM
  • I will agree with scott that if you are going to use the streamwriter then You dont need to use stringbuilder than if you
    simply use write and writeline calls.

    However if you dont want to use the streamwriter and want a simple single line to write the file contents then you can use a simple call to the My.computer.filesystem.writealltext method.

    Dim theStringBuilder as new System.Text.StringBuilder()
    theStringBuilder.Append(<yourStringHere>)
    My.Computer.Filesystem.WriteAllText(<yourfilename>,thestringbuilder, false)

    Will create a simple text document with the contents of you string.   Writing the text is a single line specifying the filename and the contents.   Using the stringbuilder simply to create the contents to be written to the file.

    You dont have to use the stringbuilder but otherwise for every line you append to a string a new string is created to take into account the appended details - so it will consume a bit more more memory - but eventually the garbage collection process will recover the old unreferencable strings when it is running short of resources.    Stringbuilder doesnt create this waste.

     

     

    Tuesday, August 29, 2006 1:50 AM
  • Every string in the CLR is an exactly sized array of characters. A string of length 50 has enough memory to store its length and 50 Unicode characters and no more. They have the constraint that the contents of the string occupy one continuous block of memory.

    Strings in the CLR are also immutable, meaning there contents can never be changed. The combination of these two properties allow most string operations (charAt, subString, indexOf, etc) to be implemented efficiently.

    However because strings cannot be changed, any time 2 strings are concatenated a third string has to be created to store the results of the concatenation.  this requires allocating a new "memory buffer" large enough to hold both strings, and then copying the two original strings into the new buffer. This means that if you are concatenating strings in a loop, that you can end up creating a lot of temporary strings and doing a lot of copying. This is wasteful of both memory and space.

    If, however, you have a "mutable" buffer into which strings can be appended, and you know ahead of time how much space you need, you can avoid generating temporary strings and doing unnecessary copies by inserting characters into the mutable buffer and then doing one copy into an "immutable" string when you are done.

    In a lot of circumstances, however, it is not always possible to know exactly how much memory one needs when doing string concatenations in a loop. The string builder handles these cases too, however, by using an algorithm that allows it to auto expand the size of the buffer when necessary. It does this in a way that keeps the amount of copying it has to do to a minimum. It does this by sacrificing memory utilization in the name of improved CPU time (unlike with regular strings, there will be times when a string builder uses more memory than it actually needs). However, the implementation is also designed to reduce this waste as much as possible when using extremely large strings. This provides for significant performance benefits over regular string concatenation.

    Data stored in files on disk, including text files, does not have the requirement of always being in one continuous area on disk, like strings do in memory. Instead the file system breaks the file up into smaller continuous chunks called "blocks" which it then organizes on the disk at sees fit. When your program writes data to a file stream, the stream / OS maintains a buffer that is some integer multiple of the block size (i.e one or more blocks in size). When the buffer gets full, the OS flushes the buffer out to the disk, and then clears the buffer and reuses it for your next write operations. 

    If you use a string builder on top of the file stream, you end up just creating extra copies of the string data in memory that aren't really needed. The string builder keeps the number of extra copies to a minimum, but it still needs to use extra memory and do extra copies. Because the file has no requirement that all of its contents fit into one continuous block of memory at the same time, there is no need to incur the space overhead of the string builder. In that case it is much more efficient just to output the data to the file as your process it

    -Scott Wisniewski

    Tuesday, August 29, 2006 1:57 AM
    Moderator
  • This is how I use the streamwriter

    Imports System.Io

    dim sw as streamwriter

    sw = file.appendtext("<filename>")

    sw.writeline "<!DOCTYPE HTML PUBLIC ""-//W3C//DTD HTML 4.01 Transitional//EN"" >

    sw.close

     

    Tuesday, August 29, 2006 12:53 PM
  • I am also having problems writing a long string to a file. Is there an underlying limitation on the length of a string that can be written to a file using streamwriter or textwriter/?

     

    I later found that thread safety is really an issue with this. I could not write files in instances of the same class without protecting the writes by a mutex. Each instance did not get an independent streamwriter. Length just seems to expose the writer longer making it morelikely that an interloper will arrive & screw things up.

     

     

     

    Monday, October 15, 2007 6:59 PM