none
200 byte records exceeding 4MB limit?!?

    Question

  • Hi all,

    Having a problem when submitting a U-SQL in Visual Studio using a custom (fixed record length) extractor. Error says the file record is too long (exceeds 4MB) but each of the file's records are nowhere near that size (each record is no more than ~200 bytes).

    It looks like the records are terminated with CR/LF. I'm confused, what gives? Any advise?

    Thanks,

    Paul

    Saturday, August 11, 2018 9:56 PM

Answers

  • Hi again,

    Update: found out the character set/file format of the input file needed to be changed. I use TextPad as a text editor and looking at the file properties, the file format was showing as "UNIX". I looked at another file I know was working successfully  and saw the file format as "PC" so I changed this other file accordingly and all is well! I think it affected the CR/LF and, as such, was likely being ignored hence exceeding the 4MB limit. Just posting here in case it may help someone in the future.

    Best regards,

    Paul


    Sunday, August 12, 2018 12:45 AM