USQL to extract ORC file format data from Azure Data Lake Store


  • Hi All,

    I'm new to .net world. I'm trying to reuse a 3rd party api created to extract data from ORC files using USQL.

    Link where the code is present -

    When i try to run the USQL job im getting the below error. Have i missed any assemblies etc. pls guide me through.

    i tried to run a usql script which will extract 3.5GB of compressed data(approx 81 million records) in ORC format into a csv file.

    when i run the usql script it gives the below error message.Can u pls help me out in this ??

    Unhandled exception from user code: "Capacity must be positive.
    Parameter name: capacity" The details includes more information including any inner exceptions and the stack trace where the exception was raised.
    Make sure the bug in the user code is fixed.
    ==== Caught exception System.ArgumentOutOfRangeException

    at System.IO.MemoryStream..ctor(Int32 capacity)
    at Orcneas.Core.OrcExtractor.GetSeekableStream(IUnstructuredReader input, SeekableInputStreamHackMode inputStreamHackMode)
    at Orcneas.Core.OrcExtractor.d__4.MoveNext()
    at ScopeEngine.SqlIpExtractorScopeEngine::CosmosInput,Extract_0_Data0.GetNextRow(SqlIpExtractorScopeEngine::CosmosInput\,Extract_0_Data0* , Extract_0_Data0* output) in d:\data\ccs\jobs\604c4a84-fa51-4cc8-9472-6dc480f13734_v0\sqlmanaged.h:line 1924
    An unhandled exception from user code has been reported when invoking the method 'Extract' on the user type 'Orcneas.Core.OrcExtractor'**



    Wednesday, September 6, 2017 2:02 PM