How to change the output schema in U-Sql IProcessor and IApplier


  • I want to know sample code in which I want to change the schema of output rowset in IProcessor and IApplier. In Scope (Cosmos) - I know that we have seperate method "Produces" which returns output schema. Do we have same method here or we have some different method? If I want to add some new columns or delete some existing columns coming from input schema, how I can do it?
    Sunday, July 17, 2016 6:02 AM

All replies

  • The schema model in U-SQL UDOs is different from Scope's. U-SQL does not execute user-code during compilation.

    Instead, you specify the schema in the PRODUCES clause in the PROCESS expression or in the table alias of an applier. The schema is then accessible inside the UDO.

    This however means you cannot do something like:

    @d = PROCESS @input PRODUCES * USING new MyRemoveColProcessor(removeCols = new SqlArray{"col3", "col4"});

    Instead you have to enumerate the columns you want to produce and use normal projections in SELECT clauses to drop columns from the rowset.

    Here is are some processor examples and the UDO code is here.

    Michael Rys

    Tuesday, July 19, 2016 9:26 PM