none
R Script - Partition and run parallely RRS feed

  • Question

  • To improve processing speed, I'm trying split my rows into parallel and execute multiple instance of my R code within same service.

    I have couple of problems here :

    1. One of the R Script step fails if there is less than 3 records. Assume one step still executed although it shouldn't have any records passed. Error :

    Execute R Script Piped (RPackage) : The following error occurred during evaluation of R script: R_tryEval: return error: Error in if (start > end) stop("'start' cannot be after 'end'") : missing value where TRUE/FALSE needed

    2. I passed 10 rows, and expecting output 10 as well. However it returned 20 - where 10 are duplicated. Hence somehow the SQL transformation doesn't seems to be working.

    Please refer to individual step settings :

    


    R.S.S.

    Monday, October 14, 2019 2:13 PM

All replies

  • Figured out that issue 1 laid with my code, hence had to singly handle no passing of records. Anyway, I was expecting that specific action should not run if upstream has no data.

    On issue 2, unticking Pick complement of the selected fold in the parallel action seems to avoid the duplicates.

    But now I end up with another error in SQL Transformation where I'm doing union of these tables. It will fail, if any of the table is empty :

    requestId = 850cc0e2f5eb41948e2ca7b642d7413b errorComponent=Module. taskStatusCode=400. {"Exception":{"ErrorId":"TooFewColumnsInDataset","ErrorCode":"0020","ExceptionType":"ModuleException","Message":"Error 0020: Number of columns in input dataset \"table2\" is less than allowed minimum of 1 column(s)."}}Error: Error 0020: Number of columns in input dataset "table2" is less than allowed minimum of 1 column(s). Process exited with error code -2


    R.S.S.

    Tuesday, October 15, 2019 1:43 AM
  • Hi R.

    Sorry for the delay. I want to check if you are still suffering from this issue now? 

    Regards,

    Yutong

    Monday, November 4, 2019 3:34 AM
    Moderator