none
Run script tasks in parallel RRS feed

  • Question

  • We have a SSIS package to extract data from multiple iSeries files and load into SQL Server. This package creates and executes another child package which will have multiple script tasks each one to extract data from one iSeries file and load to sql server. So if it needs to get data data from 20 files then it creates a child package with 20 script tasks and executes that child package. MaxConcurrentExecutables is set to 6 and we are using cwbx dll mentioned in the below article in each script task to extract data from iSeries. What we observed is only file file is getting extracted at a time even after setting MaxConcurrentExecutables to 6. Is there any setting that can be used make the multiple script tasks run in parallel and extract data at the same time? Please advice.

    http://blog.stevienova.com/2009/05/20/etl-method-fastest-way-to-get-data-from-db2-to-microsoft-sql-server/


    srinivas
    Friday, January 27, 2012 11:16 PM

Answers

  • We have a SSIS package to extract data from multiple iSeries files and load into SQL Server. This package creates and executes another child package which will have multiple script tasks each one to extract data from one iSeries file and load to sql server. So if it needs to get data data from 20 files then it creates a child package with 20 script tasks and executes that child package. MaxConcurrentExecutables is set to 6 and we are using cwbx dll mentioned in the below article in each script task to extract data from iSeries. What we observed is only file file is getting [do u mena only 5 files ...]

    extracted at a time even after setting MaxConcurrentExecutables to 6. Is there any setting that can be used make the multiple script tasks run in parallel and extract data at the same time? Please advice.

    http://blog.stevienova.com/2009/05/20/etl-method-fastest-way-to-get-data-from-db2-to-microsoft-sql-server/


    srinivas


    HI Srinivas

    What are the number of processors in your machine ?

    is the server dedicated for ssis ? If not please dont play with the property or else you will run into resources problems unless the value you are setting is <= [number of phy or logical processors + 2]

    neways @ question simply putting the tasks in paralel without precidence constraints as ramjaddu already said be is sequence container or without it should do it for you.

    6 is a small number you are playing with though but it all depends on ur processor count and resources availibilities ... remember your called control flow tasks are calling other packages as well ...  


    Hope that helps ... Kunal
    • Marked as answer by Eileen Zhao Tuesday, June 19, 2012 2:15 AM
    Monday, January 30, 2012 6:49 AM

All replies

  • You can use sequence container to execute tasks parallel by place them on container with out any precedence constraints.
    http://uk.linkedin.com/in/ramjaddu
    Saturday, January 28, 2012 6:04 PM
  • We have a SSIS package to extract data from multiple iSeries files and load into SQL Server. This package creates and executes another child package which will have multiple script tasks each one to extract data from one iSeries file and load to sql server. So if it needs to get data data from 20 files then it creates a child package with 20 script tasks and executes that child package. MaxConcurrentExecutables is set to 6 and we are using cwbx dll mentioned in the below article in each script task to extract data from iSeries. What we observed is only file file is getting [do u mena only 5 files ...]

    extracted at a time even after setting MaxConcurrentExecutables to 6. Is there any setting that can be used make the multiple script tasks run in parallel and extract data at the same time? Please advice.

    http://blog.stevienova.com/2009/05/20/etl-method-fastest-way-to-get-data-from-db2-to-microsoft-sql-server/


    srinivas


    HI Srinivas

    What are the number of processors in your machine ?

    is the server dedicated for ssis ? If not please dont play with the property or else you will run into resources problems unless the value you are setting is <= [number of phy or logical processors + 2]

    neways @ question simply putting the tasks in paralel without precidence constraints as ramjaddu already said be is sequence container or without it should do it for you.

    6 is a small number you are playing with though but it all depends on ur processor count and resources availibilities ... remember your called control flow tasks are calling other packages as well ...  


    Hope that helps ... Kunal
    • Marked as answer by Eileen Zhao Tuesday, June 19, 2012 2:15 AM
    Monday, January 30, 2012 6:49 AM