locked
Is it possible to execute package tasks in parallel with different package parameters? RRS feed

  • Question

  • I want to run a package in parallel with different start and enddates instead of one at a time in a loop.

    I found cozyroc parallel loop task, but it cant seem to cope with package parameters, complaining that it was not found in the Variables collection.

    This seems to me should be a fairly common problem, are there any standard ways to dealing with this?

    Thanks

    Jon

    Monday, October 17, 2016 5:57 PM

Answers

  • Hi Jon,

    It is rather uncommon in my view. Parallel executions can either cause grief on the data processing end or the machine resources, especially when uncontrolled.

    I suggest you ask CozyRoc support to help you.

    Using the stock SSIS tasks and components, you would need to likely utilize the Async nature of the Agent - create as many jobs each with a package set to run with its own date range and trigger them all at once.


    Arthur

    MyBlog


    Twitter

    • Proposed as answer by Seif Wang Tuesday, October 18, 2016 3:08 AM
    • Marked as answer by JLJ1976 Thursday, October 27, 2016 4:55 PM
    Monday, October 17, 2016 7:17 PM
  • What do your packages do? We run the same package in parallel and pass in a GroupID. Then we have a table (tablelist) of table names and corresponding groupids.  So each package only processes the tables from tablelist WHERE GROUPID = @GroupID. It helps improve performance, but it is not a cure all. Most of our packages time is spent in NETWORK_IO. As we are loading data from iSeries into SQL.
    • Marked as answer by JLJ1976 Thursday, October 27, 2016 4:55 PM
    Monday, October 17, 2016 9:57 PM

All replies

  • Hi Jon,

    It is rather uncommon in my view. Parallel executions can either cause grief on the data processing end or the machine resources, especially when uncontrolled.

    I suggest you ask CozyRoc support to help you.

    Using the stock SSIS tasks and components, you would need to likely utilize the Async nature of the Agent - create as many jobs each with a package set to run with its own date range and trigger them all at once.


    Arthur

    MyBlog


    Twitter

    • Proposed as answer by Seif Wang Tuesday, October 18, 2016 3:08 AM
    • Marked as answer by JLJ1976 Thursday, October 27, 2016 4:55 PM
    Monday, October 17, 2016 7:17 PM
  • What do your packages do? We run the same package in parallel and pass in a GroupID. Then we have a table (tablelist) of table names and corresponding groupids.  So each package only processes the tables from tablelist WHERE GROUPID = @GroupID. It helps improve performance, but it is not a cure all. Most of our packages time is spent in NETWORK_IO. As we are loading data from iSeries into SQL.
    • Marked as answer by JLJ1976 Thursday, October 27, 2016 4:55 PM
    Monday, October 17, 2016 9:57 PM