After doing some work today on pushing a large chunk of data through the pipeline it occurred to me that it would be great to be able to measure throughput (rows/time). Namely, in 2 places: 1. Throughput of a data-flow (admittedly this is available via a calculation using a rowcount component and the System::ContainerStartTime variable that I know is coming soon) 2. Throughput of a data-flow component
#2 would be invaluable for finding bottlenecks in a data-flow which is really what I could have done with today.
My idea for how to accomplish this would be to fire an OnInformation event containing the information on completion of the data-flow/data-flow component. That's the easy bit I suppose, actually capturing the throughput to make it available to the OnInformation event would I guess be the hard bit.
In the case of #1, perhaps this could be done by the TaskHost container??? Doesnt help with #2 though which is what I really want.
Good suggestions Jamie. Number 1 is an interesting problem, and a couple of approaches come to mind. The taskhost one isn't one of them though, since the task host doesn't even really know about the pipeline. As far as it's concerned, the pipeline is just another task. That's by design. Number 2 is an interesting approach, and it looks workable too. This is the kind of thing I hope we can take a closer look at in coming versions. Did you file a DCR for this? If not, please do, then we can track it and consider it when we're looking at features for V.Next. Thanks, K
