none
Cancellation on C++ AMP kernel?

    Question

  • Hi, does the current release of C++ AMP support cancelling a kernel being executed? If so, how to do it? If not, will this be supported in the future release? Thanks in advance.

    Wednesday, January 23, 2013 4:01 AM

Answers

  • Welp, abort() and exit() are examples of the sort of cancellation that you are looking for...at least when considered in a multi-threaded context:). Answering the question itself, no, such flexibility is not available for the time being. Consider that the GPU is an async device, that "eats" a command buffer that gets assembled on the CPU side, and that once it starts executing kernels are (typically) serialized. So there are two non-trivial issues to solve: one is that you'd have to package the cancellation call into a command buffer and pass it to the GPU - this will be latencious. Second issue is that you'd have to be able to preempt everyone and signal the cancellation - GPUs aren't necessarily very good with this as it stands, and the programming models don't expose it. Evolution is moving in that direction, but it will probably take a while before the hardware gets there, IMHO.
    Thursday, January 24, 2013 2:46 AM
  • How do you detect errors in the data? Do you mean you detect errors in the input data passed to the kernel, on the CPU, after the kernel has been dispatched? If so, as Alex mentioned in his earlier response, currently the operating system and GPU hardware do not support asynchronous cancellation of tasks queued for exection to the GPU.

    If the error is detected on the GPU itself, while executing the kernel, in theory this may be detected by communicating between threads within a tile or across tiles through a cancellation flag in global memory (or tile_static memory for communicating between threads within a tile) and use of atomic operations for reading/writing the flag. Having said that, atomic operations are very expensive and such a cancellation mechanism will incur a significant performance cost - the added cost will typically not be justfiable and if such errors are a rare occurence waiting for the kernel to finish and restart the kernel will typically be a better approach.

    -Amit


    Amit K Agarwal

    • Marked as answer by Yonglun Li Friday, January 25, 2013 4:20 AM
    Thursday, January 24, 2013 10:41 AM
    Owner

All replies

  • If by cancellation you mean calling something like ::abort() or ::exit() in a restrict(amp) context, no, that is not possible for the time being...unless you are running RefRast (the Debug device), case in which you can call either direct3d_errorf or direct3d_abort.
    Wednesday, January 23, 2013 5:14 AM
  • Not necessarily ::abort() or ::exit() or the like in a restrict(amp) context. Assume I sent a computation to the GPU, and now I recieve a request from the user or whatever to give up the computation on the GPU. If the computation has been completed, fine, I just give up the results. If the computation is still in progress, I want to stop it without shuting down the application/process. Is it possible to do so?


    • Edited by Yonglun Li Thursday, January 24, 2013 2:31 AM
    Thursday, January 24, 2013 2:29 AM
  • Welp, abort() and exit() are examples of the sort of cancellation that you are looking for...at least when considered in a multi-threaded context:). Answering the question itself, no, such flexibility is not available for the time being. Consider that the GPU is an async device, that "eats" a command buffer that gets assembled on the CPU side, and that once it starts executing kernels are (typically) serialized. So there are two non-trivial issues to solve: one is that you'd have to package the cancellation call into a command buffer and pass it to the GPU - this will be latencious. Second issue is that you'd have to be able to preempt everyone and signal the cancellation - GPUs aren't necessarily very good with this as it stands, and the programming models don't expose it. Evolution is moving in that direction, but it will probably take a while before the hardware gets there, IMHO.
    Thursday, January 24, 2013 2:46 AM
  • That makes sense. Thanks.

    One more question: if I package a computation and send it to the GPU for execution, but soon I notice there're some errors in the data, what else can I do besides waiting for its completion and starting it again?

    Thursday, January 24, 2013 7:53 AM
  • How do you detect errors in the data? Do you mean you detect errors in the input data passed to the kernel, on the CPU, after the kernel has been dispatched? If so, as Alex mentioned in his earlier response, currently the operating system and GPU hardware do not support asynchronous cancellation of tasks queued for exection to the GPU.

    If the error is detected on the GPU itself, while executing the kernel, in theory this may be detected by communicating between threads within a tile or across tiles through a cancellation flag in global memory (or tile_static memory for communicating between threads within a tile) and use of atomic operations for reading/writing the flag. Having said that, atomic operations are very expensive and such a cancellation mechanism will incur a significant performance cost - the added cost will typically not be justfiable and if such errors are a rare occurence waiting for the kernel to finish and restart the kernel will typically be a better approach.

    -Amit


    Amit K Agarwal

    • Marked as answer by Yonglun Li Friday, January 25, 2013 4:20 AM
    Thursday, January 24, 2013 10:41 AM
    Owner