none
Memory leak with concurrency::cancellation_token RRS feed

  • Вопрос

  • Hi,

    I'm struggling with a memory leak when using the concurrency runtime (library version from VS2015).

    I have a relative long living cancellation_token_source at the top level from which several token and linked_sources are derived in a hierachy. My problem is, that these linked sources for example are registring a callback to the upper level token (source), but do not unregister this callback when the down level sources are not in use any more.

    As far as I can see and understand the code, even when_all and when_any create implicitly linked sources when a cancellation token is passed.

    If there are many such calls / uses downlevel over a more or less long period of time this leads to a memory leak.

    Anybody experieced something similar and has a solution?

    Joja

    17 июля 2019 г. 9:35

Все ответы

  • >Anybody experieced something similar and has a solution?

    I think you'll need to come up with a minimal example project that reproduces what you're seeing. That would allow someone to double
    check you've not made a mistake, and if there is a bug you'd have something to report to MS in order to get it resolved.

    Dave

    18 июля 2019 г. 9:01
  • This lets constantly grow memory:
    #include <iostream>
    #include <numeric>
    #include <vector>
    #include <concrt.h>
    #include <ppltasks.h>
    
    using namespace concurrency;
    using namespace std;
    
    task<bool> create_sub_task(cancellation_token token)
    {
      return task<bool>([]
        {
          // Doing some important calculation and remote calls here
          wait(rand() % 5000);
          return rand() % 100 > 10;
        },
        token);
    }
    
    task<int> create_lengthy_task(cancellation_token token)
    {
      return task<int>([token]
        {
          while (!token.is_canceled())
          {
            vector<task<bool>> subTasks;
    
            for (int i = 0; i < 100; ++i)
              subTasks.push_back(create_sub_task(token));
    
            // Ensure some minimum time spent
            subTasks.emplace_back([] {wait(1000); return true; }, token);
    
            auto result = when_all(subTasks.begin(), subTasks.end(), token).get();
            if (accumulate(result.begin(), result.end(), true, logical_and<>()))
              cout << "passed a round\n";
            else
              cout << "failed a round\n";
          }
    
          return -1;
        },
        token);
    }
    
    constexpr int MAX_MAIN_TASKS = 5;
    
    task<int> create_command_task(cancellation_token token)
    {
      return task<int>([]
        {
          cout << "Commands: Q = Quit, 0-4: Cancel lengthy task\n";
          char command;
          cin >> command;
          return char_traits<char>::to_int_type(command);
        },
        token);
    }
    
    int main()
    {
      cancellation_token_source main_cancel_source;
      auto main_token = main_cancel_source.get_token();
    
      std::vector<pair<task<int>, cancellation_token_source>> lengthy_operations;
      
      for (int i = 0; i < MAX_MAIN_TASKS; ++i)
      {
        auto source = cancellation_token_source::create_linked_source(main_token);
        auto task = create_lengthy_task(source.get_token());
        lengthy_operations.emplace_back(task, source);
      }
    
      auto command_task = create_command_task(main_token);
    
      while (true)
      {
        vector<task<int>> tasks;
        transform(lengthy_operations.begin(), lengthy_operations.end(), back_inserter(tasks), [](auto& p) { return p.first; });
        if (command_task.is_done())
          command_task = create_command_task(main_token);
        tasks.push_back(command_task);
        tasks.emplace_back([] {wait(100); return -2; }, main_token);
    
        auto result = when_any(tasks.begin(), tasks.end(), main_token).get();
        switch (result.first)
        {
        case -1:
          for (auto& p : lengthy_operations)
          {
            if (p.first.is_done() && !p.second.get_token().is_canceled())
            {
              cout << "A lengthy task completed successfully. Restarting\n";
              p.first = create_lengthy_task(p.second.get_token());
            }
          }
          break;
        case -2:
          // Look for something and go on
          break;
        default:
          // Some command
          if (result.first >= '0' && result.first < '0' + MAX_MAIN_TASKS)
          {
            int task = result.first - '0';
            cout << "Cancel task " << task << "\n";
            lengthy_operations[task].second.cancel();
          }
          else if (result.first == 'Q')
          {
            cout << "Cancel all, exiting\n";
            main_cancel_source.cancel();
            while (!std::all_of(lengthy_operations.begin(), lengthy_operations.end(), [](auto& p) { return p.first.is_done(); }))
              wait(100);
            return 0;
          }
          else
          {
            cout << "Unknown command\n";
          }
        }
      }
    }
    
    Joja
    18 июля 2019 г. 10:51
  • This one is a much simpler example:

    #include <iostream>
    #include <numeric>
    #include <vector>
    #include <concrt.h>
    #include <ppltasks.h>
    
    using namespace concurrency;
    using namespace std;
    
    int main()
    {
      cancellation_token_source main_cancel_source;
      auto main_token = main_cancel_source.get_token();
    
      auto stop = task<void>([main_cancel_source]
        {
          string dummy;
          cout << "Press enter to quit\n";
          cin >> dummy;
          main_cancel_source.cancel();
        });
    
      while (true)
      {
        vector<task<void>> tasks;
        cancellation_token_source sub_source = cancellation_token_source::create_linked_source(main_token);
        for (int i = 0; i < 10; ++i)
        {
          tasks.emplace_back([]
            {
              wait(rand() % 1000);
            },
            sub_source.get_token());
        }
        task<void> tany[] = 
        {
          when_all(tasks.begin(), tasks.end(), main_token),
          stop
        };
        when_any(begin(tany), end(tany), main_token).get();
      }
    }


    • Изменено joja4711 18 июля 2019 г. 12:33 Missing .get() in when_any call
    18 июля 2019 г. 11:42
  • >This one is a much simpler example:

    OK, I've given your code a try with the latest VS preview, but I'm not immediately seeing an obvious memory leak. The diagnostic
    tool window illustrates memory usage around 3MB, with some minor fluctuations, no obvious rise. How much are you seeing leaking?
    Which points are you measuring the leak at in your source?

    Dave

    18 июля 2019 г. 22:45
  • Having now left your example program running for a while longer, it does look like there is a memory leak - small, but persistent.

    If you don't get a reply here in the next day or so pointing out some quirk with your code (I can't spot anything, but I'm not
    familiar with these areas) then please go ahead and use the VS Report a Problem facility to report it to MS.

    Be sure to supply this minimal example (as a VS project ideally), and clear instructions on how to reproduce it, and what someone
    should look for. Doing so will help your bug report get past the first hurdle.

    Please post a link to your bug report back here and I'll add comments and vote for it too.

    Dave

    18 июля 2019 г. 23:08