locked
Run operation in UI thread from background thread RRS feed

  • Question

  • Hi

    My Outlook initialization is asynchronous (using TPF with an StaTaskScheduler) - but when it is done, I'd like to execute certain operations that access the outlook object model - so GUI thread it is.

    I know how to do it in WPF and Winforms, but what about a VSTO add-in? 

    Regards

    Stephan

    Tuesday, March 4, 2014 4:43 PM

Answers

  • You can use data from Outlook objects in background threads, but not the Outlook objects (or Office objects) there. You also cannot and should not use the object model from a task scheduler of any kind.

    What you can use from background threads would be Extended MAPI or a 3rd party MAPI tool called Redemption (www.dimastr.com/redemption).

    As far as getting a synch context for the main thread that won't be available until the UI is active and the Window message pump has been primed in the thread.

    In the case of Redemption, which can be used from .NET code, I grab the NameSpace.MAPIOBJECT and in any background thread I instantiate a new Redemption RDOSession and set its MAPIOBJCT to the one from NameSpace in the main thread. That way the MAPI sessions are logged into the same logon. Then I can use that object model for long-running processes.


    Ken Slovak MVP - Outlook

    • Marked as answer by George Hua Wednesday, March 12, 2014 10:44 AM
    Monday, March 10, 2014 2:34 PM
  • If you are using an Attachment or Attachments those are Outlook objects and should never be used in a background thread. If you save an attachment to the file system and use that later, that's using data from an Outlook object.

    I have seen the thread context from the Outlook thread be null for quite a while after Outlook starts up, even after the UI is shown. However, if a button click is being used at that point there should be a valid thread context.

    When I use Redemption for background processing I've done things such as downloading and creating a couple of thousand contacts without impacting UI responsiveness at all. The same goes for things such as processing attachments.

    If I extract a string from MailItem.Subject and then use the string that's using data from an Outlook object. If I use MailItem.Subject in a background thread that's using an Outlook object and I'd expect Outlook to crash or hang at some point.


    Ken Slovak MVP - Outlook

    • Marked as answer by George Hua Wednesday, March 12, 2014 10:45 AM
    Monday, March 10, 2014 5:13 PM

All replies

  • Hmm.. I went through the first one - and I already had something like that that didn't work. I don't have any winforms elements in my addin, so that's out.

    I had already tried the following: In my Ribbon_Load callback, I would start an advanced search. That would return some seconds later (when the GUI is fully available), and then I'd fire off a task (with my StaTaskSource, so it's an STA thread), and once that completes, I added a continuation context

                    Task processResultsTask = myFactory.StartNew(() => processAdvancedSearchResult(allContacts, attachments));
                    synchronizeContactListWithDatabase();
                    processResultsTask.ContinueWith((result) => { synchronizeContactListWithDatabase(); }, tokenSource.Token, TaskContinuationOptions.None, TaskScheduler.FromCurrentSynchronizationContext());

    And, upon execution of that second line, I'm being told that

    " The current SynchronizationContext may not be used as a TaskScheduler."

    As per Stephen Cleary (http://msdn.microsoft.com/en-us/magazine/gg598924.aspx) that should capture the current sync context (being the gui thread since we're in an advanced search callback)

    Any other ideas?

    Tuesday, March 4, 2014 6:30 PM
  • Hi Stephan,

    I'm trying to involve some senior engineers into this issue and it will take some time.

    Your patience will be greatly appreciated.

    Sorry for any inconvenience and have a nice day!


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    • Edited by George Hua Thursday, March 6, 2014 7:58 AM
    Thursday, March 6, 2014 7:26 AM
  • Hi Stephan,

    Windows Forms & WPF are .NET-based UI framework (they have built-in support for .NET framework). But Outlook is written in unmanaged code (C++), I’m afraid not all features of the .NET parallel library could be applied to Office Object Model. For a workaround, I suggest you to use the default TaskScheduler:

    Task.ContinueWith<TResult> Method (Func<Task, TResult>, CancellationToken)

    Task processResultsTask = myFactory.StartNew(() => processAdvancedSearchResult(allContacts, attachments));
    processResultsTask.ContinueWith((result) => { synchronizeContactListWithDatabase(); }, tokenSource.Token);

    By default, the UI could be updated after all tasks finished. Hope it will help.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, March 10, 2014 5:10 AM
  • Will this really run the synchronizeContactListWithDatabase method in a threadpool thread? processAdvancedSearchResults certainly will. And, isn't it dangerous to run processAdvancedSearchResult in a threadpool thread given that it will access the outlook object model ? I have the plugin working every day in my outlook now with no issues, but that alone doesn't really mine it is fine. The only other way I see though would be to slowly process the result, using a timer that ticks e.g. once per every couple of seconds, but that means keeping at least the attachments objects around for a long time, which in itself is also problematic. As I understand it, you're not supposed to ever use the outlook object model frmo a background thread, but for proccessing a large number of items, you basically have no choice. So I wrote my code so that most of the processing (the fast stuff) is done from the advanced search callback, and only the slow stuff (saving attachments) is done in a background thread, and I free up every com resource at the earliest possible point in time (that's why I now only send the attachments to processAdvancedSearchResult)
    Monday, March 10, 2014 11:05 AM
  • You can use data from Outlook objects in background threads, but not the Outlook objects (or Office objects) there. You also cannot and should not use the object model from a task scheduler of any kind.

    What you can use from background threads would be Extended MAPI or a 3rd party MAPI tool called Redemption (www.dimastr.com/redemption).

    As far as getting a synch context for the main thread that won't be available until the UI is active and the Window message pump has been primed in the thread.

    In the case of Redemption, which can be used from .NET code, I grab the NameSpace.MAPIOBJECT and in any background thread I instantiate a new Redemption RDOSession and set its MAPIOBJCT to the one from NameSpace in the main thread. That way the MAPI sessions are logged into the same logon. Then I can use that object model for long-running processes.


    Ken Slovak MVP - Outlook

    • Marked as answer by George Hua Wednesday, March 12, 2014 10:44 AM
    Monday, March 10, 2014 2:34 PM
  • Ken - I'm not sure what to make of your last sentence. Given the advanced search... it returns an object containing a collection of ContactItem objects (in my case as i'm performing a contact search)... So I'm treating those ContactItem objects in the GUI thread, check if they have an attachment, and process those Attachments in a separate thread (because the Save (don't recall the real name right now) blocks for way too long and there's no async save method). So, is that "using data from the Outlook object" or using "Outlook objects"? And can't you only "use data from the Outlook object" by using the "Outlook object"? Is it a matter or read vs. write? (common wisdom dictates that write operations on properties that may be GUI bound is never a good idea)

    I noted in my earlier tries that if I processed the entire set of results in the background thread, about a minute into the start of Outlook I'd get an exception telling me about different COM threads that no longer communicate with each other (even though processing had long since ended), and I was under the impression I was releasing every single object I touched - but that had me separate result processing into a "fast" (runs in GUI thread) and "slow" (runs in background thread... saving attachments in my case) part and that seems to work but before I roll this out to any clients I want to be on the safe side.

    >As far as getting a synch context for the main thread that won't be available until the UI is active and the Window message pump has been primed in the thread.

    Well... I rewrote the code so the search can only be started by pressing a button that I'm adding to the ribbon.. so the sync context cannot be empty in this scenario. If I press the buttons say 2 minutes after outlook has fully started it must be something else that is not working out.

    Monday, March 10, 2014 4:42 PM
  • If you are using an Attachment or Attachments those are Outlook objects and should never be used in a background thread. If you save an attachment to the file system and use that later, that's using data from an Outlook object.

    I have seen the thread context from the Outlook thread be null for quite a while after Outlook starts up, even after the UI is shown. However, if a button click is being used at that point there should be a valid thread context.

    When I use Redemption for background processing I've done things such as downloading and creating a couple of thousand contacts without impacting UI responsiveness at all. The same goes for things such as processing attachments.

    If I extract a string from MailItem.Subject and then use the string that's using data from an Outlook object. If I use MailItem.Subject in a background thread that's using an Outlook object and I'd expect Outlook to crash or hang at some point.


    Ken Slovak MVP - Outlook

    • Marked as answer by George Hua Wednesday, March 12, 2014 10:45 AM
    Monday, March 10, 2014 5:13 PM
  • Hmm... I've now rewritten everything to save images unprocessed, then going through the saved images in a background thread. But going through my 666 contacts, saving all images using the code below (starting from the advanced search callback) still takes 10 seconds.. I even went as far as to parallelize processing, but it seems to have little impact. The machine I'm running this on has a fast SSD so I doubt IO is an issue, so I'm really puzzled as to why it seems to work fine for Ken, but results in nogo performance on my box. 

    Here's the code I'm using:

    void Application_AdvancedSearchCompleteBg(Outlook.Search SearchObject)
            {
                log("Advanced search complete, nb results " + SearchObject.Results.Count, 4);
                if (SearchObject.Results.Count > 0)
                {
                    List<Outlook.ContactItem> contacts = new List<Outlook.ContactItem>();
                    foreach (object obj in SearchObject.Results)
                    {
                        if (obj is Outlook.ContactItem)
                        {
                            contacts.Add((Outlook.ContactItem)obj);
                        }
                    }
                    List<OutlookContact> allContacts = processContacts(contacts);
                }
                else
                {
                    Marshal.ReleaseComObject(SearchObject);
                }
            }
    		
    		
            private List<OutlookContact> processContacts(List<Outlook.ContactItem> results)
            {
                List<OutlookContact> allContacts = new List<OutlookContact>();
                try
                {
                    bool writeImages = true;
                    string tempPath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "MyCompany", "Pictures");
                    bool directoryExists = false;
                    try
                    {
                        if (!Directory.Exists(tempPath))
                        {
                            log(tempPath + " does not exist, going to create folder..", 4);
                            Directory.CreateDirectory(tempPath);
                        }
                        else
                        {
                            directoryExists = true;
                            Directory.Delete(tempPath);
                            Directory.CreateDirectory(tempPath);
                        }
                    }
                    catch (Exception e)
                    {
                        log("Unable to create/clear path " + tempPath + ": " + e.Message, 2);
                        writeImages = false;
                    }
                    DateTime start = DateTime.Now;
                    Parallel.ForEach(results, (myItem) =>
                    {
                        OutlookContact contact = generateOutlookContact(myItem, false);
                        if (writeImages)
                        {
                            if (myItem.HasPicture)
                            {
                                Outlook.Attachment picture = myItem.Attachments["ContactPicture.jpg"];
                                if (picture == null)
                                {
                                    picture = myItem.Attachments["ContactPhoto"];
                                    if (picture != null)
                                    {
                                        saveAttachment(picture, tempPath, myItem.EntryID);
                                        Marshal.ReleaseComObject(picture);
                                    }
                                }
                                else
                                {
                                    saveAttachment(picture, tempPath, myItem.EntryID);
                                    Marshal.ReleaseComObject(picture);
                                }
                            }
                        }
                        allContacts.Add(contact);
                    });
                    TimeSpan duration = DateTime.Now.Subtract(start);
                    log("Processing of " + results.Count + " search results took " + duration.TotalMilliseconds + "ms", 4);
                }
                catch (AggregateException aex)
                {
                    foreach (Exception ex in aex.Flatten().InnerExceptions)
                    {
                        log("exception processing advanced search result: " + ex.Message, 2);
                    }
                }
                finally
                {
                    foreach (Outlook.ContactItem myItem in results)
                        Marshal.ReleaseComObject(myItem);
                }
                return allContacts;
            }
    		
    		private void saveAttachment(Outlook.Attachment picture, string path, string entryId)
            {
                string fileName = Path.Combine(path, entryId + ".jpg");
                try
                {
                    picture.SaveAsFile(fileName);
                }
                catch (Exception e)
                {
                    log("Unable to save " + fileName + ": " + e.Message, 2);
                }
            }
    		
    		
    	    private OutlookContact generateOutlookContact(Outlook.ContactItem contact, bool processImage = true)
            {
                OutlookContact myContact = new OutlookContact
                {
                    Email = contact.Email1Address,
                    FirstName = contact.FirstName,
                    LastName = contact.LastName,
                    CompanyName = contact.CompanyName,
                    Department = contact.Department,
                    EntryId = contact.EntryID, FullName = contact.FullName, 
                    JobTitle = contact.JobTitle, 
                    LastUpdate = contact.LastModificationTime, 
                    Login = contact.Account, 
                    OfficeCity = contact.BusinessAddressCity, 
                    OfficeLocation = contact.OfficeLocation, 
                    PresenceId = contact.IMAddress, 
                    Title = contact.Title, 
                    Numbers = new List<OutlookContactNumber>()
                };
                if (!string.IsNullOrEmpty(contact.PrimaryTelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    { 
                        ContractEntryId = myContact.EntryId, 
                        Number = contact.PrimaryTelephoneNumber, 
                        E164Number = stripNumber(contact.PrimaryTelephoneNumber),
                        NumberType = "primaryPhone"
                    });
                if (!string.IsNullOrEmpty(contact.CompanyMainTelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.CompanyMainTelephoneNumber,
                        E164Number = stripNumber(contact.CompanyMainTelephoneNumber),
                        NumberType = "mainPhone"
                    });
                if (!string.IsNullOrEmpty(contact.MobileTelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.MobileTelephoneNumber,
                        E164Number = stripNumber(contact.MobileTelephoneNumber),
                        NumberType = "mobile"
                    });
                if (!string.IsNullOrEmpty(contact.BusinessTelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.BusinessTelephoneNumber,
                        E164Number = stripNumber(contact.BusinessTelephoneNumber),
                        NumberType = "officetelephonenumber"
                    });
                if (!string.IsNullOrEmpty(contact.Business2TelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.Business2TelephoneNumber,
                        E164Number = stripNumber(contact.Business2TelephoneNumber),
                        NumberType = "officetelephonenumber2"
                    });
                if (!string.IsNullOrEmpty(contact.HomeTelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.HomeTelephoneNumber,
                        E164Number = stripNumber(contact.HomeTelephoneNumber),
                        NumberType = "homePhone"
                    });
                if (!string.IsNullOrEmpty(contact.Home2TelephoneNumber))
                    myContact.Numbers.Add(new OutlookContactNumber
                    {
                        ContractEntryId = myContact.EntryId,
                        Number = contact.Home2TelephoneNumber,
                        E164Number = stripNumber(contact.Home2TelephoneNumber),
                        NumberType = "homePhone2"
                    });
    
                if (processImage && contact.HasPicture)
                {
                    Outlook.Attachment picture = null;
                    picture = contact.Attachments["ContactPicture.jpg"];
                    if (picture == null)
                        picture = contact.Attachments["ContactPhoto"];
                    if (picture != null)
                        addContactImage(picture, myContact);
                }
                return myContact;
            }

    If I'm doing something wrong, I'd appreciate any pointers as I'm all out of ideas. As far as my performance measurements go, it's picture.SaveAsFile that is the bottleneck and since there's no async API for that...

    Monday, March 17, 2014 11:09 AM
  • Using the Outlook object model for saving the attachments will of course block the UI and will also take a long time. What I do in similar situations is to use Redemption.

    Once I grab NameSpace.MAPIOBJECT from the main thread I can start a background thread. I pass MAPIOBJECT to the new thread and start a Redemption RDOSession and pass it the MAPIOBJECT. That lets the Redemption session in the thread to use the same Extended MAPI session as Outlook is using.

    From Redemption I could then get the default Contacts folder, iterate each item in the folder's RDOItems collection and save any attachments to the file system.

    All of those operations would be running on the background thread I started, so the Outlook UI would not be impacted.

    Unfortunately, using only the Outlook object model would block the UI and would take a while. Using Redemption might not be faster in this case, but it wouldn't block the UI so it wouldn't matter so much.


    Ken Slovak MVP - Outlook

    Monday, March 17, 2014 1:53 PM
  • So the bottom line is, I'm not doing something wrong, but I need to use another API that allows doing the heavy lifting in a background thread (as it should be done anyway... initially everything was done in the background but that caused problems pretty much from the getgo).
    Monday, March 17, 2014 2:14 PM
  • You're correct. Doing long-running operations in the background is the only way to keep the UI responsive. It's just that the Outlook object model is the wrong tool for that job. Using Extended MAPI directly or through the Redemption wrapper allows you to keep the UI responsive.

    If only recent versions of Exchange are to be used with Outlook then another alternative might be a background thread using Exchange Web Services (EWS).


    Ken Slovak MVP - Outlook

    Monday, March 17, 2014 2:26 PM
  • From the UI thread save the context

    System.Windows.Forms.

    WindowsFormsSynchronizationContextcontext =

    newSystem.Windows.Forms.WindowsFormsSynchronizationContext();

    From the STA Thread run in the UI thread with

    context.Post(

    newSystem.Threading.SendOrPostCallback((o) => {}),object_passed_to_UI) 

    {

    Wednesday, November 18, 2015 4:24 PM
  • That will work, and was part of what was being discussed, but only if there's a valid synchronization context to begin with. That's not always the case when Outlook is started, even after the UI is responsive.

    Ken Slovak MVP - Outlook

    Wednesday, November 18, 2015 5:01 PM
  • I create and set it myself like this in ThisAddIn_Startup() and it works.

                SynchronizationContext.SetSynchronizationContext(new WindowsFormsSynchronizationContext());
                this._context = SynchronizationContext.Current;
    


    Jason Orphanidis

    Sunday, November 22, 2015 10:57 AM
  • That's fine, as long as SynchronizationContext.Current != null. That's something that should be checked.

    If null I usually set up to handle some Explorer events such as Activate(), BeforeFolderSwitch() and SelectionChange(). I check for SynchronizationContext.Current != null in those event handlers and if not null I set my context variable.


    Ken Slovak MVP - Outlook

    Monday, November 23, 2015 3:58 PM
  • I not sure I understand: if SynchronizationContext.Current != null, why should I set it up?

    Jason Orphanidis

    Monday, November 23, 2015 4:48 PM
  • As I had mentioned, often the context is null on startup and even after that. It does no good to persist an invalid or null context, it won't do anything for you. So you need to be able to get a context, check it for not being null, and persist it for later reference when it's not null.

    Ken Slovak MVP - Outlook

    Monday, November 23, 2015 5:00 PM
  • Ok that's what I'm saying.

    I check for the context being null in ThisAddIn_Startup() and since it *is* always null, I create a context of my own, which I persist for later use.

    So I only set it up when SynchronizationContext.Current == null.

    I think we're saying the same thing :)


    Jason Orphanidis

    Monday, November 23, 2015 5:32 PM
  • If that's what you're doing we're not saying the same thing at all.

    All Outlook object model calls must run on the main addin thread. Anything else will crash Outlook and/or get your addin disabled by Outlook for making OOM calls on a different thread.

    The main thread context is also what should be used to synch to for all UI operations.

    If you create your own context from a different thread that is not what is required for use of the OOM from a COM addin, or for use of the UI from a COM addin.


    Ken Slovak MVP - Outlook

    Monday, November 23, 2015 9:52 PM
  • Oh now I see. So you say that the thread that ThisAddIn_Startup() is running, is not the main thread?

    I thought it was.

    But how do we know it isn't the main thread?

    And why would the threads of Activate(), BeforeFolderSwitch() and SelectionChange() events, *be* the main thread but not that of ThisAddIn_Startup() ?


    Jason Orphanidis

    Tuesday, November 24, 2015 7:43 AM
  • The thread for ThisAddIn *is* the addin thread and is also the UI thread.

    However, even though the thread is running it may not have a valid context when Outlook is started. That depends on when the Windows message pump starts running on that thread. It may not be valid on startup, and if it is not valid those event handlers will allow you to get a valid thread context when the message pump does start running.


    Ken Slovak MVP - Outlook

    Tuesday, November 24, 2015 2:48 PM
  • So, is it bad to go ahead and create my own new WindowsFormsSynchronizationContext() inside ThisAddIn_Startup(), (and set it up as the Current sync context)  instead of mercyfully waiting when Windows will start pumping and when WinForms will do all this for me?

    After all, my concern is to leave out of ThisAddin_Startup() as soon as possible (so that Outlook metrics don't flag me), *with* my sync context in hand. If someone is going to create a WindowsFormsSynchronizationContext, it might as well be me, don't you think?


    Jason Orphanidis

    Tuesday, November 24, 2015 5:10 PM
  • I've been trying to tell you that what you're doing is most definitely bad. Very, very, very bad.

    You are supposed to use what's provided to you as a thread and synch context, not to create your own context.

    You can leave the Startup() handler without a context, which is very likely in my experience. You can then use the Explorer events I mentioned as triggers to try to get the context when the message pump does start pumping.

    If you exit startup as quickly as possible you still can and will get tripped up by the startup metrics if your code is what starts the Framework. That takes a couple of seconds, so your addin will be flagged as slow starting no matter what you do.


    Ken Slovak MVP - Outlook

    Tuesday, November 24, 2015 6:06 PM
  • Hi Ken

    Thanks for your advise. I did a little research and indeed, noone is supposed to set the sync context unless some very exceptional circumstances apply, which is not the case here.

    So, I call Application.DoEvents() and the message loop creates the context, right after.

    Thanks


    Jason Orphanidis

    Thursday, November 26, 2015 5:05 PM
  • DoEvents() will prime the message pump if it's already running. Just make sure you keep an eye on things, I've seen some reports that Outlook doesn't like the managed code version of DoEvents(). Just make sure things work properly after the call.

    Ken Slovak MVP - Outlook

    Friday, November 27, 2015 3:01 PM
  • "As far as getting a synch context for the main thread that won't be available until the UI is active and the Window message pump has been primed in the thread."

    Hi Ken,

    Is above statement still true? I noticed this answer is a few years old and thought there might be a way or a workaround now?

    I am having the same issue when I update the image of my button within the ribbon, the new image only takes effect if the outlook window is active/focused. E.g. If I have 2 monitors and outlook is in the extended screen (not focused) and my add-in updates the button's image in the background which is triggered by SignalR > WebApi Call > SynchronizationContext.Current.Post... The button only applies the new image when I click/focus on outlook. Otherwise it is working fine when outlook is currently focused when the background update happens.



    • Edited by dmcintyre83 Wednesday, August 8, 2018 7:37 AM
    Wednesday, August 8, 2018 7:36 AM
  • Nothing has changed as far as UI and threading and synch contexts.

    Ken Slovak MVP - Outlook

    Wednesday, August 8, 2018 2:32 PM
  • That's bad, considering that people are mostly using multiple screen displays at work now. Would have been nice if we were able to update the ribbon/button with an image to show some sort of count/image from background sync even when the window is not active.
    Wednesday, August 8, 2018 10:59 PM