none
Best way for continuously searching for emails when in Outlook online mode RRS feed

  • Question

  • Hi,

    I'm hoping someone here has experience in this and knows of the pitfalls / best practices of searching for emails in Outlook.  The requirements for this are:

    • must be able to work in the background / not affect UI for the users
    • need to search all folders and sub-folders
    • search will being run every time user sends / receives email
    • need to be able to only return emails sent / received since previous search
    • must pass back EntryIDs of emails so these can be iterated over and data extract from the emails
    • must work with all versions of Exchange (including Exchange Online / Office 365)
    • must not affect Exchange server performance / user Outlook performance
    • must be able to handle 1000s of folders and 100,000s of emails

    So far the options I have come across are:

    AdvancedSearch (current method I use)

    Pros

    • can be run in the background / has SearchComplete event
    • fast with large numbers of emails / folders / sub-folders
    • can return just EntryIDs so not too heavy

    Cons

    • doesn't always fire SearchComplete event / just doesn't work
    • has performance issues when running in Outlook online mode (server CPU increase / Outlook slowdown)
    • relies on indexing of emails to be fast?

    Items.Restrict

    Pros

    • works well with large number of emails
    • might work better in online mode

    Cons

    • can cause issues with Exchange since I've heard the server has a limit of the number of restrict queries it can run
    • not as easy to run in the background / untested whether affects Outlook performance

    RDOFolderSynchronizer (Redemption)

    Pros

    • is the same method Outlook uses to synchronise folder changes
    • can work in the background?
    • should be most accurate?

    Cons

    • unknown how it affects performance of Outlook
    • need to manually iterate over all folders / sub-fodlers
    • need to persist and store sync hash
    • not sure whether hash works across multiple instances of Outlook for same user

    So my question is, what would people recommend to better support when Outlook is in online mode?  Are there performance / known issues with any of these methods and ways to get around these?

    Thanks for your help!

    Tom

    Monday, November 14, 2016 12:11 PM

All replies

  • Hello,

    If AdvancedSearch doesn't meet your requirements then you need to use Extended MAPI.

    The best option is to use a low-level API (Extended MAPI) on which Outlook is based on. It supports running the code from secondary threads, so it is an ideal way to go. As you may know Redemption is just a wrapper around that API, so it can be used from secondary threads too. Note, you will have to keep your data for each profile/account separately (hash).

    The most vital thing when working in the online mode is to release underlying COM objects instantly. Use System.Runtime.InteropServices.Marshal.ReleaseComObject to release an Outlook object when you have finished using it. This is particularly important if your add-in attempts to enumerate more than 256 Outlook items in a collection that is stored on a Microsoft Exchange Server. If you do not release these objects in a timely manner, you can reach the limit imposed by Exchange on the maximum number of items opened at any one time. Recent Outlook versions allows to hold more than 256 items, but the principle remains the same. Don't forget to set a variable to Nothing in Visual Basic (null in C#) to release the reference to the object after. Read more about that in the Systematically Releasing Objects article.



    profile for Eugene Astafiev at Stack Overflow, Q&A for professional and enthusiast programmers

    Monday, November 14, 2016 2:40 PM
  • Hi Eugene,

    Thanks for your help here!  As mentioned we are currently using the AdvancedSearch method however for a number of clients who are moving away from cached mode to online mode this is causing an issue.  Either Outlook is slowing down too much or the AdvancedSearch complete event isn't firing.

    Our code has followed guidelines on single dot annotation and releasing COM objects as soon as possible / setting variables to Nothing, so I'm hoping this isn't the cause of the issues.

    Do you think Exchange servers can quickly get overloaded with AdvancedSearches if all users are doing these whenever they send / receive emails?

    You also mentioned using Extended MAPI, what would be the preferred search method if we were to go this route, using the FolderSynchronizer as I described or is there another better way of searching emails?

    Cheers,

    Tom

    Monday, November 14, 2016 2:50 PM
  • Can you give some background on why you need to search and on what fields?

    Dmitry Streblechenko (MVP)
    http://www.dimastr.com/redemption
    Redemption - what the Outlook
    Object Model should have been
    Version 5.5 is now available!

    Monday, November 14, 2016 8:44 PM
  • Hi Dmitry,

    We need a method of quickly and reliably syncing data from emails to our app as they are sent and received in Outlook, e.g. 2-5 secs after.  We must extract data from the email headers (subject, recipients, timestamps...etc), and also need the body as a string to extract data from.  The syncing solution has to work across all different Outlook / Exchange setups, and must also work if a user switches between two machines running Outlook (we persist a timestamp at our app side for the last email they synced).

    The biggest two problems we currently have is that the AdvancedSearch method we use doesn't always work, and that the searching / extraction process can sometimes cause Outlook to be laggy for users running in Online mode.

    Thanks,

    Tom

    Tuesday, November 15, 2016 8:46 AM
  • But why do you need to search for anything instead of simply processing the item being sent or received?

    Dmitry Streblechenko (MVP)
    http://www.dimastr.com/redemption
    Redemption - what the Outlook
    Object Model should have been
    Version 5.5 is now available!

    Tuesday, November 15, 2016 1:34 PM
  • We have to search due to three reasons:

     - we need to initially process all historic emails they have looking back 6-12 months

     - if the user has been away or has Outlook closed when an email is sent/received then we need to capture these next time Outlook is opened

     - the NewMailEx event isn't guaranteed to fire 100% of the time when they receive an email

    Tuesday, November 15, 2016 2:43 PM
  • Tom,

    the NewMailEx event isn't guaranteed to fire 100% of the time when they receive an email

    Well, you can keep the list of items in the Inbox and check them out periodically. Or just use a low-level API notifications. Read more about possible solutions in the following series of articles:

    Outlook NewMail event unleashed: the challenge (NewMail, NewMailEx, ItemAdd)

    Outlook NewMail event: solution options

    Outlook NewMail event and Extended MAPI: C# example

    Outlook NewMail unleashed: writing a working solution (C# example)

    Also you can take a look at EWS, see EWS Managed API, EWS, and web services in Exchange for more information, in case if you deal with Exchange accounts only. Or just use EWS in case of Exchange online. 


    profile for Eugene Astafiev at Stack Overflow, Q&A for professional and enthusiast programmers

    Tuesday, November 15, 2016 8:58 PM
  • You can assume all unread emails in the Inbox folder are new - this will not process emails read from a different computer or phone and/or emails moved by rules to different folders, but I think it is a reasonable limitation that is easy to explain to the end users.

    You can also store the creation time of the last processed message and next time only process newer items using Items.Find/FindNext or Items.Restric.


    Dmitry Streblechenko (MVP)
    http://www.dimastr.com/redemption
    Redemption - what the Outlook
    Object Model should have been
    Version 5.5 is now available!

    Tuesday, November 15, 2016 11:20 PM