none
EWS searchFilter not working correctly with EWS FindItems call RRS feed

  • Question

  • I Am using a SearchFilter collection to restrict what emails are returned by a request to an Exchange 2010 mailbox using EWS.

    I have no problem connecting to the service, and opening the mailbox.

    The problem is that my searchFilter is being ignored, and all the emails are being returned by the request to EWS.

    Here is my code:

    static void Main(string[] args)
    {
    ServicePointManager.ServerCertificateValidationCallback = CertificateValidationCallBack;
    
    //creates an object that will represent the desired mailbox
    Mailbox mb = new Mailbox(@"bbtest@bocuk.local");
    
    // Find all items where the body contains "move reports".
    //string qstring = "Body:\"move reports\"";
    
    // Identify the item properties to return.
    //view.PropertySet = new PropertySet(BasePropertySet.IdOnly,
    //ItemSchema.Subject);
    
    //creates a folder object that will point to inbox fold
    FolderId fid = new FolderId(WellKnownFolderName.Inbox, mb);
    
    //this will bind the mailbox you're looking for using your service instance
    Folder inbox = Folder.Bind(service, fid);
    
    List<SearchFilter> searchFilterCollection = new List<SearchFilter>();
    
    searchFilterCollection.Add(new SearchFilter.SearchFilterCollection(LogicalOperator.And, new SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, false)));
    searchFilterCollection.Add(new SearchFilter.SearchFilterCollection(LogicalOperator.And, new SearchFilter.IsEqualTo(EmailMessageSchema.HasAttachments, true)));
    searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "FATS")));
    searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Assignment")));
    searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Sandbox: Assignment")));
    
    SearchFilter searchFilter = new SearchFilter.SearchFilterCollection(LogicalOperator.Or, searchFilterCollection.ToArray());
    
    ItemView view = new ItemView(100);
    
    string sAttachmentPath = "C:\\Dev\\EWSHelloWorld\\attachments\\";
    
    // Find the first email message in the Inbox that has attachments. This results in a FindItem operation call to EWS.
    FindItemsResults<Item> results = service.FindItems(WellKnownFolderName.Inbox, searchFilter, view);
    
    foreach (EmailMessage email in results)
    // looping through all the emails
    {
    
    System.Diagnostics.Debug.Write("Found attachemnt on msg with subject: " + email.Subject);
    
    .... code removed for brevity!

    So, according to my understanding of the searchFilter, only unread emails with attachments should be returned and they should not have FATS or Sandbox: Assignment as the subject.

    But that's not working, the request to EWS just returnes all the emails.

    What am I doing wrong please?

    Friday, March 21, 2014 1:26 PM

All replies

  • It would suggest when you doing this you enable tracing and then look at the restriction that is being sent to the server eg your searchfilter combination generates

            <m:Restriction>
              <t:Or>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="message:IsRead" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="false" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="item:HasAttachments" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="true" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:Not>
                  <t:Contains ContainmentMode="Substring" ContainmentComparison="IgnoreCase">
                    <t:FieldURI FieldURI="item:Subject" />
                    <t:Constant Value="FATS" />
                  </t:Contains>
                </t:Not>
                <t:Not>
                  <t:Contains ContainmentMode="Substring" ContainmentComparison="IgnoreCase">
                    <t:FieldURI FieldURI="item:Subject" />
                    <t:Constant Value="Assignment" />
                  </t:Contains>
                </t:Not>
                <t:Not>
                  <t:Contains ContainmentMode="Substring" ContainmentComparison="IgnoreCase">
                    <t:FieldURI FieldURI="item:Subject" />
                    <t:Constant Value="Sandbox: Assignment" />
                  </t:Contains>
                </t:Not>
              </t:Or>
            </m:Restriction>

    Where in your example you just need the collection of SearchFilter to be included under one AND grouping (eg IsUnread and HasAttachment and not subject1 and not subject2) so you need something like

                SearchFilter.SearchFilterCollection searchFilterCollection = new SearchFilter.SearchFilterCollection(LogicalOperator.And);
                searchFilterCollection.Add(new SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, false));
                searchFilterCollection.Add(new SearchFilter.IsEqualTo(EmailMessageSchema.HasAttachments, true));
                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "FATS")));
                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Assignment")));
                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Sandbox: Assignment")));
    
                ItemView view = new ItemView(100);
    
               // Find the first email message in the Inbox that has attachments. This results in a FindItem operation call to EWS.
                FindItemsResults<Item> results = service.FindItems(WellKnownFolderName.Inbox, searchFilterCollection, view);

    My suggestion would be to avoid such complicated SearchFilters as when you try to apply these against folders with very large Item counts your gong to suffer a performance penalty and possibly search timeouts. If you just have a simple query on UnreadItems with Attachments that will always give good performance no matter what or how big the folder your running it against and then just filter out the false positives on the client side (given your only looking at unread with attachments these will be pretty few). Unless you have a real restiction on Bandwidth the complex query like your using is going to work against you. When you use SearchFilters it places a temporary restriction on the folder in question so you might want to take of read of http://technet.microsoft.com/en-us/library/cc535025%28EXCHG.80%29.aspx to understand the implication of overusing these. This is where using AQS is an advantage over using searchfilters

    Cheers
    Glen 

    Monday, March 24, 2014 4:32 AM
  • @Glen

    Maybe I am miss-understanding something here (very possible as I am quite new to EWS).  When you say

    "My suggestion would be to avoid such complicated SearchFilters as when you try to apply these against folders with very large Item counts your gong to suffer a performance penalty and possibly search timeouts.  If you just have a simple query on UnreadItems with Attachments that will always give good performance no matter what or how big the folder your running it against and then just filter out the false positives on the client side".

    Okay so let me just try and understand this - you are still saying that we should use a filter, just not a complicated one.  You are also saying that it is better performance to query the emails with a simple filter that returns lots of results and then perform client side filtering?  This seems counter intuitive to me.

    If this was a SQL database (which I realise it is not) I would not want my client code to be doing a SELECT WHERE IsRead = false and then filtering the results on the client side.

    It also seems to me that you are making an assumption that the number of unread emails is likely to be small - but just in case it is not small, then we would have to add pagination code to the client side too right?  Potentially this might result in several calls to the server with lots of data getting shipped back each time and then the client having to filter this data - and this is better performance?

    So in my case I am searching for a single email - but the prospect of having to add code to handle pagination and doing client side filtering on a large set of results does not feel right.  I would much rather just add the details to a search filter and let the server do the filtering.  I don't understand why the server, which should have direct access to the data source, would be slower at filtering than my remote client.

    I tried to understand the document you linked (it looks like it is for Exchange 2007) but it looks like it talks a lot about restricted views which seem to me to be a bit like a kind of cached search results that exists and are maintained for up to 40 days.  If I understand this correctly, that would be a great thing right, because now subsequent queries will be much faster.

    Perhaps you can help me to understand?

    Can you also help me understand why you recommend using a AQS query rather than a search filter, yet this microsoft page seems to suggest the opposite?


    Friday, November 20, 2020 5:04 PM
  • It depends on the situation and as in anything your developing you test it yourself and see but lets take this case especially this filter.

                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "FATS")));
                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Assignment")));
                searchFilterCollection.Add(new SearchFilter.Not(new SearchFilter.ContainsSubstring(ItemSchema.Subject, "Sandbox: Assignment")))

    One sub-string query is going to be expensive but three ?, even if this was SQL would you do it this way ?. Recipient queries are another area where people tend to over-complicate the query and see bad performance mainly this is due to the way the data is stored in Exchange (in a separate collection). But as with anything you need to evaluate the results especially on a folder that pushes 100,000 items which is more common then you think.

    It would help to know what type of fields your searching on ? sometimes you can't avoid complex search-filters but in this example there are better options. Some fields are indexed some are not so KQL/AQS would work well for somethings while not for other. Also if your search is kind of static SearchFolders are a good choice as they get built and maintained in the background so at query time your performance is gong to much better where a more complex query needs to be applied. But first case is establish a baseline for you current query that works and then look a ways of improving it (eg in SQL you might add indexes which you can't do with Exchange so you need to tweak the query). Pagination isn't that bad (depending on the result set size) but things like considering the DateTime eg in the above example they didn't probably need to query every email in a Mailbox that probably goes back 10 years (or more) so having a good DateTime filter would have improved the performance greatly. 


    Sunday, November 22, 2020 11:26 PM
  • http://www.networksteve.com/exchange/topic.php/EWS_searchFilter_not_working_correctly_with_EWS_FindItems_call/?TopicId=40606&Posts=1

    hassansayedissa

    Sunday, November 22, 2020 11:51 PM