Logical Condition and Map Each Filtering Bug

Question

I’ve been looking at the logical conditions used in combination with Map Each.

A Map Each operation has a condition input that allows you to filter or select records in a kind of where clause. While looking at this I think I've found a bug where filtering doesn't happen as you'd hope/expect.

I’ve been using the following data as an input and tried to filter a company by name  or a company by department name.

<ns0:source xmlns:ns0="http://schemas.microsoft.com/Integration/Samples/EmployeeRecordsSource" xmlns:s0="http://schemas.microsoft.com/Integration/Samples/FullTimeAndContractEmployeeRecords"><company><name></name></company><?xml version="1.0" encoding="utf-8"?>
<ns0:Source xmlns:s0="http://schemas.microsoft.com/Integration/Samples/FullTimeAndContractEmployeeRecords" xmlns:ns0="http://schemas.microsoft.com/Integration/Samples/EmployeeRecordsSource">
  <Company>
    <Name>C1</Name>
    <Department>
      <Name>D1</Name>
      <Employee Type="Contract">
        <FirstName>C1_D1_PTE1_FN</FirstName>
        <LastName>C1_D1_PTE1_LN</LastName>
        <Age>C1_D1_PTE1_Age</Age>
      </Employee>
      <Employee Type="Contract">
        <FirstName>C1_D1_PTE2_FN</FirstName>
        <LastName>C1_D1_PTE2_LN</LastName>
        <Age>C1_D1_PTE2_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C1_D1_FTE1_FN</FirstName>
        <LastName>C1_D1_FTE1_LN</LastName>
        <Age>C1_D1_FTE1_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C1_D1_FTE2_FN</FirstName>
        <LastName>C1_D1_FTE2_LN</LastName>
        <Age>C1_D1_FTE2_Age</Age>
      </Employee>
    </Department>
    <Department>
      <Name>D2</Name>
      <Employee Type="Contract">
        <FirstName>C1_D2_PTE1_FN</FirstName>
       <LastName>C1_D2_PTE1_LN</LastName>
        <Age>C1_D2_PTE1_Age</Age>
      </Employee>
      <Employee Type="Contract">
        <FirstName>C1_D2_PTE2_FN</FirstName>
        <LastName>C1_D2_PTE2_LN</LastName>
        <Age>C1_D2_PTE2_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C1_D2_FTE1_FN</FirstName>
        <LastName>C1_D2_FTE1_LN</LastName>
        <Age>C1_D2_FTE1_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C1_D2_FTE2_FN</FirstName>
        <LastName>C1_D2_FTE2_LN</LastName>
        <Age>C1_D2_FTE2_Age</Age>
      </Employee>
    </Department>
  </Company>
  <Company>
    <Name>C2</Name>
    <Department>
      <Name>D3</Name>
      <Employee Type="Contract">
        <FirstName>C2_D1_PTE1_FN</FirstName>
        <LastName>C2_D1_PTE1_LN</LastName>
        <Age>C2_D1_PTE1_Age</Age>
      </Employee>
      <Employee Type="Contract">
        <FirstName>C2_D1_PTE2_FN</FirstName>
         <LastName>C2_D1_PTE2_LN</LastName>
         <Age>C2_D1_PTE2_Age</Age>
       </Employee>
       <Employee Type="FullTime">
         <FirstName>C2_D1_FTE1_FN</FirstName>
        <LastName>C2_D1_FTE1_LN</LastName>
         <Age>C2_D1_FTE1_Age</Age>
       </Employee>
      <Employee Type="FullTime">
        <FirstName>C2_D1_FTE2_FN</FirstName>
        <LastName>C2_D1_FTE2_LN</LastName>
        <Age>C2_D1_FTE2_Age</Age>
      </Employee>
    </Department>
    <Department>
      <Name>D4</Name>
      <Employee Type="Contract">
        <FirstName>C2_D2_PTE1_FN</FirstName>
        <LastName>C2_D2_PTE1_LN</LastName>
        <Age>C2_D2_PTE1_Age</Age>
      </Employee>
      <Employee Type="Contract">
        <FirstName>C2_D2_PTE2_FN</FirstName>
        <LastName>C2_D2_PTE2_LN</LastName>
        <Age>C2_D2_PTE2_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C2_D2_FTE1_FN</FirstName>
        <LastName>C2_D2_FTE1_LN</LastName>
        <Age>C2_D2_FTE1_Age</Age>
      </Employee>
      <Employee Type="FullTime">
        <FirstName>C2_D2_FTE2_FN</FirstName>
        <LastName>C2_D2_FTE2_LN</LastName>
        <Age>C2_D2_FTE2_Age</Age>
      </Employee>
    </Department>
  </Company>
</ns0:Source><company><department><employee Type="FullTime"></employee></department> </company> </ns0:source>

Filtering by Company Name

When filtering by company name, everything works fine. The Logical Expression can be set to Name == “C1” or Name  == “C2” and the map filters the records correctly.

Filtering companies by Department Name

The problem comes when attempting to filter by department name. The data above has 4 distinct departments – D1 to D4 with D1/2 being assigned to C1 and D3/4 assigned to C2. The map below shows how the logical expression is now linked to the Department Name

So what happens?

• Name = “D1” – filters correctly, only selecting company C1
• Name = “D2” – Creates a blank document – not what I was hoping for
• Name = “D3” – filters correctly, only selecting company C2
• Name = “D4” – Creates a blank document – not what I was hoping for, once more

It appears that the filtering may only work on the first node. In fact, if you swap the order of D1 and D2, the filtering will work on D2 but not D1.

Answer 1

Hi Rob,

 What you are observing here is by design. What you should do to over come the problem is move the Logical Expression inside the department mapeach and connect it to department mapeach. (Please see the picture below).

Please Note that we show a warning for such scenario :

I will try to explain what is happening,

In the first case when you connect the Logical Expression functoid to Company/Name, you are in the company mapeach, hence the mapping engine iterates over all the Company records and hence can match CompanyName with the given value correctly.

In the second case since the Logical Expression is still in Company Scope, it will try to get the value from Department/Name, which will give it the first value from the collection (i.e something like CurrentCompany.Department[0].Value),  the logical expression will then use this value to compare it with the given value.

Hence in your case Name="D1" filters correctly as Name is nothing but Department[0].Value ( because of a missing loop).

Name = "D2" does not filter correctly since Name is set to  CurrentCompany.Department[0].Value which is "D1" .

Name="D3" filters correctly as Name is nothing but CurrentCompany.Department[0].Value ( because of a missing loop).

similarly Name = "D4" does not filter correctly siince Name is set to CurrentCompany.Department[0].Value which is D3.

Here "CurrentCompany" refferes to the current item being processed by Company Mapeach.

In order to filter correctly you should move the Logical Expression inside Department Mapeach and connect the output of the logical functoid to DepartmentMapping.

---

Harsh

Answer 2

Hi Harsh

Would your proposed solution allow me to filter a company by department name though? Surely it would just knock out the departments that don't match and still map the parent company?

Thanks

Rob

Answer 3

You are right Rob, that would still map the company. If  you have to filter both by company name and department name the solution is to use list initialization like this.

IN this solution the list contains one column by the name of DepartmentNames,

in the select value functoid you need to filter appropriately by specifying the condition as item.DepartmentNames=="D2".

in the final Logical expression you need to specify Input_1 =="D2", so that result is a boolean expression.

Please note that this is not the only solution, I could have created two columns in my list, and then avoided the final Logical Expression.

Hope that helps!

---

Harsh

Answer 4

Thanks Harsh

Don't you think this could be made simpler though? It's still better than the BizTalk mapper could do but it could be A LOT better.

Cheers

Rob

Answer 5

Hi Rob,

         I understand that this is a little bit complicated, but the reason why we have to do it this way is that the underlying model follows a concept of scopes and functoids present in a scope can only be used in that or some child scope. This model mimics functional programming model and unlike Biztalk mapper, tries to surface this in the UI ( The Biztalk mapper used to add loops implicitly making the maps hard to understand).

There is another solution though as I mentioned, you can use two columns in the list as below,

The second column is a boolean Filter column which is always true for every row addedin the list.

in the add item i gave the conditon as the input, and in the select value i select the filter column( which is boolean hence I can directly consume it in the company mapeach).

---

Harsh

Answer 6

Hi Harsh

What would be really good, is a way to customise the scope of MapEach and ForEach. I'm still thinking about XPath or Linq To XML to allow a custom scope to be created. Surely that would reduce the number of map operations in use and fewer map operations makes a cleaner map, to me anyway.

Maybe it seems like a power user feature but it would allow the mapper more freedom. The BizTalk mapper limitations are often overcome with XSLT so it seems natural that you could allow the scopes to be defined in a custom way.

Thanks

Rob