BCS Data Model Crawl in SharePoint 2010

Question

Background : We have PLM*Product Life Cycle) Management LOB which has Item and Revisions Data.

In SharePoint we have created two Data Models

1. Item Model

2. Revisions Model (Item ID is Primary key in Item Model and Foreign key in Revion Model)

Each item can have multiple revisions for same Item Id.

Now when we are crawling the Revision Data model it is never successfull and always terminated with error message " The filtering process has bene terminated. We did some error logging in revisiosn ReadList method whih has been set as RootFinder mathod for crawl and found that it s crawled for each Item ID multiple time and so takes a long time and we assume this could be one reason for terminating the crawl filtering process". We need to change this behaviour so that we can crawl the revisions data model without any issue.

Thanks

Naveen

Answer 1

You have used the term "model" to say that you have created models for Item and Revision. I am assuming that you actually created External Content Types (ECT) called "Item" and "Revision". I am further assuming that you have created an Associator between Item and Revision. Is this correct?

Also, you state that you are crawling the Revision ECT directly. Did you create a Content Source against the Revision ECT directly?

Normally, when there are two ECTs that use an Associator and we want to crawl the association, we use either the AttachmentAccessor or DirectoryLink property to specify the behavior. AttachmentAccessor associations return the parent record whent he child is located. DirectoryLink associations return a separate result for each child.

Try reworking the ECTs to reflect one of these two approaches.

Next, consider the horizontal width of your returned result record. The UseClientCachingForSearch property is normally set by the SharePoint Designer, but you should not use this property if the record is wider than 30Kb.

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 2

Hi Scott

Thanks for your time Please find my comments inline.

You have used the term "model" to say that you have created models for Item and Revision. I am assuming that you actually created External Content Types (ECT) called "Item" and "Revision". I am further assuming that you have created an Associator between Item and Revision. Is this correct? Yes there are 2 ECT created Item and revison But there is no Association defined between two.

Also, you state that you are crawling the Revision ECT directly. Did you create a Content Source against the Revision ECT directly?

Yes we have created a LOB Content Source based on Revision ECT and crawling it.

Normally, when there are two ECTs that use an Associator and we want to crawl the association, we use either the AttachmentAccessor or DirectoryLink property to specify the behavior. AttachmentAccessor associations return the parent record whent he child is located. DirectoryLink associations return a separate result for each child.

Try reworking the ECTs to reflect one of these two approaches. We don't have any Association defined between these two ECT's We Just have multipel record in revision for an Item id in Item ECT. And we are able to fetch the data in these 2 lists successfully it is only when crawling we are getting errors.

Next, consider the horizontal width of your returned result record. The UseClientCachingForSearch property is normally set by the SharePoint Designer, but you should not use this property if the record is wider than 30Kb. Will try this option. This is something i have not tried

Answer 3

So it sounds like the "Item" model is irrelevant to the problem. Does your "Revision" model have it's own unique primary key then? If not, just add one as an identity.

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 4

Hi Scott

We are able to resolve teh above issue by adding the  UseClientCachingForSearch and now during crawl ReadItem is called only once. But now there is another issue while searching for Revisions it is only searching the first revions e.g if revsions ECT has 4 reviosns records  A,B,C,D based on Item ID it is fetching only first record Revions A.

The revision record format is

ItemID ,RevisonID,RevisonsStatus

000111      A                  Approved

000111      B                  Approved

While in list it shows both of these records. But while Search it shows only the first one. Is it something related to Identity column ?

We have created 2 custom scopes for each model one Item Scope and another Revision Scope.

Thanks

Naveen

[phone]

Answer 5

Two things to look at:

First, does your Default Finder actually return all of the revision records when called? Search can't crawl them if they are not returned.

Second, each revision should have a unique key as I mentioned previously. If there isn't really a unique key, make one by using an Identity column.

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 6

Hi Scott

The repetative calling to Readitem method when crawling is resolved as  UseClientCachingForSearch  was missing . But now we are able to crawl only A revisiosn type of record even after defining the unique key. The crawl process is indexing only one record(First)  for each Item ID and leaving other records.

Thanks

Naveen

Answer 7

It sounds like it thinks the ItemID is the key field.

In your SpecificFinder, what field is used to retrieve a single record? It can't be ItemID, because that's not unique. It would have to be the new identity key you defined. Make sense?

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 8

Hi Scott

We had tried various combination of keys in the table for Primary Key including GUID to make that Revisions table crawl all records (Currently it is crawling only first record of any itemID Which is primary key).  None of approach is working . Please guide if we are missing any step in this complete scenario.

Thanks

Naveen

Answer 9

My challenge in trying to help further is that the answer is straightforward. Search is going to crawl whatever comes back from the Finder method that you have designated as the Root Finder. So the simple way to see what that is would be to make an External List and examine the Root Finder data set. Your Specific Finder must then complement that functionality.

I don't think your problem is search. I think it is the design of your Finder and SpecificFinder.

If you had actualy established an Associator between the ECTs, then it's a little more complicated, but not too much. Since you didn't do that, it all comes down to the Root Finder.

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 10

Hi Scott

Below is my Model Code :

<?xml version="1.0" encoding="utf-8"?>
<Model xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schemas.microsoft.com/windows/2007/BusinessDataCatalog" Name="BOMView">
  <LobSystems>
    <LobSystem Name="PLMBDCRevisionModel" Type="DotNetAssembly">
      <LobSystemInstances>
        <LobSystemInstance Name="PLMBDCRevisionModel">
          <Properties>
            <Property Name="ShowInSearchUI" Type="System.String">x</Property>
          </Properties></LobSystemInstance>
      </LobSystemInstances>
      <Entities>
        <Entity Name="PLMBDCRevision" Namespace="PLMBDCIntegration.PLMBDCRevisionModel" EstimatedInstanceCount="1000" Version="1.0.0.46">
          <Properties>
            <Property Name="Class" Type="System.String">PLMBDCIntegration.PLMBDCRevisionModel.PLMBDCRevisionService, PLMBDCRevisionModel</Property>
          </Properties>
          <Identifiers>
            <Identifier Name="RevisionUnKey" TypeName="System.String" />
            <!-- TODO: Change the name of the ID and if needed the TypeName of your identifier. -->
          </Identifiers>
          <Methods>
            <!-- start finder method -->
            <Method Name="ReadList">
              <Properties>
                <Property Name="RootFinder" Type="System.String">x</Property>
                <Property Name="UseClientCachingForSearch" Type="System.String">x</Property>
              </Properties>
              <!-- TODO: Change the name of the method if needed. -->
              <Parameters>
                <Parameter Direction="Return" Name="returnParameter">
                  <TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[[PLMBDCIntegration.PLMBDCRevisionModel.PLMBDCRevision, PLMBDCRevisionModel]]" IsCollection="true" Name="PLMBDCRevisionList">
                    <TypeDescriptors>
                      <TypeDescriptor TypeName="PLMBDCIntegration.PLMBDCRevisionModel.PLMBDCRevision, PLMBDCRevisionModel" Name="PLMBDCRevision">
                        <TypeDescriptors>
                          <TypeDescriptor TypeName="System.String" IdentifierName="RevisionUnKey" Name="ItemID" />
                          <TypeDescriptor TypeName="System.String" Name="RevisionID" />
                          <!-- TODO: Add TypeDescriptors when you add properties to Entity1. -->
                          <TypeDescriptor Name="RevisionStatus" TypeName="System.String" />
                          <TypeDescriptor Name="DateReleased" TypeName="System.String" />
                          <TypeDescriptor Name="RefDocString" TypeName="System.String" />
                          <TypeDescriptor Name="ChangeRecord" TypeName="System.String" />
                          <TypeDescriptor Name="BOMView" TypeName="System.String" />
                          <TypeDescriptor Name="RevisionString" TypeName="System.String" />
                          <TypeDescriptor Name="RevisionUnKey" TypeName="System.String" /></TypeDescriptors>
                      </TypeDescriptor>
                    </TypeDescriptors>
                  </TypeDescriptor>
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Type="Finder" ReturnParameterName="returnParameter" Default="true" Name="ReadList" DefaultDisplayName="Entity1 List">
                  <Properties>
                    <Property Name="RootFinder" Type="System.String">x</Property>
                    <Property Name="UseClientCachingForSearch" Type="System.String">x</Property>
                  </Properties>
                  </MethodInstance>
              </MethodInstances>
            </Method>
            <!-- end finder method -->
            <!-- start specific finder method -->
            <Method Name="ReadItem">
              <Parameters>
                <Parameter Direction="In" Name="id">
                  <TypeDescriptor TypeName="System.String" IdentifierName="RevisionUnKey" Name="RevisionUnKey" />
                </Parameter>
                <Parameter Direction="Return" Name="returnParameter">
                  <TypeDescriptor Name="PLMBDCRevision" TypeName="PLMBDCIntegration.PLMBDCRevisionModel.PLMBDCRevision, PLMBDCRevisionModel">
                    <TypeDescriptors>
                      <TypeDescriptor Name="ItemID" TypeName="System.String" IdentifierName="RevisionUnKey" />
                      <TypeDescriptor Name="RevisionID" TypeName="System.String" />
                      <TypeDescriptor Name="RevisionStatus" TypeName="System.String" />
                      <TypeDescriptor Name="DateReleased" TypeName="System.String" />
                      <TypeDescriptor Name="RefDocString" TypeName="System.String" />
                      <TypeDescriptor Name="ChangeRecord" TypeName="System.String" />
                      <TypeDescriptor Name="BOMView" TypeName="System.String" />
                      <TypeDescriptor Name="RevisionString" TypeName="System.String" />
                      <TypeDescriptor Name="RevisionUnKey" TypeName="System.String" /></TypeDescriptors></TypeDescriptor>
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Type="SpecificFinder" ReturnParameterName="returnParameter" Default="true" Name="ReadItem" DefaultDisplayName="Read Entity1">
                  <Properties>
                    <Property Name="UseClientCachingForSearch" Type="System.String">x</Property>
                  </Properties> 
                </MethodInstance>
              </MethodInstances>
            </Method>
            <!-- end specific finder method -->
          </Methods>
        </Entity>
      </Entities>
    </LobSystem>
  </LobSystems>
</Model>

My Doubt is Specific Finder method as i doubt if it should have also

UseClientCachingForSearch” Property for making all revisions crawl.

Thanks

Naveen

Answer 11

Hi Scott

Above is code of my Model which has a Finder and Specific Finder method.

As per my knowledge i can't find any issue in design of Finder and Specific Finder which is stopping the index process to index all records for same id.

Please have  alook and guide if there is some design issue in Finder and Specific Finder method for my revisions Model. As i mentioned earlier I am not using any association.

<MethodInstances>
                <MethodInstance Type="SpecificFinder" ReturnParameterName="returnParameter" Default="true" Name="ReadItem" DefaultDisplayName="Read Entity1">
                  <Properties>
                    <Property Name="UseClientCachingForSearch" Type="System.String">x</Property>
                  </Properties>
                </MethodInstance>
              </MethodInstances>
In Above code i tried adding UseClientCachingForSearch to Specific Finder but it is also crawkling the same number of records.

Thanks

Naveen

Answer 12

Is RevisionUnKey unique to each individual revision record?

Have you tried making an External List from the model and verified that you see the expected result set?

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training

Answer 13

Yes revisionkey is Unique for each record

ItemID   RevisionID   RevisionStatus   DateReleased   RefDocString   ChangeRecord   BOMView   RevisionString   RevisionUnKey

 000180   A      000180/A;1-Assembly-Front          0ddb2fab-0fd1-44e9-8f42-1e511ca06fda
 000181   B      000181/B;1-Steering           b082a3ea-24d4-467d-95d1-54e6e4e70d9c
 000181   A BE4Released 7/17/2012 10:35:20 AM    000181/A;1-Steering       c9621ffa-f74f-4778-99a0-76274ec0cfa5
 000182   A      000182/A;1-CarSeat           dee07bdf-0f2d-4930-8a8b-3a8f42383820
 000183   A      000183/A;1-ArmRest           b1d39401-42f2-487f-a7b4-d3ab4417362d
 000184   B      000184/B;1-HeadRest           d38535f2-4388-47d3-bdd3-3615cbb78010

Below is snapshot of list from Above configuration. The last column of list is Unique ID and this is unique to each record.

Thanks

Naveen

Answer 14

Hi Scott

Thanks for your help. We are able to reesolve the issue after looking the design of Specific Finder . Actually there were some gap in complimenting teh functionality of ReadItem for ReadList.Got your book Professional SharePoint 2010 Development . Finding it good :-)

Thanks Again

Naveen

Answer 15

Excellent. I figured it was just the model design.

Glad you got it working.

Scot

---

Author, Professional Business Connectivity Services
Author, Inside SharePoint 2010 Blog, www.shillier.com
Twitter, @ScotHillier
SharePoint Trainer, Critical Path Training