locked
Index a multiple value column in BCS external content type "SharePoint 2013" RRS feed

  • Question

  • Regarding SharePoint 2013 Search, any body knows how to index/crawl a multiple values column ( in this column, values are seperated with  comma  or written in xml) ? In the index, I don't want the pure text value for this column but have a structured text value.

    The reason to do so is that We have a product database and the product could be related to a couple of categories, so in the category column it has multiple values and it is meaningless to crawl this column in pure text value.

    In the FAST ESP, I know we may create a custom index profile in which we can specify the way to crawl a specific property in XML format.

    Monday, December 24, 2012 2:30 AM

Answers

All replies

  • I read this article http://social.technet.microsoft.com/wiki/contents/articles/1144.multi-value-property-support-in-fast-search-server-for-sharepoint-en-us.aspx and it says in FAST Search 2010, to do this I can create a custom pipeline extensibility and have it seperate the values by \u2029 character. I tried replacing ; with \u2029 directly to the column value but it doesn't work.

    My Question is : Does SharePoint 2013 Search support to create custom pipeline extensibility?

    Tuesday, December 25, 2012 1:50 AM
  • I got a feeling that SharePoint 2013 Search doesn't even support this kind of situations.

    I just read some topics reagarding crawling a multi-value property in FAST for SharePoint 2010. In FAST for SharePoint 2010, the crawled property has a Multi-Valued attribute.

    However, in the new version this attribute is even GONE...

    Tuesday, December 25, 2012 7:52 AM
  • I find it. In SP 2013, this feature is replaced by "Custom content processing with the Content Enrichment web service callout"

    http://msdn.microsoft.com/en-us/library/jj163968.aspx

    An implementation sample:

    http://www.blendmaster.net/blog/2012/09/using-content-enrichment-web-service-callout-in-sharepoint-2013-preview/

    • Marked as answer by Avantee Thursday, December 27, 2012 5:00 AM
    Thursday, December 27, 2012 4:59 AM
  • I managed to get the content enrichment for multiple values property to work.

    The multiple value manged property comes into the item.ItemProperties as List<string> and the original property value is in its [0]. To make it to work, just split the values by ';' and insert them into a new List<string> object and add it to the outputprocessed item properties. Here is my sample code.

    You can also add a log feature to the web services to debug purpose.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Runtime.Serialization;
    using System.ServiceModel;
    using System.ServiceModel.Web;
    using System.Text;
    using Microsoft.Office.Server.Search.ContentProcessingEnrichment;
    using Microsoft.Office.Server.Search.ContentProcessingEnrichment.PropertyTypes;
    using System.IO;
    
    
    namespace ContentEnrichmentService
    {
       public class ContentEnrichmentService :IContentProcessingEnrichmentService
        {
            
    
            public ProcessedItem ProcessItem(Item item)
            {
                ProcessedItem processedItem = new ProcessedItem
                {
                    ItemProperties = new List<AbstractProperty>()
                };
    
                WriteLog("Start to process content enrichment...\n");
                processedItem.ItemProperties.Clear();
                try
                {
                    var categoryProperty = item.ItemProperties.Where(p => p.Name == "BCSVendorCategory").FirstOrDefault();
                    Property<List<string>> categoryProp = categoryProperty as Property<List<string>>;
    
                    if (categoryProp != null && categoryProp.Value != null && categoryProp.Value.Count > 0)
                    {
                        string[] propValues = categoryProp.Value.First().Split(';');
                        List<string> listpropValues = propValues.ToList();
                        propValues.ToList().Remove("");
                        Property<List<string>> newCategoryProp = new Property<List<string>>();
                        newCategoryProp.Name = "VendorCategory";
                        newCategoryProp.Value = listpropValues;
                        processedItem.ItemProperties.Add(newCategoryProp);
                        WriteLog(string.Format("{0} values are added to  property {1}.", newCategoryProp.Value.Count, newCategoryProp.Name));
                    }
                }
                        
                catch (Exception ex)
                {
                    WriteLog(string.Format("Exception is encountered. Error Message: {0}",ex.Message + "\n\t" + ex.StackTrace));
                }
                return processedItem;
            }
    
            private void WriteLog(string msg)
            {
                using (var writer = new StreamWriter("C:\\Temp\\ContentEnrichmentWebServiceTraceLog_" + System.DateTime.Today.ToString("yyyyMMdd") + ".txt", true))
                {
                    writer.Write(System.DateTime.Now.ToString("MM/dd/yyyy HH:mm:ss") + "\t" + msg);
                    writer.WriteLine("");
                }
            }
        }
    }
    

    • Proposed as answer by rk_muddala Saturday, February 8, 2014 10:50 AM
    Saturday, December 29, 2012 2:20 AM
  • Thanks for the links! I implemented a variation of the sample. My crawl doesn't seem to kick off the web service at all. I don't get any errors or warnings. I removed the trigger to force the web service to be used but it doesn't seem to be working.

    Was there anything you had to do to get things to fire? I ran all the PowerShell commands and added the configuration.

    Thanks!

    -=Steve

    Tuesday, June 18, 2013 4:28 PM
  • 1. Did you map the crawled property to a managed property and set in the InputProprties (in content enrichment configuration)?

    2. Can you access this Content Enrichment web services in browser?

    3. If the Content Enrichment web service and crawler server are on the same server, how is the memory usage status when doing full crawl? If memory is not suffcient, the app pool of this content enrichment web service will go down as well.

    4. Do you have more than one Search Service Application created in the sp farm? If yes, remove the other search service application and try it again.

    These are the problems I experienced in the past couple of months.

    Also, I suggest you implement a simple txt log feature to your content enrichment web service to check the process.

    Tuesday, June 18, 2013 4:39 PM
  • Thanks. I do have event log code to trace but nothing is in there. I only have 1 SSA and I can access the web service from the browser.

    I created a Managed Property to be the one that is multi-value (and is returned from the web service) but it is not mapped to a crawled property. Do I need to map that to something?

    -=Steve

    Tuesday, June 18, 2013 5:14 PM
  • No you don't need to map the outputproperty to any crawled property.

    Maybe you can try restart the Osearch service and Search Controller service, or delete and recreate a SSA.

    BTW, can you paste out your content enrichment powershell configuration?

    Tuesday, June 18, 2013 6:00 PM
  • $ssa = Get-SPEnterpriseSearchServiceApplication
    $config = New-SPEnterpriseSearchContentEnrichmentConfiguration
    $config.DebugMode = $false
    $config.Endpoint = "http://sp2013/_vti_bin/EventsEnrichmentService.svc"
    $config.FailureMode = "ERROR"
    $config.InputProperties = "EventDepartments"
    $config.OutputProperties = "EventDeptMulti"
    $config.SendRawData = $false
    Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication $ssa –ContentEnrichmentConfiguration $config

    EventDepartments contains string values like -> "HR;IT;Marketing;Finance"

    Tuesday, June 18, 2013 6:14 PM
  • It looks like an authentication problem.

    I guess http://sp2013 is your sharepoint root site collection and it requires auth.

    Try moving content enrichment web service out to some other annonymous IIS website.

    This web services can be hosted anywhere as long as the search service can reach it.


    • Edited by Avantee Tuesday, June 18, 2013 6:24 PM correct wrong word
    Tuesday, June 18, 2013 6:22 PM
  • Thanks. I restarted my search servers and now I am getting some (error) feedback from the crawl. I need to dig in.

    -=Steve

    Tuesday, June 18, 2013 6:53 PM
  • Ok. I fixed my crawl and it is trying to call the enrichment web service. I am getting errors that the item cannot be sent to the content enrichment service. I moved the web service to a non SharePoint address and received the same error. The ULS logs shows attempts to call http://server/service/enrichmentservice.svc:ProcessItem but no errors. I can't navigate to the service with the ProcessItem call. What do you get when you navigate to that?

    I am probably going to put this aside for now and focus on moving forward with other items.

    -=Steve

    Wednesday, June 19, 2013 4:36 PM
  • Paste out your enrichmentservice.svc.cs code to see if there is anything wrong in your code.

    Wednesday, June 19, 2013 5:56 PM
  • I turned on Verbose logging and found that there is an authentication issue:

    The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'Negotiate,NTLM'.

    I had Anonymous and Windows Authentication Enabled on the web service web site. When I turn Anonymous off, I get the following when attempting to browse the service:

    The authentication schemes configured on the host ('IntegratedWindowsAuthentication') do not allow those configured on the binding 'BasicHttpBinding' ('Anonymous').  Please ensure that the SecurityMode is set to Transport or TransportCredentialOnly.  Additionally, this may be resolved by changing the authentication schemes for this application through the IIS management tool, through the ServiceHost.Authentication.AuthenticationSchemes property, in the application configuration file at the <serviceAuthenticationManager> element, by updating the ClientCredentialType property on the binding, or by adjusting the AuthenticationScheme property on the HttpTransportBindingElement.

    My web.config probably "ain't right".  I am trying to add the bindings and such as explained on other sites. I can't add an endpoint as per the example, they remove the IService.cs, and therefore I don't have a Contract on the service.

    What does your web.config look like? What settings in IIS are correct?

    Thanks!

    -=Steve

    Wednesday, June 26, 2013 7:20 PM
  • I dorked around some more and got it to work. I re-enabled Anonymouse and used this as my bindings:

    <bindings>
          <basicHttpBinding>
            <binding name="httpBinding">
              <security mode="TransportCredentialOnly">
                <transport clientCredentialType="None" />
              </security>
            </binding>
          </basicHttpBinding>
        </bindings>

    Now I just need to put the real code back into the service!

    Wednesday, June 26, 2013 7:57 PM
  • <system.serviceModel>
        <behaviors>
          <serviceBehaviors>
            <behavior>
              <!-- To avoid disclosing metadata information, set the values below to false before deployment -->
              <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
              <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
              <serviceDebug includeExceptionDetailInFaults="true"/>
            </behavior>
          </serviceBehaviors>
        </behaviors>
        <protocolMapping>
            <add binding="basicHttpsBinding" scheme="https" />
        </protocolMapping>   
        <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
        <bindings>
         <basicHttpBinding>
         <!-- The service will accept a maximum blob of 8 MB. -->
            <binding maxReceivedMessageSize = "8388608">
               <readerQuotas maxDepth="32"
                maxStringContentLength="2147483647"
                maxArrayLength="2147483647"  
                maxBytesPerRead="2147483647"  
                maxNameTableCharCount="2147483647" /> 
                <security mode="None" />
            </binding>
         </basicHttpBinding>
      </bindings>
      </system.serviceModel>
    Thursday, June 27, 2013 1:47 AM
  • Any chance someone the enrichment service it to work on a farm with multiple SSA's + multiple servers?

    It does work on my Dev environment (one SSA, one server, service is locally).

    It is registered:

    Endpoint         : http://10.0.0.22:8811/EnrichmentService.svc
    Timeout          : 9999
    FailureMode      : Error
    InputProperties  : {AgileProject}
    OutputProperties : {AgileProject}
    Trigger          :
    DebugMode        : False
    SendRawData      : False
    MaxRawDataSize   : 100000

    But it doesn't work, and doesn't write ULS logs either. So I'm clueless.

    The service app pool is the same one that runs the search. Anonymous access is set, tried everything there is in the post regarding the web.config.

    Any ideas?

    Tuesday, May 13, 2014 3:21 PM
  • It works in my multiple servers environment. For multiple SSAs, I am not sure if it work or not. I have never run into that situation.

    Anyway, you may delete the SSAs and have only one SSA in the farm to narrow it down.

    Wednesday, May 14, 2014 1:26 AM
  • Thanks. Deleting the second SSA did solve it.

    The thing is that there are multiple SSA's on the production environment,

    So, at this moment the enrichment service won't be happening, unless there is a solution for multiple SSA's,

    or someone comes up with another way to modify the refinement in runtime...

    Thanks.

    Thursday, May 15, 2014 10:53 AM