User44709200 posted
Hello,
I have a multilanguage website where www.mysite.com is recognized as a french website for all french browsers, and in english for any other foreign browser.
By default Google and Bing crawlers seem to come without any language, so in that case we explain them the www. is in french.
So thousands of pages from www. are indexed in french by Google and Bing, which is perfect (and a en.mysite.com subdomain is dedicated to english).
The problem comes from Yahoo: it is the only one to see the www. in english.
So my conclusion was that the Yahoo crawler was detected as english. In order to solve this issue, I have implemented a crawler detection: if a crawler comes with a language, then it sees the site in french.
Here it is:
If HttpContext.Current.Request.UserLanguages Is Nothing Then
' robots see
www.mysite.com as a french site
' ok for google and bing since don't have any language, but wrong for yahoo which seems to arrive
in english
sLanguage = "fr"
Else
' case robot has a language, especially for yahoo
Dim myBrowserCaps As System.Web.HttpBrowserCapabilities = HttpContext.Current.Request.Browser
If (CType(myBrowserCaps, System.Web.Configuration.HttpCapabilitiesBase)).Crawler Then
sLanguage = "fr"
End If
End If
But it didn't change anything after 3 weeks (I don't know how to see the date of the pages in cache under Yahoo so it's hard to debug).
Do you have any idea ? Anybody noticed a different behaviour from Yahoo ?
Thanks for your help.