locked
Custom Website Search Tool RRS feed

  • Question

  • User1051163829 posted

    Hey,

    I have built and used my own search tools before but none for a site quite as big as I'm currently working on and just wandered if anyone had any ideas on how flawed (or maybe not) my logic is for what I propose to do? And if anyone has any better ideas with regards to implementing a self-built search tool then please share them :)

    I lay it out in steps:

    1. Read from DB at set inervals and create text file(s) to store indexed information of site's current content.
    2. How to store these so a search is as fast and efficient as possible? Any ideas. Currently I set a max limit on the physical size of the text file and create a new one if necessary. (2MB)
    3. Install a windows service on the webserver to routinley re-index the text files and thus keep the possible search results as new as possible. (Interval could be daily, site's only mid-sized - not huge).
    4. Search the files using a .dll (a .dll currently carries out most processing with regards to fetching and computing for the website) - but this is where I think I fall down. I need some help on searching with phrases or single word. Obviously I can search using standard methods, i.e. Contains(str), IndexOf(str), even split phrases into single words and search for each (I think this could get slow though...?). Also should there be a precompiled result list of favourite searches for even greater speed in generating results?

    Is this a good way to go about the tool or am I barking up the wrong tree?

    Any help is appreciated,

    Thanks.

    Thursday, December 16, 2010 2:17 PM

Answers

  • User-952121411 posted

    Is this a good way to go about the tool or am I barking up the wrong tree?

     

    Right concept, but I would go a different way about it harnessing the power or SQL Server.

    1st, extract the data out of your text files (metadata, search terms, etc) and get it into tables within SQL Server. Then set up a Full Text Index using SQL Server on the columns that have the search data. Once you have the full text index setup you can use the CONTAINS and FREETEXT functions to quickly and efficiently do free form text searches of the content you have stored.

    With this method you don't have to create your own searching algorithms; SQL Server has it already provided.  Start with the following link which will explain all that you need to do. I myself have used this method and it works well.

    Querying SQL Server Using Full-Text Search:

    http://msdn.microsoft.com/en-us/library/ms142559.aspx

     

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, December 16, 2010 2:57 PM

All replies

  • User-952121411 posted

    Is this a good way to go about the tool or am I barking up the wrong tree?

     

    Right concept, but I would go a different way about it harnessing the power or SQL Server.

    1st, extract the data out of your text files (metadata, search terms, etc) and get it into tables within SQL Server. Then set up a Full Text Index using SQL Server on the columns that have the search data. Once you have the full text index setup you can use the CONTAINS and FREETEXT functions to quickly and efficiently do free form text searches of the content you have stored.

    With this method you don't have to create your own searching algorithms; SQL Server has it already provided.  Start with the following link which will explain all that you need to do. I myself have used this method and it works well.

    Querying SQL Server Using Full-Text Search:

    http://msdn.microsoft.com/en-us/library/ms142559.aspx

     

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, December 16, 2010 2:57 PM
  • User1051163829 posted

    That's great, thanks for the  help. I will indeed use this method, as the search algorithm alone will save me time!

    Many thanks, 

    Friday, December 17, 2010 9:15 AM