search-engine

Do you know where I can find a high level explanation of Lucene Similarity Class algorithm. I will like to understand it without having to decipher all the math and terms involved with searching and indexing. Lucene's built-in Similarity is a fairly standard "Inverse Document Frequency" scoring algorithm. The Wikipedia article is brief, but covers the basics. The book Lucene in Action breaks down the Lucene formula in more detail; it doesn't mirror the current Lucene formula perfectly, but all of the main concepts are explained. Primarily, the score varies with number of times that term occurs

I've been writing a program which extracts data from web searches. To get more data, I'd ideally like to extract more results per query through a script (let's say 100 or so). My question is, is there a way to modify the URL for Google, Yahoo, or Bing (preference in that order) so that I can get more than 10 results per query? For Google, appending &num=99 used to work at one point but no longer works :( I saw a similar append of &count=50 but that didn't work on any of the search engines either. The reason num=99 doesn't work for Google is because the num parameter's actual value isn't used,

I am currently using whoosh to dev a website, and I'll need to choose something more powerful once the website will be in production. If anyone of you used both of these engines, which one gave you the most meaningful results one the long road? Rui Carneiro Solr is the best option. Its well documented and the community is huge. Almost a year ago I benchmarked Xapian vs Solr: My dataset had +8000 emails: Solr index time: 3s index size: 5.2mb Xapian index time: 30s index size: 154mb Another great reading about benchmarks between Xapian and Solr is this document: Cross-instance Search System -

I need to detect search engines that refers to my website. Since every search engine has different query strings for searching(e.g. google uses 'q=', yahoo uses 'p=') I created a database for search engines with their url regex patterns. As an example: http://www.google.com/search?q=blabla&ie=utf-8&oe=utf-8&aq=t&rls=com.ubuntu:en-GB:official&client=firefox-a the regex I created for google is: (http:)(\\/)(\\/)(www)(\\.)(google)(\\.).*(\\/)(search).*(&q=|\\?q=).* (I am a newbie on regex, but so far it works) This detects that the url belongs to Google. My problem is that I need to extract the

I have been coding up a small search engine and need to find out if there is a faster way to find set intersections. Currently, I am using a Sorted linked list as explained in most search engine algorithms. i.e for every word I have a list of documents sorted in a list and then find the intersection among the lists. The performance profiling of the case is here . Any other ideas for a faster set intersection? An efficient way to do it is by "zig-zag": Assume your terms is a list T : lastDoc <- 0 //the first doc in the collection currTerm <- 0 //the first term in T while (lastDoc != infinity):

I would like to set the Default Search Engine Provider of IE with IOpenServiceManager::InstallService: Belong to the link http://www.opensearch.org/Specifications/OpenSearch/1.1#OpenSearch_description_elements . I created the SearchProviderInfo.xml like this: <?xml version="1.0" encoding="UTF-8"?> <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"> <ShortName>Web Search</ShortName> <Description>Use Example.com to search the Web.</Description> <Tags>example web</Tags> <Contact>admin@example.com</Contact> <Url type="application/atom+xml" template="http://example.com/?q=