search-engine

I know that Lucene has extensive support for wildcard searches and I know you can search for things like: Stackover* (which will return Stackoverflow ) That said, my users aren't interested in learning a query syntax. Can Lucene perform this type of wildcard search using an out-of-box Analyzer? Or should I append "*" to every search query? Doing this with string manipulations is tricky to get right, especially since the QueryParser supports boosting, phrases, etc. You could use a QueryVisitor that rewrites TermQuery into PrefixQuery. public class PrefixRewriter : QueryVisitor { protected

I am working on an application where I need to integrate the search engine. This should do crawling also. Please suggest a good Java based search engine. Thank you in advance. Ajay Nutch ( Lucene ) is an Open Source engine which should satisfy your needs. In the past I worked with terrier , a search engine written in Java: Terrier is a highly flexible, efficient, effective, and robust search engine, readily deployable on large-scale collections of documents. Terrier implements state-of-the-art indexing and retrieval functionalities. Terrier provides an ideal platform for the rapid development

Closed. This question is off-topic. It is not currently accepting answers. Learn more . Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 7 years ago . I have a network of about 200 blogs (Wordpress Multisite), and all of them show links to all the other ones on a sidebar on the right hand side (basically 200+ links on the right hand side of every single page). I have it set to rel="nofollow" now, but I was wondering if changing it to rel="noindex, nofollow" would be a good idea? Thank you for any input. nofollow nofollow only means that a bot

I am a little confused by the lucene scoring strategy. I know that Lucene's scoring formula is like: score(q,d) = coord(q,d) x queryNorm(q) X SUM <t_in_q> ( tf(t_in_d) x idf(t)^2 x t.getBoost() x norm(t,d)) I understand every component in this formula except queryNorm(q) . As explained by the official documentation, queryNorm(q) is a normalizing factor used to make scores between queries comparable. This factor does not affect document ranking (since all ranked documents are multiplied by the same factor), but rather just attempts to make scores from different queries (or even different

I want to write a basic semantic web crawler using Python, I know that semantic apps use RDF files, but what else? I have some Python RDF modules installed and I started learning how they work. Could you introduce me to the technologies and techniques used in a semantic application? Lev Khomich The next things you might want to learn are: embedding samantic data in HTML - RDFa , microformats , microdata . Some stats: microformats and RDFa deployment across the Web via DAM.co.uk : querying RDF data - SPARQL most popular ontologies currently in use list of available SPARQL endpoints You can find

How can I resist the bad unidentified bots to crawl my website? Some bad bots whose name is not present in cPanel of Apache are badly accessing my website bandwidth. I had tried robots.txt on batgap.com/robots.txt and also blocked with .htaccess but there is no improvement in bandwidth usage. I don't know the IP of those bots so unable to block them by IP address. These bots are consuming too much bandwidth of site and hence a result I need to increase it from server. I'm from Incapsula and we deal with bad bots on a regular basis. We've recently release a bot-related research that provides