information-retrieval

I've been wondering about this for a while and I can't see why Google haven't tried it yet - or maybe they have and I just don't know about it. Is there a search engine that you can type a question into which will give you a single answer rather than a list of results which you then have to trawl through yourself to find what you want to know? For example, this is how I would design the system: User’s input: “Where do you go to get your eyes tested?” System output: “Opticians. Certainty: 95%” This would be calculated as follows: The input is parsed from natural language into a simple search

What are the standard tf-idf implementations/api available in python? I've come across the one in nltk. I want to know the other libraries that provide this feature. Gunjan there is a package called scikit which calculates tf-idf scores. you can refer to my answer to this question Python: tf-idf-cosine: to find document similarity and also see the question code from this. Thankz. Try the libraries which implements TF-IDF algorithm in python. http://code.google.com/p/tfidf/ https://github.com/hrs/python-tf-idf Unfortunately, questions asking for a tool or library are offtopic on SO. There are

People often throw around the terms IR, ML, and data mining, but I have noticed a lot of overlap between them. From people with experience in these fields, what exactly draws the line between these? doug This is just the view of one person (formally trained in ML); others might see things quite differently. Machine Learning is probably the most homogeneous of these three terms, and the most consistently applied--it's limited to the pattern-extraction (or pattern-matching) algorithms themselves. Of the terms you mentioned, "Machine Learning" is the one most used by Academic Departments to

I have a scenario where i have retreived information/raw data from the internet and placed them into their respective json or .txt files. From there on i would like to calculate the frequecies of each term in each document and their cosine similarity by using tf-idf. For example: there are 50 different documents/texts files that consists 5000 words/strings each i would like to take the first word from the first document/text and compare all the total 250000 words find its frequencies then do so for the second word and so on for all 50 documents/texts. Expected output of each frequecy will be

In this question I asked clarifications about the precision-recall curve. In particular, I asked if we have to consider a fixed number of rankings to draw the curve or we can reasonably choose ourselves. According to the answer , the second one is correct. However now I have a big doubt about the Average Precision (AP) value: AP is used to estimate numerically how good is our algorithm given a certain query. Mean Average Precision (MAP) is average precision on multiple queries. My doubt is: if AP changes according to how many objects we retrieve then we can tune this parameter to our advantage

In the Google Web History interface I can see all the search queries I have used over the years, and the pages I visited for a particular query. Is there a way I can retrieve this history using a computer program? I couldn't find a Google API that does it. Do you know of a tool that can do this, or suggest a way to achieve this? There's an RSS feed . Update : the link is now broken. The RSS feed in the accepted answer above does not exist anymore. Google does not provide an API that allows you retrieve Google searches, but it does allow you to download an archive of all past searches, via

I am confused by the following comment about TF-IDF and Cosine Similarity . I was reading up on both and then on wiki under Cosine Similarity I find this sentence "In case of of information retrieval, the cosine similarity of two documents will range from 0 to 1, since the term frequencies (tf-idf weights) cannot be negative. The angle between two term frequency vectors cannot be greater than 90." Now I'm wondering....aren't they 2 different things? Is tf-idf already inside the cosine similarity? If yes, then what the heck - I can only see the inner dot products and euclidean lengths. I