data-mining

At least 3 types of n-grams can be considered for representing text documents: byte-level n-grams character-level n-grams word-level n-grams It's unclear to me which one should be used for a given task (clustering, classification, etc). I read somewhere that character-level n-grams are preferred to word-level n-grams when the text contains typos, so that "Mary loves dogs" remains similar to "Mary lpves dogs". Are there other criteria to consider for choosing the "right" representation? Evaluate . The criterion for choosing the representation is whatever works . Indeed, character level (!=

recently I came across this term,but really have no idea what it refers to.I've searched online,but with little gain. Thanks. If you don't have enough data to train your algorithm you can increase the size of your training set by (uniformly) randomly selecting items and duplicating them (with replacement). Take a sample of the time of day that you wake up on Saturdays. Some Friday nights you have a few too many drinks, so you wake up early (but go back to bed). Other days you wake up at a normal time. Other days you sleep in. Here are the results: [3.1, 4.8, 6.3, 6.4, 6.6, 7.3, 7.5, 7.7, 7.9,

Lets say I have 100000 email bodies and 2000 of them contains an abitrary common string like "the quick brown fox jumps over the lazy dog" or "lorem ipsum dolor sit amet". What techniques could/should I use to "mine" these phrases? I'm not interested in mining single words or short phrases. Also I need to filter out phrases that I already know occur in all mails. Example: string mailbody1 = "Welcome to the world of tomorrow! This is the first mail body. Lorem ipsum dolor sit AMET. Have a nice day dude. Cya!"; string mailbody2 = "Welcome to the world of yesterday! Lorem ipsum dolor sit amet

I want to know whether the k-means clustering algorithm can do classification? If I have done a simple k-means clustering . Assume I have many data , I use k-means clusterings, then get 2 clusters A, B. and the centroid calculating method is Euclidean distance. Cluster A at left side. Cluster B at right side. So, if I have one new data. What should I do? Run k-means clustering algorithm again, and can get which cluster does the new data belong to? Record the last centroid and use Euclidean distance to calculating to decide the new data belong to? other method? The simplest method of course is

In the field of Data Mining, is there a specific sub-discipline called 'Similarity'? If yes, what does it deal with. Any examples, links, references will be helpful. Also, being new to the field, I would like the community opinion on how closely related Data Mining and Artificial Intelligence are. Are they synonyms, is one the subset of the other? Thanks in advance for sharing your knowledge. Yin Zhu In the field of Data Mining, is there a specific sub-discipline called 'Similarity'? Yes. There is a specific subfield in data mining and machine learning called metric learning, which aims to