How much text can Weka handle?

风流意气都作罢 提交于 2019-12-12 04:58:20

问题


I have a sentiment analysis task and I need to specify how much data (in my case text) weka can handle. I have a corpus of 2500 opinions already tagged. I know that it´s a small corpus but my thesis advisor is asking me to specifically argue on how much data can Weka handle.


回答1:


Your limitation with Weka will be on whatever learning algorithm you use and how much memory you have available for training. Most classifiers require the whole set be loaded into memory for training, but there are options for streaming data as well. See the weka page on big data for more information.

For a dataset as small as yours, you will not have any problem. With any big data issue, though, you hit a point where you no longer can just script it on a single machine. With Weka it is no different and there are ways of making it work once you get there. To my knowledge, there is no hard limit on the amount of data you will be able to handle, given enough hardware resources, time, and ingenuity.



来源:https://stackoverflow.com/questions/25979182/how-much-text-can-weka-handle

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!