问题
I wanted to cluster my data-set using carrot2 workbench. I have an input xml file with 65536 documents. I am using Lingo clustering algorithm.
But when I start the process, the workbench returns the result within few seconds having all the documents in the "other topics" cluster.
I have checked the clustering with smaller data-sets and I am getting the results.
回答1:
Carrot2 Lingo algorithm was designed for small data sets, up to a thousand or so of documents. For larger data sets, you may want to try STC, which scales better.
Regardless of the algorithm, Carrot2 processes all data in-memory, so it will not scale to millions of documents. In the latter case you may want to look at Apache Mahout, for example.
来源：https://stackoverflow.com/questions/28980710/carrot2-workbench-not-able-to-process-large-data