Nutch in Hadoop 2.x

假如想象 提交于 2019-12-13 00:29:09

问题


I have a three-node cluster running Hadoop 2.2.0 and HBase 0.98.1 and I need to use a Nutch 2.2.1 crawler on top of that. But it only supports Hadoop versions from 1.x branch. By now I am able to submit a Nutch job to my cluster, but it fails with java.lang.NumberFormatException. So my question is pretty simple: how do I make Nutch work in my environment?


回答1:


At the moment it's impossible to integrate Nutch 2.2.1 (Gora 0.3) with HBase 0.98.x. See: https://issues.apache.org/jira/browse/GORA-304

Official Nutch tutorial recommends only 0.90.x HBase branch: http://wiki.apache.org/nutch/Nutch2Tutorial

Also you can download HBase 0.94.24-hadoop-2.5.0 version which I created and tested today: https://github.com/dobromyslov/hbase/releases/tag/0.94.24-hadoop-2.5.0

Take a note that Nutch 2.2.1 does not support HBase 0.94.x and you have to get the latest Nutch 2.x from Git branch: https://github.com/apache/nutch/tree/2.x



来源:https://stackoverflow.com/questions/23436168/nutch-in-hadoop-2-x

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!