Bulk load to multiple HBase tables in single job

社会主义新天地 提交于 2020-01-07 04:09:25

问题


I want to bulk load data to mulitple tables using a single mapreduce job.Since the data volumes is high ,It would be time consuming to iterate through dataset twice and load using multiple jobs.Is there any way to do this ? Thanks in advance.


回答1:


I am using Hbase. But i didnt need bulk load yet. But I came across this article which might help you.

http://hbase.apache.org/book/arch.bulk.load.html

The bulk load feature uses a MapReduce job to output table data in HBase's internal data format, and then directly loads the generated StoreFiles into a running cluster. Using bulk load will use less CPU and network resources than simply using the HBase API.



来源:https://stackoverflow.com/questions/19079370/bulk-load-to-multiple-hbase-tables-in-single-job

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!