HBase Scan Performance

前端 未结 2 1219
我寻月下人不归
我寻月下人不归 2020-12-31 11:43

I am performing a range scan that is giving me 500k records. If I set scan.setCaching(100000) it took less than one second, but if scan.setCaching(100000)

2条回答
  •  [愿得一人]
    2020-12-31 12:30

    Hbase has 2 types of cache structures - memory store and block cache.
    memory store is implemented as MemStore and the cache you use for reading is block cache.
    When a data block is read from HDFS, it is cached in the BlockCache. Subsequent reads of neighboring data are simply served from the BlockCache.
    So, when you manually set scan.set Block Cache(false) then , it will stop caching the rows it reads from hdfs.
    scan.set-caching(100000) is a client side optimisation related to scanners. So it will still work unaffected

提交回复
热议问题