How to use method bulkSaveToCassandra with spark-cassandra-connector

六月ゝ 毕业季﹏ 提交于 2019-12-11 04:43:58

问题


I'm trying to use the method bulkSaveToCassandra with spark-cassandra-connector to optimize my insertions in Cassandra Database. However, I can't find out the method and I don't know how to import the lib.

Currently, I'm using this dependency:

<dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector_2.11</artifactId>
            <version>2.0.2</version>
</dependency>

Below the reference of method bulkSaveToCassandra from Datastax: http://docs.datastax.com/en/datastax_enterprise/4.7/datastax_enterprise/migration/migratingBulkSparkRDD.html

I used the import presented in link but it don't work anyway.

Best regards,


回答1:


That method is only available in DataStax Enterprise.

The required class is inside of the dse-spark*.jar which comes with the DSE Installation.

For example this would be available from the dse spark shell.

I would recommend just using the normal write method though for most use cases as the bulkMethod uses the sstable writer which has availability issues which don't jive well with production use cases.



来源:https://stackoverflow.com/questions/45072442/how-to-use-method-bulksavetocassandra-with-spark-cassandra-connector

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!