geospark

Read a binary column in spark using java language

六月ゝ 毕业季﹏ 提交于 2021-02-07 21:00:34
问题 I have a DataFrame witch contains a Binary column Type. DataFrame : +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Read a binary column in spark using java language

ⅰ亾dé卋堺 提交于 2021-02-07 20:57:39
问题 I have a DataFrame witch contains a Binary column Type. DataFrame : +--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

GeoSpark librairy using Spark Java

末鹿安然 提交于 2020-07-19 06:51:49
问题 I have the following dataframe o spark : __________________________ │Longitude │latitude | ├────────────┼─────────────┼ | -7.07378166|33.826661 | └────────────┴─────────────┴ I want to apply the ST_Geomfromtext from GeoSpark , but I don't know how to do it I tried the following code , but id doesn't work for me Dataset<Row> dataframe = df.withColumn("ST_Geomfromtext ", ST_GeomFromText(col("Longitude"),col("Latitude"))); I need your help . Thank you 来源: https://stackoverflow.com/questions

GeoSpark librairy using Spark Java

泄露秘密 提交于 2020-07-19 06:51:36
问题 I have the following dataframe o spark : __________________________ │Longitude │latitude | ├────────────┼─────────────┼ | -7.07378166|33.826661 | └────────────┴─────────────┴ I want to apply the ST_Geomfromtext from GeoSpark , but I don't know how to do it I tried the following code , but id doesn't work for me Dataset<Row> dataframe = df.withColumn("ST_Geomfromtext ", ST_GeomFromText(col("Longitude"),col("Latitude"))); I need your help . Thank you 来源: https://stackoverflow.com/questions

How to avoid gc overhead limit exceeded in a range query with GeoSpark?

此生再无相见时 提交于 2019-12-11 16:38:11
问题 I am using Spark 2.4.3 with the extension of GeoSpark 1.2.0. I have two tables to join as range distance. One table ( t1 ) if ~ 100K rows with one column only that is a Geospark's geometry. The other table ( t2 ) is ~ 30M rows and it is composed by an Int value and a Geospark's geometry column. What I am trying to do is just a simple: val spark = SparkSession .builder() // .master("local[*]") .config("spark.serializer", classOf[KryoSerializer].getName) .config("spark.kryo.registrator",