Starting thrift server in spark

问题

Can anyone help me with starting spark thrift server? I am running my script in standalone mode and I want to fetch data in my business intelligence tool.

In order to do that I need to start thrift server. I tried running shell script:

$SPARK_HOME/sbin/start-thriftserver.sh

but I get an error:

error "ps unknown option --0"

回答1:

The sbin/ scripts work fine under linux, but they are not prepared to run in Windows (see Failed to start master for Spark in Windows). But spark thriftserver can be start in foreground independent from OS using command:

java -cp conf/:jars/* org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal

Then you can test connection using beeline supplied with spark:

$ ./bin/beeline -u jdbc:hive2://localhost:10000
Connecting to jdbc:hive2://localhost:10000
16/11/22 13:09:57 INFO Utils: Supplied authorities: localhost:10000
16/11/22 13:09:57 INFO Utils: Resolved authority: localhost:10000
16/11/22 13:09:57 INFO HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10000
Connected to: Spark SQL (version 2.0.1)
Driver: Hive JDBC (version 1.2.1.spark2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.2.1.spark2 by Apache Hive
0: jdbc:hive2://localhost:10000> show databases;
+---------------+--+
| databaseName  |
+---------------+--+
| default       |
| elo           |
+---------------+--+
2 rows selected (0,26 seconds)

回答2:

In Spark 2.2.1

cd %SPARK_HOME%\bin
spark-class org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal

this started the spark thrift server in windows. In another terminal run the below beeline command to open a jdbc connection

cd %SPARK_HOME%\bin
beeline -u jdbc:hive2://localhost:10000

来源：https://stackoverflow.com/questions/40653248/starting-thrift-server-in-spark

标签

apache-spark

apache-spark-sql

spark-thriftserver