Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database

Question

I am trying to run SparkSQL :

val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)  

But the error i m getting is below:

Answer 1

an lck(lock) file is an access control file which locks the database so that only a single user can access or update the database. The error suggests that there is another instance which is using the same database. Thus you need to delete the .lck files. In your home directory, go to metastore_db and delete any .lck files.

Answer 2

Another case where you can see the same error is a Spark REPL of an AWS Glue dev endpoint, when you are trying to convert a dynamic frame into a dataframe.

There are actually several different exceptions like:

• pyspark.sql.utils.IllegalArgumentException: u"Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':"
• ERROR XSDB6: Another instance of Derby may have already booted the database /home/glue/metastore_db.
• java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader

The solution is hard to find with google but eventually it is described here.

The loaded REPL contains an instantiated SparkSession in a variable spark and you just need to stop it before creating a new SparkContext:

>>> spark.stop()
>>> from pyspark.context import SparkContext
>>> from awsglue.context import GlueContext
>>>
>>> glue_context = GlueContext(SparkContext.getOrCreate())
>>> glue_frame = glue_context.create_dynamic_frame.from_catalog(database=DB_NAME, table_name=T_NAME)
>>> df = glue_frame.toDF()

Answer 3

The error came because of the multiple spark shell you are trying to run in same node or due to system failure its shut down without proper exit the spark shell, In any of the reason you just find out the process id and kill them, for that us

[hadoop@localhost ~]$ ps -ef | grep spark-shell
hadoop    11121   9197  0 17:54 pts/0    00:00:00 grep --color=auto spark-shell
[hadoop@localhost ~]$ kill 9197

Answer 4

I got this error by running sqlContext._get_hive_ctx() This was caused by initially trying to load a pipelined RDD into a dataframe I got the error Exception: ("You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt assembly", Py4JJavaError(u'An error occurred while calling None.org.apache.spark.sql.hive.HiveContext.\n', JavaObject id=o29)) So you could running this before rebuilding it, but FYI I have seen others reporting this did not help them.

Answer 5

I am getting this error while running test cases in my multi maven spark setup. I was creating sparkSession in my test classes separately as unit test cases required different spark parameters every time which I am passing it through a configuration file. To resolve this I followed this approach. While creating the sparkSession in Spark 2.2.0

//This is present in my Parent Trait.
def createSparkSession(master: String, appName: String, configList: List[(String, String)]): SparkSession ={
    val sparkConf = new SparkConf().setAll(configList)
    val spark = SparkSession
      .builder()
      .master(master)
      .config(sparkConf)
      .enableHiveSupport()
      .appName(appName)
      .getOrCreate()
    spark
  }

In my test classes

//metastore_db_test will test class specific folder in my modules.
val metaStoreConfig = List(("javax.jdo.option.ConnectionURL", "jdbc:derby:;databaseName=hiveMetaStore/metastore_db_test;create=true"))
    val configList = configContent.convertToListFromConfig(sparkConfigValue) ++ metaStoreConfig
    val spark = createSparkSession("local[*]", "testing", configList)

And post that in maven clean plugin I am cleaning this hiveMetaStore directory.

//Parent POM
<plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>3.1.0</version>
                    <configuration>
                        <filesets>
                            <fileset>
                                <directory>metastore_db</directory>
                            </fileset>
                            <fileset>
                                <directory>spark-warehouse</directory>
                            </fileset>
                        </filesets>
                    </configuration>
                </plugin>

Child Module POM

<plugin>
                <artifactId>maven-clean-plugin</artifactId>
                <configuration>
                    <filesets>
                        <fileset>
                            <directory>hiveMetaStore</directory>
                            <includes>
                                <include>**</include>
                            </includes>
                        </fileset>
                        <fileset>
                            <directory>spark-warehouse</directory>
                        </fileset>
                    </filesets>
                </configuration>
            </plugin>