Spark master picking up wrong IP on windows

自闭症网瘾萝莉.ら 提交于 2020-01-17 05:05:36

问题


When I start a spark master locally on my windows 7 machine for some reason to WebUI displays the URL and REST URL with the wrong ip address.

Below is my batch script I use to start master where %IP% is my local IP address.

set SPARK_MASTER_IP=%IP%
set SPARK_PUBLIC_DNS=%IP%
set SPARK_MASTER_PORT=7077
set SPARK_MASTER_WEBUI_PORT=8080
set SPARK_MASTER_REST_PORT=6066

START "spark-master" /MIN "C:/spark-1.6.0-bin-hadoop2.6/bin/spark-class.cmd" ^
    org.apache.spark.deploy.master.Master

The master Web UI displays the wrong IP. For some reason it is picking up the IP address of the IPv4 Address belonging to the VirtualBox installed on the same machine. (Note I'm am not using Virutalbox in any way to run Spark)

URL: spark://169.254.237.14:7077
REST URL: spark://169.254.237.14:6066 (cluster mode)

回答1:


I think the issue is caused by hostname to ip resolution order after VirtualBox is installed. I solved the problem by added my hostname and IP the my hosts file.

c:\Windows\System32\Drivers\etc\hosts

MY_HOST_NAME MY_IP



来源:https://stackoverflow.com/questions/36135742/spark-master-picking-up-wrong-ip-on-windows

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!