MMLSpark

集群提交lightGBM算法

烈酒焚心 提交于 2019-12-26 17:11:17
【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>> ## mmlspark https://mvnrepository.com/artifact/Azure/mmlspark/0.15 ## lightgbmlib https://mvnrepository.com/artifact/com.microsoft.ml.lightgbm/lightgbmlib/2.2.200 [root@hadoop-1-1 ~]# more lgbm.sh /app/spark2.3/bin/spark-submit \ --master yarn \ --jars /root/external_pkgs/mmlspark-0.15.jar,/root/external_pkgs/lightgbmlib-2.2.200.jar \ --class com.sf.demo.lgmClassifier /root/lgbm_demo.jar nohup sh lgbm.sh > lgbm_20191226_001.log 2>&1 & package com.xx.demo import org.apache.spark.SparkConf import org.apache.spark.sql.SparkSession import org.apache.spark.ml