How to submit a topology in storm production cluster using IDE

后端 未结 4 823
臣服心动
臣服心动 2020-12-25 15:02

I am facing an issue Must submit topologies using the \'storm\' client script so that StormSubmitter knows which jar to upload while submitting a topology to a

4条回答
  •  伪装坚强ぢ
    2020-12-25 15:17

    I have resolved this this problem based on @abhi and @Nishu Tayal's answers, I'd like to post my code here:

    public static void submitLocalTopologyWay1(String topologyName, Config topologyConf, 
            StormTopology topology, String localJar) {
        try {
            //get default storm config
            Map defaultStormConf = Utils.readStormConfig();
            defaultStormConf.putAll(topologyConf);
    
            //set JAR
            System.setProperty("storm.jar",localJar);
    
            //submit topology
            StormSubmitter.submitTopology(topologyName, defaultStormConf, topology);
    
        } catch (Exception e) {
            String errorMsg = "can't deploy topology " + topologyName + ", " + e.getMessage();
            System.out.println(errorMsg);
            e.printStackTrace();
        } 
    }
    
    public static void submitLocalTopologyWay2(String topologyName, Config topologyConf, 
            StormTopology topology, String localJar) {
        try {
            //get nimbus client
            Map defaultStormConf = Utils.readStormConfig();
            defaultStormConf.putAll(topologyConf);
            Client client = NimbusClient.getConfiguredClient(defaultStormConf).getClient();
    
            //upload JAR
            String remoteJar = StormSubmitter.submitJar(defaultStormConf, localJar);
    
            //submit topology
            client.submitTopology(topologyName, remoteJar, JSONValue.toJSONString(topologyConf), topology);
    
        } catch (Exception e) {
            String errorMsg = "can't deploy topology " + topologyName + ", " + e.getMessage();
            System.out.println(errorMsg);
            e.printStackTrace();
        } 
    }
    

    then here is a test, and you must build your code to a JAR file first.

    public void testSubmitTopologySubmitLocalTopologyWay1() {   
        Config config = new Config();
        config.put(Config.NIMBUS_HOST,"9.119.84.179");   
        config.put(Config.NIMBUS_THRIFT_PORT, 6627);
        config.put(Config.STORM_ZOOKEEPER_SERVERS, Arrays.asList("9.119.84.177","9.119.84.178","9.119.84.176")); 
        config.put(Config.STORM_ZOOKEEPER_PORT,2181);
    
        config.put(Config.TOPOLOGY_WORKERS, 3);
    
        RemoteSubmitter.submitLocalTopologyWay1("word-count-test-1", config, 
                WordCountTopology.buildTopology(), // your topology
                "C:\\MyWorkspace\\project\\storm-sample-0.0.1-SNAPSHOT-jar-with-dependencies.jar");//the JAR file
    }
    

提交回复
热议问题