issue in file creation while running topology in remote cluster using storm

问题

I have created a topology which should read from a file and write it to a new file. My program is running properly in local cluster but while submitting in remote cluster i am not getting any error but file is not getting created. below is my code to submit topolgy in remote cluster :-

public static void main(String[] args)  {
        final Logger logger = LoggingService.getLogger(FileToFileTopology.class.getName());

        try{
        Properties prop =new Properties();
        prop.load(new FileInputStream(args[0]+"/connection.properties"));

        LoggingService.generateAppender("storm_etl",prop, "");
        logger.info("inside main method...." +args.length);

        System.out.println("inside main sys out");

        Config conf= new Config();
        conf.setDebug(false);
        conf.put(Config.TOPOLOGY_MAX_SPOUT_PENDING,1);
        TopologyBuilder builder = new TopologyBuilder();

            builder.setSpout("file-reader",new FileReaderSpout(args[1]));
            builder.setBolt("file-writer",new WriteToFileBolt(args[1]),2).shuffleGrouping("file-reader");
            logger.info("submitting topology");
            StormSubmitter.submitTopology(args[0], conf, builder.createTopology());


    }
    catch(Exception e){
        System.out.println("inside catch");
        logger.info("inside catch"+e.getMessage());
        logger.error("inside error", e);
        e.printStackTrace();
    }
    }

I have also used log4j to create my own logfile for my topology, log file gets created but no error in my log file. pls help

回答1:

I had same issue with Hortonworks2.2. This happened because of permissions.

Even if you are submitting to the cluster as Root user, when submitting the storm jar command, it executes as 'storm' user. It can read the file from source, but won't write, because it doesn't have the necessary rights.

Modify the permissions of destination folder where you want to write file with all permissions.

chmod 777 -R /user/filesfolder

来源：https://stackoverflow.com/questions/19494125/issue-in-file-creation-while-running-topology-in-remote-cluster-using-storm