Run my own application master on a specific node in a YARN cluster

Question

First of all, I'm using Hadoop-2.6.0. I want to launch my own app master on a specific node in a YARN cluster in order to open a server on a predetermined IP address and port. To that end, I wrote a driver program in which I created a ResourceRequest object and called setResourceName method to set a hostname, and attached it to a ApplicationSubmissionContext object by calling setAMContainerResourceRequest method.

I tried several times but couldn't launch the app master on a specific node. After searching code, I found that RMAppAttemptImpl invalidates what I've set in ResourceRequest as follows:

    // Currently, following fields are all hard code,                                                                                                                                                                                                                                                                   
    // TODO: change these fields when we want to support                                                                                                                                                                                                                                                                
    // priority/resource-name/relax-locality specification for AM containers                                                                                                                                                                                                                                            
    // allocation.                                                                                                                                                                                                                                                                                                      
    appAttempt.amReq.setNumContainers(1);
    appAttempt.amReq.setPriority(AM_CONTAINER_PRIORITY);
    appAttempt.amReq.setResourceName(ResourceRequest.ANY);
    appAttempt.amReq.setRelaxLocality(true);

Is there another way to launch a container for an application master on a specific node in Hadoop-2.6.0?

Answer 1

ApplicationSubmissionContext provides a function setAMContainerResourceRequest where you can pass a ResourceRequest object, In ResourceRequest object you can set "false" the RelaxLocality flag, which tells the ResourceManager if the application wants locality to be loose (i.e. allows fall-through to rack or any) or strict (i.e. specify hard constraint on resource allocation). Then set the hostname using setResourceName. I hope it helps!