发表新帖

发表新帖

Difference between `yarn.scheduler.maximum-allocation-mb` and `yarn.nodemanager.resource.memory-mb`?

前端未结

关注

 1  1255

夕颜 2020-12-07 18:08

What is difference between yarn.scheduler.maximum-allocation-mb and yarn.nodemanager.resource.memory-mb?

I see both of these in yarn-


      
      
        
          1条回答        

        
                    
            
            
                         
                
              
              
                
                   孤城傲影
                                             
                
                
                (楼主)
            
              
              
                2020-12-07 18:25
              

            
            
                        
Consider in a scenario where you are setting up a cluster where each machine having 48 GB of RAM. Some of this RAM should be reserved for Operating System and other installed applications.



yarn.nodemanager.resource.memory-mb:

Amount of physical memory, in MB, that can be allocated for containers. It means the amount of memory YARN can utilize on this node and therefore this property 
should be lower than the total memory of that machine.

yarn.nodemanager.resource.memory-mb
40960 




The next step is to provide YARN guidance on how to break up the total resources available into Containers. You do this by specifying the minimum unit of RAM to allocate for a Container.

In yarn-site.xml

yarn.scheduler.minimum-allocation-mb 
     
 
        热议问题