azkaban 多执行端部署验证

我的未来我决定 提交于 2019-12-31 13:31:45

azkaban web 端添加配置 azkaban.properties:

azkaban web 端添加配置  azkaban.properties	
azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1
azkaban.activeexecutor.refresh.milisecinterval=1000
azkaban.activeexecutor.refresh.flowinterval=1
azkaban.executorinfo.refresh.maxThreads=1
Servers	Executor servers:
192.168.11.164
192.168.11.142
192.168.11.105

Azkaban web:
192.168.11.105

Mysql:
192.168.11.86
use azkaban;	
 	 

NO.	Operation	Tables & SQL	Result(my test result sample)
1	After R&D config the azkaban web/executors and restart,executors should be automatically registered successfully;	Check the registration results:
select * from  executors;	| id | host  | port  | active |
+----+-----------------------------------+-------+--------+
| 62 | THA-168-11-164.h.chinabank.com.cn | 12321 |      1 |
| 64 | THA-168-11-142.h.chinabank.com.cn | 12321 |      1 |
| 66 | THA-168-11-105.h.chinabank.com.cn | 12321 |      1 
2	Execute the old Azkaban job, its should be randomly assigned to one of the executors ;	
Check this job's executor id:
SELECT executor_id FROM `execution_flows` where exec_id= 	executor_id |
+-------------+
|          64 |
3	Create data ingestion job,it should be randomly assigned to one of the executors ;	Check the executor id :
SELECT executor_id FROM `execution_flows` where exec_id=	executor_id |
+-------------+
|          66 |
4	Execute multiple Azkaban jobs ,they assigned to the executor randomly;	Check the executors id:
SELECT executor_id FROM `execution_flows` where exec_id=;	executor_id |
+-------------+
|          64 |
|          66 |
|          64 |
|          66 |
|          64 |
|          62 |
|          64 |
5	The same Azkaban job execute on the different executor servers(Run the job many times manually);	Check the executors id:
SELECT project_id, executor_id FROM `execution_flows` ORDER BY exec_id desc LIMIT 2;	project_id | executor_id |
+------------+-------------+
|       2562 |          66 |
|       2562 |          64 |
6	When Azkaban job is running, stop the server it is using,this job's execution result should be failed, re-executed it manually should be run on another executor:	Check the executors id:
SELECT project_id, exec_id,executor_id FROM `execution_flows` ORDER BY exec_id desc limit 5;	| project_id | executor_id |
+------------+-------------+
|       2581 |          64 |
|       2581 |          66 |
7	Multiple Azkaban jobs are executed at the same time,they should be randomly assigned to the executor;	Check the executors id:
SELECT project_id, exec_id,executor_id FROM `execution_flows` ORDER BY exec_id desc limit 55;(Since we reconfigured 105 server, its id changed from 66 to 68)	 project_id | exec_id | executor_id |
+------------+---------+-------------+
|       2624 |   88642 |          62 |
|       2627 |   88641 |          62 |
|       2619 |   88640 |          62 |
|       2353 |   88639 |          64 |


标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!