Pentaho-kettle: Need to create ETL Jobs dynamically based on user input

*爱你&永不变心* 提交于 2019-12-06 00:07:37

问题


In my application, user can specify the format of their file. Based on user input we dynamically create SSIS package.

http://lakshmik.blogspot.com/2005/05...eate-ssis.html

Dynamically created SSIS package is used for processing user's files.

We want to evaluate Pentaho-Kettle for this requirement. Is this possible with Kettle to dynamically create ETL jobs based on user's inputs?

If not Pentaho, is there any Java ETL tool which allows use to dynamically create ETL jobs?


回答1:


I dont know about others, but this is traditionally quite tricky in Kettle but people have done it in various ways.

The best option for this is the (brand new) inject step which lets you do really clever stuff with metadata - but it only works for some basic steps. I think it will do what you want, read about it in Matt Casters (PDI Creator and god) blog here:

http://www.ibridge.be/?s=inject&submit=Go

If that doesnt work; then your other options are to go down the generic field name route (nasty) or dynamically generate the transformation. This is easier than it sounds - but you will need to get much more heavily involved in the Java side than usual for an ETL tool.




回答2:


It is possible, and not very hard.

You can use the Kettle API to dynamically create transformations that can do anything Kettle does. The GUI designer uses the API to create transformations, so anything you can do with the GUI, you can do thru the API.

If you look in the 'test' source tree you will find lots of examples of how to create transformations dynamically.



来源:https://stackoverflow.com/questions/6610759/pentaho-kettle-need-to-create-etl-jobs-dynamically-based-on-user-input

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!