kettle | 易学教程

KETTLE实现复杂的流程

阅读更多关于 KETTLE实现复杂的流程

KETTLE是一款将数据从来源端经过抽取（extract）、转换（transform）、加载（load）至目的端的非常好用的一款ETL工具。学会它，对于跨数据库的表处理或者定时生成文本，excel等常见需求会处理的很容易。上图就是一个很复杂的逻辑运算，包括了生成EXCEL、多表关联、调用存储过程、JS处理等均有涉及。不仅如此，还可以针对多表做简单的配置，实现成百上千个表的数据抽取处理工作。接下来，笔者就从简单到复杂，从单表操作到实际应用中的复杂情况逐一介绍。欢迎关注公众号，学习ETL、数据库，获取更多资料来源： https://www.cnblogs.com/dalaba/p/12013180.html

In pentaho..How to pass a text file which contains all the definition of the connection parameters in the job?

阅读更多关于 In pentaho..How to pass a text file which contains all the definition of the connection parameters in the job?

问题 I am using jdbc connection and i am passing parameters with example ${sample_db_connection} and that parameters has been defined in server in a text file as sample_db_connection=localhost and i want to pass the text file in the job step so that whenever the job ran and it found this parameter ,automatically it will take the value defined in text file. 回答1: You need to create a KTR file using " Property Input " as the input step and " Modified Java Script " Step to define the key value mapping

How can unzip file in kettle where zip content cyrillic

阅读更多关于 How can unzip file in kettle where zip content cyrillic

问题 I try to unzip file.zip with files (a, b, c) in pentaho kettle (file management -> unzip file). it working fine. But if i try to unzip file.zip with files (a, b, ж), for example, i have errors: 2016/01/18 17:46:17 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp 2016/01/18 17:46:17 - Unzip file - ERROR (version 6.0.1.0-386, build 1 from 2015-12-03 11.37.25 by buildguy) : Could not unzip file [file:///D:/projects

How can unzip file in kettle where zip content cyrillic

阅读更多关于 How can unzip file in kettle where zip content cyrillic

I try to unzip file.zip with files (a, b, c) in pentaho kettle (file management -> unzip file). it working fine. But if i try to unzip file.zip with files (a, b, ж), for example, i have errors: 2016/01/18 17:46:17 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp 2016/01/18 17:46:17 - Unzip file - ERROR (version 6.0.1.0-386, build 1 from 2015-12-03 11.37.25 by buildguy) : Could not unzip file [file:///D:/projects/loaders/loader_little_files/src.zip]. Exception : [MALFORMED] 2016/01/18 17:46:17 - Unzip file - ERROR

Kettle 学习知识总结（二）

阅读更多关于 Kettle 学习知识总结（二）

kettle 循环控制方式设置： Select_Count_IALClaim_TRG 转化内容如下图：将所有的表输入的查询结果复制到结果中。循环控制器配置，如图： Deal_IALClaim_fr_IALClaim_TRG 转化步骤，为要循环体中主要循环执行的内容来源： oschina 链接： https://my.oschina.net/u/225084/blog/215671

Kettle 学习知识总结(一)

阅读更多关于 Kettle 学习知识总结(一)

【基础知识】 Kettle 分为两部分 Job （作业）和 Transaction （转换） Job 的执行可以通过 kitten 组件 Transaction 的执行可以通过 pan 组件输入组件中是将所有查询后获取到的数据全部转给下一个节点使用，而不是一条数据执行到尾节点后，在从新执行下一条数据。数据有两种发送形式：分发：从数据源依次发送到下一个节点复制：将数据源的所有结果全部发送到下一个节点【注意事项】 1、 Kettle 中时间查询使用的格式为 yyyy/MM/dd HH:mm:ss 。但是在 db2 中查询条件中使用的时间格式是 yyyy-MM-dd HH:mm:ss ，所以在 SQL 语句中的查询条件必须统一格式为“ yyyy-MM-dd HH:mm:ss ”。这样才能正确查询。 2、在 SQL 语句中时间参数必须使用单引号或者双引号括起来，否则 SQL 语法正确。 3、在生成错误文件时，如果不想每次执行都创建一个文件，需要把“ Don not create file at start ”复选框勾上，这样就不会每次执行都生成文件了。 4、对于在 kettle 中使用 set variables 设置的参数，使不能在 javaScript 中调用的。 5、如果想要让执行改变为并行，使用“ Launch next entires in

Kettle '?' not working Table Input Step

阅读更多关于 Kettle '?' not working Table Input Step

问题 I want to get all the table names from the database and then get all the rows from the tables. So I created a transformation like this: Get Table Names: Added the database connection and stored the table name in a output field called "tablename". Table Input: Marked "Replace variables in script" and "Execute for each row". Added the first step in "Insert data from step". SQL is "SELECT * from ?". I have read up a lot of tutorials online, including the documentation. My problem is that

Using PDI transformation in Pentaho BI server as data source for report parameters

阅读更多关于 Using PDI transformation in Pentaho BI server as data source for report parameters

问题 Any advice on how to use PDI transformation as data source for report parameters in BI server's console? I've uploaded the prpt reports to BI server but the I get a message "Error parsing parameter information". The .prpt and .ktr files are both in the same directory. 回答1: Actually, just realized that the issue could be solved by adding Transformation (KTR) as a resource. In this case, one can use the File-Resources menu selection. In the dialog select the transformation you wish to import

Filling data gaps in a stream in Pentaho Data Integration, is it possible?

阅读更多关于 Filling data gaps in a stream in Pentaho Data Integration, is it possible?

问题 I have a CSV file with currency exchanges EUR-USD. The file was downloaded from the Bank of Canada. I downloaded the CSV with data since Oct 10th, 2013 onwards. There are, nevertheless, gaps in the data, ie. days without the conversion rates. I've been fighting (1st day with Spoon Kettle) to find out a simple (but general) way to fill the gaps, say, with the last non-null value. And the only way I've managed to accomplish this is by chaining 4 "Get previous row fields" and the using the NVL

（BUG）Kettle8.1.0.0-365注册自定义插件BUG

阅读更多关于（BUG）Kettle8.1.0.0-365注册自定义插件BUG

写了两个kettle插件，这两个插件是两个插件项目，且在两个分类，开发完成打包到放到kettle的plugins目录下，启动kettle，结果在kettle设计器中只显示了一个插件。后来改了插件名称，插件在plugins文件夹中的文件夹的名字，发现始终是先扫描到谁就可以注册进来，后续的不管有几个都进不来，后果跟了一下kettle的源代码，发现是kettle里面存储插件TreeSet的Comparator写的有问题。负责注册插件的类org.pentaho.di.core.plugins.PluginRegistry中声明了一个Map存放所有的插件类型及插件。key是插件类型如作业项插件、步骤插件类型等，value是一个TreeSet，TreeSet中存储的是改插件类型所有的插件。（Kettle8.1.0.0-365版本在PluginRegistry的91行） private final Map<Class<? extends PluginTypeInterface>, Set<String>> categoryMap = new HashMap<>(); 下面看一下注册插件的方法 public void registerPlugin( Class<? extends PluginTypeInterface> pluginType, PluginInterface plugin )