etl

How to pass user credentials to WSDL (web service) inside script task/SSIS?

﹥>﹥吖頭↗ 提交于 2020-02-05 03:28:41
问题 Requirement -I have user credentials to access a WSDL file and I can access the file via browser. WSDL link - XYZ userdID- asdf pwd - ***** I need to use C# code inside SSIS script task to call this web service (specifically a single method call) using appropriate user credentials. Get the response(which would be in XML) , parse the XML to get specific values from it and insert that into a SQL table. If anyone know/have any idea/reference link on how to achieve the above requirement, please

SSIS Package Runs from Catalog, not from SQL Server Agent (Same User)

二次信任 提交于 2020-02-05 02:39:07
问题 I have a SSIS package, developed for SQL Server 2012, that uses a script component to open an Excel workbook and execute a macro. This package runs in visual studio, and I am able to deploy to SQL Server 2012. In SQL Server, I am able to execute the package by selecting it from the Integration Services Catalogs and Right Click --> Execute... (with 32-bit runtime). I understand that this executes the package as the user that I am logged in to the server as. The package executes successfully in

Constraint database

痞子三分冷 提交于 2020-02-03 04:15:31
问题 I know the intuition behind constraint programming, so to say I never really experienced programming using a constraint solver. Although I think it is a different situation to be able to achieve what we would define as consistent data. Context: We have a set of rules to implement on a ETL server. These rules are either: acting on one row. acting inter-rows, in one or different tables. acting the same way between two runs (It should maintain the same constraint on all data, or just the last n

Etl工具的使用

北城以北 提交于 2020-01-28 12:02:17
ETL工具的使用 使用指南 开始 使用指南 这是一篇使用指南,不讲原理! 开始 双击文件夹 ‘data-integration’下的‘Spoon.bat’进入到以下界面 使用方法很简单 1.点击文件-新建-转换(转换的名称尽量不要使用中文) 2.演示思路:从同一个数据库中其中一张表的数据清洗到另一张表中 注意:字段可以不一样 3.点击 ‘输入’-‘表输入’ 4.点击拖到右边 5.同样操作点击‘输出’- ‘插入/更新’ 6.同样操作拖入到右边的控制台 7.按住‘shift’鼠标拖一条线把‘表输入’和‘插入/跟新’相连接 8.双击‘表输入’连接数据库和写sql 9.双击插入/跟新,同样连接数据库和调整字段 10.匹配字段,意思就是你想把第一张表中的字段的值给第二张表的字段匹配 11.点击运行 11.控制台有数据显示信息 12.如有问题随时提问,一定会尽力解决 来源: CSDN 作者: panbinglin 链接: https://blog.csdn.net/panbinglin/article/details/103583116

ETL工具KETTLE常用设计之——多表或整库数据全量迁移

隐身守侯 提交于 2020-01-27 01:26:29
一、思路: 有主作业、子作业两个作业组成这个流程,在主作业中,首先调用一个转换用来获取指定库中所有的表名、指定的表名等内容复制到结果记录,然后调用一个子作业,子作业中先调用一个转换用来结果记录中的信息,然后设置变量,变量有效范围为parent_job,然后子作业再调用一个转换,用来执行sql脚本创建表结构和获取源数据表中的数据再输出到对应的目标数据库中表中即可,因为字段名称是一致的,所以不需要指定字段映射;(如果是需要字段映射的,那么就不能这样多表或整库数据全量迁移了,只能每个表配置对应的转换进行抽取数据) 二、自动循环配置过程见博客: https://blog.csdn.net/LSY_CSDN_/article/details/92062738 来源: CSDN 作者: LSY_csdn_ 链接: https://blog.csdn.net/LSY_CSDN_/article/details/92063035

Java Pentaho Exception MongoDB

痴心易碎 提交于 2020-01-26 00:17:47
问题 I have designed a transformation in Pentaho Data Integration ui tool and wrote a java code to execute the transformation. I followed below resources link as it is, try { /** * Initialize the Kettle Enviornment */ KettleEnvironment.init(); /** * Create a trans object to properly assign the ktr metadata. * * @filedb: The ktr file path to be executed. * */ TransMeta metadata = new TransMeta("Districts.ktr"); Trans trans = new Trans(metadata); // Execute the transformation trans.execute(null);

How change the XPATH and XML output format in DataStage?

混江龙づ霸主 提交于 2020-01-25 08:04:33
问题 I HAVE some problem in XPATH In XML Output I Want The Output Like <categories> <category NAME="A"> <redcord a="1"></redcord> <redcord a="2"></redcord> </category> <category NAME="B"> <redcord b="ASD"></redcord> <redcord b="QWE"></redcord> </category> </categories> I HAVE XPATH LIKE : /categories/category[@NAME="A"]/redcord/@a AND /categories/category[@NAME="B"]/redcord/@b The Output that i've been making is <categories> <category NAME="A"> <redcord a="1" b="ASD"></redcord> <redcord a="2" b=

Empty string results in “Errors were detected in the command line arguments, please make sure all arguments are set correctly”

99封情书 提交于 2020-01-24 04:28:46
问题 On a SQL 2016 Server I have a job that calls an SSIS package. That package is in a project in the SSISDB and has parameters. One of those parameters is a string type that is blank as a default. I ran the job with the blank value for this parameter, and it ran successfully. I then opened the job properties, went to the step that calls that package and went into the configuration and gave that parameter a value. I ran the job again and it ran successfully, and the parameter value had the

Change data types from Advanced Editor vs Data Conversion Transformation

点点圈 提交于 2020-01-23 13:09:05
问题 I am using SSIS to create some packages. I have gotten confused around the Data Conversion Transformation component and changing columns data types from Advanced editor If I can go into the advanced editor and change the data type of my output, why would I need to input the Data Conversion Transformation? Is this just down to preference or is there a difference between using both approaches? 回答1: Before showing the difference between both approaches, i will try to give an overview of the data

Time parsing Issue using DateTime.ParseExact()

生来就可爱ヽ(ⅴ<●) 提交于 2020-01-21 22:05:11
问题 I have a Time Value stored as string with the format HH:mm I used the following code to parse it as date with Today Date DateTime.ParseExact("09:00","HH:mm",New System.Globalization.CultureInfi("En-GB")) The Result Is 2017-03-15 09:00:00 When Running this Code From another machine i get the following Result: 1899-12-31 09:00:00 I replaced my code to the following DateTime.ParseExact(DateTime.Now.ToString("yyyy-MM-dd") & " 09:00","yyyy-MM-dd HH:mm",New System.Globalization.CultureInfi("En-GB")