etl

wrong Dates values in ssis

旧巷老猫 提交于 2019-12-10 17:33:43
问题 I am working with ssis. I have a column that contains date in format integer YYYYMMDD : For example to day I get 20190214 .I am using derived column to convert it in format date YYYY-MM-DD in my case for example 2019-02-14 But sometimes I receive wrong values for example 20188101 . How could I test that I have good values to be converted to date? 回答1: There are 3 methods to achieve that (1) Using Another Derived Column Beside of the first Derived COlumn, you can add another derived column

groupby计数_折线图

送分小仙女□ 提交于 2019-12-10 17:21:52
groupby计数_折线图 date_count = data [ [ 'cust_isn' , 'last_etl_acg_dt' ] ] . groupby ( 'last_etl_acg_dt' , as_index = False ) . count ( ) plt . figure ( figsize = [ 12 , 6 ] ) plt . plot ( date_count . iloc [ : , 0 ] , date_count . iloc [ : , 1 ] , '.-' ) plt . xticks ( rotation = 45 ) plt . grid ( ) plt . ylim ( [ 0 , 200000 ] ) plt . show ( ) 来源: CSDN 作者: 御剑归一 链接: https://blog.csdn.net/wj1298250240/article/details/103477465

SSIS Script Task - VB - Date is extracting as INT instead of date string

爱⌒轻易说出口 提交于 2019-12-10 16:15:14
问题 I am using the code below to build a sql input statement, which extracts data row by row from an excel sheet. v_sql_input_string += "'" & Trim(Replace(Convert.ToString(.Rows(i).Item(3)), "'", "''")) & "', " One of my columns is formatted as "Custom" MM/dd/yyyy format in excel. Date displays as 01/22/2018 in excel as well. When being extracted into this string, it reformats as the date INT. Example 43102. I'm somewhat new to VB, so forgive me for my ignorance. What am I doing wrong here? 回答1:

How to prevent CAST errors on SSIS?

跟風遠走 提交于 2019-12-10 15:49:57
问题 The question Is it possible to ask SSIS to cast a value and return NULL in case the cast is not allowed instead of throwing an error ? My environment I'm using Visual Studio 2005 and Sql Server 2005 on Windows Server 2003. The general context Just in case you're curious, here is my use case. I have to store data coming from somewhere in a generic table (key/value structure with history) witch contains some sort of value that can be strings, numbers or dates. The structure is something like

Copy results from a PostgreSQL view in one DB to a table in another

流过昼夜 提交于 2019-12-10 15:22:06
问题 Complete PostgreSQL newb. I have 7-8 views worth of data in db1 that I need to copy into tables with a matching schemae (schemas?) in a different database, db2. The destination database may be the same instance of PostgreSQL, or one on a different box altogether. I know 2-3 different ways to accomplish this with the databases I'm familiar with, but I'm helpless on this one. Can someone suggest some basic strategies for me? In a perfect world, I'd prefer not to have to do anything that feels

Unable to pass a variable value to a stored procedure in ssis

☆樱花仙子☆ 提交于 2019-12-10 14:50:02
问题 When executing an SSIS package, the following error appears: [OLE DB Source [83]] Error: The SQL command requires a parameter named "@Sales_person", which is not found in the parameter mapping. [SSIS.Pipeline] Error: OLE DB Source failed the pre-execute phase and returned error code 0xC0207014. Below is the screenshot of my OLE DB Source editor I do see Param direction tab in Set Query parameters, how is that used? In my case will I be using Input or Output or InputOutput 回答1: After searching

Stop running Kettle Job/Transformation using Java

☆樱花仙子☆ 提交于 2019-12-10 13:37:35
问题 I'm developing a web-app based ETL too (with Kettle engine), using Java. I'm running into issues, while trying to stop a running Job. I'm not sure if using the CarteSingleton.java is correct. I'm using a custom singleton map. My code is as below Job job = new Job(null, jobMeta); job.setLogLevel(LogLevel.DETAILED); job.setGatheringMetrics(true); job.start(); Once job.start() is invoked, I'm trying to store that job object in a custom singleton map and retrieve the exact Job object that was

How to avoid manually browsing DLL in Add Reference of Script Task when deploying package on production?

孤街浪徒 提交于 2019-12-10 13:25:24
问题 I use EPPlus.dll library for generating Excel files on the fly for attachment and generate mailer in Script Task of SSIS package. When there is new requirement for change in mailer comes, I do the change in Script Task on my local machine and send the built package file ( .dtsx ) to DBA team for deployment. Now everytime I have to ask the DBA team to share production server screen with me where I: Open the file in Visual Studio Data Tools solution Browse to the dll location Add Reference to

OrientDB import edges only using ETL tool

 ̄綄美尐妖づ 提交于 2019-12-10 11:29:39
问题 I already used the OETL to insert all my Vertex to the graph. Now I have a file that outlines the edges at the following way: node_1,rel_type,node_2 11000001,relation_A,10208879 11000001,relation_A,10198662 11000001,relation_B,10159927 11000001,relation_C,10165779 How can I import it using the OrientDB OETL tool? I tried the following: "transformers": [ { "csv": {} }, { "command" : { "command" : "create edge ${rel_type} from (select flatten(@rid) from V where node_id= ${node_1}) to (select

PowerShell script to extract .xls file from specific Outlook folder

眉间皱痕 提交于 2019-12-10 10:44:54
问题 I want to extract and save an .xls file from an email I receive daily. I have a rule set up which saves the email in an Outlook mailbox, within a specific subfolder of the Inbox. The Outlook folder structure looks like this: -> Inbox --> Data (subfolder of "Inbox") ---> ToExtract (subfolder of "Data") I need to extract the .xls file from the "ToExtract" folder. I found a script that does most of the work for me, but it requires the user to supervise the script and manually select which