etl

问题 So I have around 13 years of stock market data of daily low high open close. The problem is the markets are closed sometimes in between and hence Monday to Friday might not appear continuously sometimes. Look below Date Day Open High Low Close Adjusted Close 0 17-09-2007 Monday 6898 6977.2 6843 6897.1 6897.100098 1 18-09-2007 Tuesday 6921.15 7078.95 6883.6 7059.65 7059.649902 2 19-09-2007 Wednesday 7111 7419.35 7111 7401.85 7401.850098 3 20-09-2007 Thursday 7404.95 7462.9 7343.6 7390.15 7390

问题 I am unable to apply transformation using below code. getting error The year function doesn't support dt_wstr. The expression iam using is: (DT_I4)((DT_WSTR,4)YEAR(fisc_wk_end_dt) + RIGHT("0" + (DT_WSTR,2)MONTH(fisc_wk_end_dt),2) + RIGHT("0" + (DT_WSTR,2)DAY(fisc_wk_end_dt),2)) 回答1: Problem From the expression you mentioned, it looks like fisc_wk_end_dt column data type is string while YEAR, MONTH, DAY function's parameters must be of type date. From the official documentation: Syntax YEAR

问题 I have lookup component a with a lookup table that retusn a varchar(4) column with 3 possible values: "T", "R" or "" (empty string). I'm using an OLE DB connection for the lookup table, and have tried direct access to the table, as well as specifying a query with an RTRIM() on the column, to get sure that the string is empty and not a "blank string of some length". If I set the cache mode to "Partial cache" everything works fine (either with direct reading of the table, or using the trimming

问题 In an SSIS package that I'm writing, I have a CSV file as a source. On the Connection Manager General page, it has 65001 as the Code page (I was testing something). Unicode is not checked. The columns map to a SQL Server destination table with varchar (among others) columns. There's an error at the destination: The column "columnname" cannot be processed because more than one code page (65001 and 1252) are specified for it. My SQL columns have to be varchar , not nvarchar due to other

问题 In an SSIS package that I'm writing, I have a CSV file as a source. On the Connection Manager General page, it has 65001 as the Code page (I was testing something). Unicode is not checked. The columns map to a SQL Server destination table with varchar (among others) columns. There's an error at the destination: The column "columnname" cannot be processed because more than one code page (65001 and 1252) are specified for it. My SQL columns have to be varchar , not nvarchar due to other

问题 In an SSIS package that I'm writing, I have a CSV file as a source. On the Connection Manager General page, it has 65001 as the Code page (I was testing something). Unicode is not checked. The columns map to a SQL Server destination table with varchar (among others) columns. There's an error at the destination: The column "columnname" cannot be processed because more than one code page (65001 and 1252) are specified for it. My SQL columns have to be varchar , not nvarchar due to other

问题 I am new to AWS. I want to write the ETL script using AWS Glue to transfer the data from one mysql database to another RDS mysql database . Please suggest me to how to do this job using AWS glue Thanks 回答1: You can use pymysql or mysql.connector as a seperate zip file added to the glue job. We have used pymysql for all our production jobs running in AWS Glue/Aurora RDS Use this connectors to connect to both the RDS Mysql instances. Read data from RDS Source db1 into a dataframe, perform the

问题 I've successfully run an ML Pipeline experiment and published the Azure ML Pipeline without issues. When I run the following directly after the successful run and publish (i.e. I'm running all cells using Jupyter), the test fails! interactive_auth = InteractiveLoginAuthentication() auth_header = interactive_auth.get_authentication_header() rest_endpoint = published_pipeline.endpoint response = requests.post(rest_endpoint, headers=auth_header, json={"ExperimentName": "***redacted***",

问题 I have a table over which I have built an ETL service. Goods records (arrival / departure) go to the table. I have done that my table will be erased. When the item identifier arrives in the database for the second time, both records are deleted. label cost time x2 29 14/5/2020 01:00:00 x3 20 14/5/2020 01:02:00 x2 29 15/5/2020 03:12:02 Now ETL service remove records (every 30s): label cost time x3 20 14/5/2020 01:02:00 I delete it using the function: with todelete as ( select *, count(*) over

问题 First I would like to say that I am new to the stackoverflow community and relatively new to SQL itself and so please pardon me If I didn't format my question right or didn't state my requirements clearly. I am trying to implement a type 2 SCD in Oracle. The structure of the source table ( customer_records ) is given below. CREATE TABLE customer_records( day date, snapshot_day number, vendor_id number, customer_id number, rank number ); INSERT INTO customer_records (day,snapshot_day,vendor_id