snowflake-schema

How to avoid sub folders in snowflake copy statement

て烟熏妆下的殇ゞ 提交于 2021-02-11 18:15:51
问题 I have a requirement to exclude certain folder from prefix and process the data in snowflake (Copy statement) In the below example I need to process files under emp/ and exclude files from abc/ Input : s3://bucket1/emp/ E1.CSV E2.CSV /abc/E11.csv s3://bucket1/emp/abc/ - E11.csv Output : s3://bucket1/emp/ E1.CSV E2.CSV Is there any suggestion around pattern to handle this ? 回答1: With the pattern keyword you can try to exclude certain files. However when using the pattern matching with the NOT

How to avoid sub folders in snowflake copy statement

柔情痞子 提交于 2021-02-11 18:14:15
问题 I have a requirement to exclude certain folder from prefix and process the data in snowflake (Copy statement) In the below example I need to process files under emp/ and exclude files from abc/ Input : s3://bucket1/emp/ E1.CSV E2.CSV /abc/E11.csv s3://bucket1/emp/abc/ - E11.csv Output : s3://bucket1/emp/ E1.CSV E2.CSV Is there any suggestion around pattern to handle this ? 回答1: With the pattern keyword you can try to exclude certain files. However when using the pattern matching with the NOT

How to retrieve all the catalog names , schema names and the table names in a database like snowflake or any such database?

拟墨画扇 提交于 2021-02-11 12:32:28
问题 I need to drop some columns and uppercase the data in snowflake tables. For which I need to loop through all the catalogs/ dbs, its respective schemas and then the tables. I need this to be in python to list of the catalogs schemas and then the tables after which I will be exicuting the SQL query to do the manipulations. How to proceed with this? 1.List all the catalog names 2.List all the schema names 3.List alll the table names I have established a connection using python snowflake

List all columns wtih datatype in specific table in Snowflake

旧街凉风 提交于 2021-01-28 14:09:20
问题 I am looking for a programmatic way to get the Snowflake table schema, is there a way for that? 回答1: You can use a SHOW command to list table columns e.g. SHOW COLUMNS IN TABLE MYSCHEMA.MYTABLE; Note that SHOW commands have the advantage that they do not require a running warehouse to execute, so are zero cost queries. 回答2: Use this query: select ordinal_position as position, column_name, data_type, case when character_maximum_length is not null then character_maximum_length else numeric

How to run snowflake side effect functions like SYSTEM$GENERATE_SCIM_ACCESS_TOKEN within a procedure with owner rights?

两盒软妹~` 提交于 2020-07-16 07:59:07
问题 Basically I want to do SCIM integration in snowflake. For that I have to use this command for getting the token which will be passed to Azure AD: call system$generate_scim_access_token('<value>'); This command can only run with AccountAdmin. And running it with AccountAdmin I am able to get token but In future I will not be having rights of AccountAdmin, so for that what I did, I created a procedure with AccountAdmin and execute it as owner. So that, when ever any other role which is having

Significance of Constraints in Snowflake

喜欢而已 提交于 2020-07-08 13:22:55
问题 Snowflake allows UNIQUE, PRIMARY KEY, FOREIGN KEY and NOT NULL constraints but I read that it enforces only NOT NULL constraint. Then what is the purpose of other keys and under what circumstances do we have to define them? I appreciate any examples. Thank you, Prashanth. 回答1: They express intent, helping people understand your data models. Data modeling tools can use them to generate diagrams. You can also programmatically access them to validate data integrity yourself. 来源: https:/

Adding current date in create script for Snowflake

流过昼夜 提交于 2020-06-29 05:24:50
问题 I have a requirement where I have to create tables with the date/datetime in the table name when they were created dynamically.Wondering if this option is possible in Snowflake? Eg: I would need somethinglike this. CREATE TABLE someNewTable_YYYYMMDD Thank you for your responses; Best, AB 回答1: You can achieve this using SQL variables and the IDENTIFIER keyword. Here's an example that adds the current date into the table-name: SET table_name=(SELECT 'someNewTable_' || TO_VARCHAR(CURRENT_DATE(),

SnowFlake Query if an Id exists on each of the last 7 days

假如想象 提交于 2020-01-06 05:18:04
问题 We INSERT new records every day to a table with say id and created_on column. How do i identify if records with a particular identifier existed every day in the last 7 days ? 回答1: This can be done with a Stored Procedure: CREATE OR REPLACE PROCEDURE TIME_TRAVEL(QUERY TEXT, DAYS FLOAT) RETURNS VARIANT LANGUAGE JAVASCRIPT AS $$ function run_query(query, offset) { try { var sqlText = query.replace('"at"', " AT(OFFSET => " + (offset + 0) + ") "); return (snowflake.execute({sqlText: sqlText}))

How to use the percent key word in the snowflake query

不打扰是莪最后的温柔 提交于 2019-12-13 22:38:39
问题 SELECT NBR, Customers, status FROM ( SELECT NBR, Customers, Code AS status FROM CCC AS CS INNER JOIN AAA AS AC ON CCC.B2= ACT.B1 AND CSS.B2 = ACT.B1 ) AS rst WHERE status IN ('A', 'T') ORDER BY NBR LIMIT 100 PERCENT 回答1: I saw you other post. Not 100% sure what you are trying to do, but you might want to consider using a windows function like ratio_to_report. See the example below. Here is a link to the windows functions https://docs.snowflake.net/manuals/sql-reference/functions-analytic.html

Star vs Snowflake schema in data warehousing?

我的梦境 提交于 2019-12-03 14:14:48
问题 Currently, I've been involved in an warehouse based intelligent transaction analysis banking system featuring customer churn behavior, fraud detection & CRM analysis. We've been using Oracle as the database & it's completely a data warehousing project with data mining algorithms used for analysis. We have records of about 1000 customers of a bank. For modeling, whether it is better to use the star schema or snowflake schema or constellation schema? I know the basic difference of star and