amazon-athena

Athena is analytics service for retrieving data from s3 using sql query. I have queried data in s3 using t aws console Need access to aws athena using nodejs code I am using athena like following way in my nodejs project : download JDBC driver from AWS . Create a connector.js file. npm install jdbc NPM . Paste followings: var JDBC = require('jdbc'); var jinst = require('jdbc/lib/jinst'); if (!jinst.isJvmCreated()) { jinst.addOption("-Xrs"); jinst.setupClasspath(['./AthenaJDBC41-*.jar']); } var config = { // Required url: 'jdbc:awsathena://athena.*.amazonaws.com:443', // Optional drivername:

I have been playing around with AWS Glue for some quick analytics by following the tutorial here While I have been able to successfully create crawlers and discover data in Athena, I've had issues with the data types created by the crawler. The date and timestamp data types get read as string data types. I followed this up by creating an ETL job in GLUE using the data source created by the crawler as the input and a target table in Amazon S3 As part of the mapping transformation, I converted the data types of the date and timestamp as string to timestamp but unfortunately the ETL converted

Amazon Athena Log Analysis Services with S3 Glacier We have petabytes of data in S3. We are https://www.pubnub.com/ and we store usage data in S3 of our network for billing purposes. We have tab delimited log files stored in an S3 bucket. Athena is giving us a HIVE_CURSOR_ERROR failure. Our S3 bucket is setup to automatically push to AWS Glacier after 6 months. Our bucket has S3 files hot and ready to read in addition to the Glacier backup files. We are getting access errors from Athena because of this. The file referenced in the error is a Glacier backup. My guess is the answer will be: don't

I'm getting an HIVE_PARTITION_SCHEMA_MISMATCH error that I'm not quite sure what to do about. When I look at the 2 different schemas, the only thing that's different is the order of the keys in one of my structs (created by a glue crawler). I really don't care about the order of the data, and I'm receiving the data as a JSON blob, so I cannot guarantee the order of the keys. struct<device_id:string,user_id:string,payload:array<struct<channel:string,sensor_id:string,type:string,unit:string,value:double,name:string>>,topic:string,channel:string,client_id:string,hardware_id:string,timestamp

From the documentation, AWS::Athena::NamedQuery , it is unclear how to attach Athena to an S3 bucket specified in the same stack. If I had to guess from the example , I would imagine that you can write a template like, Resources: MyS3Bucket: Type: AWS::S3::Bucket ... other params ... AthenaNamedQuery: Type: AWS::Athena::NamedQuery Properties: Database: "db_name" Name: "MostExpensiveWorkflow" QueryString: > CREATE EXTERNAL TABLE db_name.test_table (...) LOCATION s3://.../path/to/folder/ Would a template like the above work? Upon stack creation, will the table db_name.test_table be available to

I have a set of Json data files that look like this [ {"client":"toys", "filename":"toy1.csv", "file_row_number":1, "secondary_db_index":"4050", "processed_timestamp":1535004075, "processed_datetime":"2018-08-23T06:01:15+0000", "entity_id":"4050", "entity_name":"4050", "is_emailable":false, "is_txtable":false, "is_loadable":false} ] I have created a Glue Crawler with the following custom classifier Json Path $[*] Glue returns the correct schema with the columns correctly identified. However, when I query the data on Athena... all the data is landing in the first column and the rest of the