bigdata

问题 i have problem with ordering data in cassandra Database . this is my table structure: CREATE TABLE posts ( id uuid, created_at timestamp, comment_enabled boolean, content text, enabled boolean, meta map<text, text>, post_type tinyint, summary text, title text, updated_at timestamp, url text, user_id uuid, PRIMARY KEY (id, created_at) ) WITH CLUSTERING ORDER BY (created_at DESC) and when i run this query , i got the following message: Query : select * from posts order by created_at desc;

问题 I'm trying to graph a huge data set (about 1.6 million points) using Kendo UI. This number is too large, but I have figured out that many of these points are repeating. The data is currently stored in this format: [ [x,y], [x,y], [x,y]...] with each x and y being a number, thus each subarray is a point. The approach I have in mind is to create a second empty array, and then loop through the very long original array, and only push each point to the new one if it isn't already found there. I

问题 I have read this article. It states, that Neo4j can scale horizontally, but only to increase read performance and fault tolerance... so the stored graph is copied to each server in a cluster. But what if I have a dataset that is larger than one server can store? Does Neo4j fail in this situation? Do I have to scale vertically in this situation and buy larger HDD? Thank you 回答1: Yes. You need enough hard drive space to contain the full graph on all nodes of the cluster, no way around that. If

问题 In Hive, you can create two kinds of tables: Managed and External In case of managed table, you own the data and hence when you drop the table the data is deleted. In case of external table, you don't have ownership of the data and hence when you delete such a table, the underlying data is not deleted. Only metadata is deleted. Now, recently i have observed that you can not create an external table over a location on which you don't have write (modification) permissions in HDFS. I completely

问题 I'm trying to sort a big table stored in a file. The format of the file is (ID, intValue) The data is sorted by ID , but what I need is to sort the data using the intValue , in descending order. For example ID | IntValue 1 | 3 2 | 24 3 | 44 4 | 2 to this table ID | IntValue 3 | 44 2 | 24 1 | 3 4 | 2 How can I use the Linux sort command to do the operation? Or do you recommend another way? 回答1: How can I use the Linux sort command to do the operation? Or do you recommend another way? As others

问题 I'm trying to sort a big table stored in a file. The format of the file is (ID, intValue) The data is sorted by ID , but what I need is to sort the data using the intValue , in descending order. For example ID | IntValue 1 | 3 2 | 24 3 | 44 4 | 2 to this table ID | IntValue 3 | 44 2 | 24 1 | 3 4 | 2 How can I use the Linux sort command to do the operation? Or do you recommend another way? 回答1: How can I use the Linux sort command to do the operation? Or do you recommend another way? As others

问题 Let's say I have a length 30 array with 4 bad values in it. I want to create a mask for those bad values, but since I will be using rolling window functions, I'd also like a fixed number of subsequent indices after each bad value to be marked as bad. In the below, n = 3: I would like to do this as efficiently as possible because this routine will be run many times on large data series containing billions of datapoints. Thus I need as close to a numpy vectorized solution as possible because I

问题 i have an external partitioned table named employee with partition(year,month,day), everyday a new file come and seat at the particular day location call for today's date it will be at 2016/10/13. TABLE SCHEMA: create External table employee(EMPID Int,FirstName String,.....) partitioned by (year string,month string,day string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/.../emp'; so everyday we need to run command which is working fine as ALTER TABLE

问题 i have an external partitioned table named employee with partition(year,month,day), everyday a new file come and seat at the particular day location call for today's date it will be at 2016/10/13. TABLE SCHEMA: create External table employee(EMPID Int,FirstName String,.....) partitioned by (year string,month string,day string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' LOCATION '/.../emp'; so everyday we need to run command which is working fine as ALTER TABLE

问题 I have a json file with the following format [ { "id": 2, "createdBy": 0, "status": 0, "utcTime": "Oct 14, 2014 4:49:47 PM", "placeName": "21/F, Cunningham Main Rd, Sampangi Rama NagarBengaluruKarnatakaIndia", "longitude": 77.5983817, "latitude": 12.9832418, "createdDate": "Sep 16, 2014 2:59:03 PM", "accuracy": 5, "loginType": 1, "mobileNo": "0000005567" }, { "id": 4, "createdBy": 0, "status": 0, "utcTime": "Oct 14, 2014 4:52:48 PM", "placeName": "21/F, Cunningham Main Rd, Sampangi Rama