bigdata

I've tried to figure out what will be the best use cases that suit for Amazon dynamoDB. When I googled most of the blogs says DyanmoDb will be used only for a large amount of data (BigData). I'm having a background of relational DB. NoSQL DB is new for me.So when I've tried to relate this to normal relation DB knowledge. Most of the concepts related to DynamoDb is to create a schema-less table with partition keys/sort keys. And try to query them based on the keys.Also, there is no such concept of stored procedure which makes queries easier and simple. If we managing such huge Data's doing such

This question already has an answer here : Spark SQL security considerations (1 answer) Closed 2 years ago . Scenario: Say there is a table in Hive , and it is queried using the below SparkSql in Apache Spark , where table name is passed as an argument and concatenated to the query. In case of non-distributed system, I have basic understanding of SQL-Injection vulnerability and in the context of JDBC understand the usage of createStatement/preparedStatement in the those kind of scenario. But what about this scenario in the case of sparksql, is this code vulnerable? Any insights ? def main(args

I have a bunch of binary files compressed into *gz format. These are generated on a remote node and must be transferred to HDFS located one of the datacenter's server. I'm exploring the option of sending the files with Flume; I explore the option of doing this with a Spooling Directory configuration, but apparently this only works when the file's directory is located locally on the same HDFS node. Any suggestions how to tackle this problem? arghtype There is no out-of-box solution for such case. But you could try these workarounds: You could create your own source implementation for such

I have a table that contains 40 million data. I want to reduce the query time. is it possible to do it using views?. if yes, could you please explain how?. Yes, It is possible to reduce query time using view because it have clustered index assigned and they'll store temporary result that can speed up resulting queries. Well, the MySQL statement for creating a view is: CREATE ALGORITHM = MERGE VIEW my_view AS SELECT ... Where the algorithm is one of MERGE, TEMPTABLE, or UNDEFINED. However, as always the answer to how to increase performance is, "it depends". We really need more details on the

Closed . This question is opinion-based . It is not currently accepting answers. Want to improve this question? Update the question so it can be answered with facts and citations by editing this post . Closed 3 years ago . I have went through a lot of articles but I dont seem to get a perfectly clear answer on what exactly a BIG DATA is. In one page I saw "any data which is bigger for your usage, is big data i.e. 100 MB is considered big data for your mailbox but not your hard disc". Whereas another article said "big data to be usually more than 1 TB with different volume / variety / velocity