Transfer data from vertica to Redshift using Apache Nifi

Question

I want to transfer data from vertica to redshift using apache nifi. which are the processors and configuration I need to set?

Answer 1

If Vertica and Redshift have "well-behaved" JDBC drivers, you can set up a DBCPConnectionPool for each, then a SQL processor such as ExecuteSQL, QueryDatabaseTable, or GenerateTableFetch (the latter of which generates SQL for use in ExecuteSQL). These will get your records into Avro format, then (prior to NiFi 1.2.0) you can use ConvertAvroToJSON -> ConvertJSONToSQL -> PutSQL to get your records inserted into Redshift.

In NiFi 1.2.0, you can use set up an AvroReader for use in PutDatabaseRecord. Then you will only need the SQL processor to get the records out of Vertica, directly to PutDatabaseRecord to put them into Redshift.