How to extract and route only specified columns from a CSV files and drop all other columns [duplicate]

问题

I want to extract few fields along with its value from a CSV file and drop/delete all other fields in the file. Please help. I think we can use RoutText processor.Please tell me how to write the regular expression for the routing only specified fields and dropping everything else. Thanks

Example- from he snapshot attached I only want to route 'Firstname,Lastname and Siblings' fields along wit hits value(each record/row). Delete the remaining columns like 'State, Age, Apt no,Country,Gender'.

Please tell me what is the correct processor for this and what configuration properties to use in order to achieve this. Thanks

Attaching snapshot for reference.

回答1:

You can use ConvertRecord for this. Provide the full schema to the CSVReader, and provide the schema with only the fields you want to the CSVRecordSetWriter. If you don't know the input schema (but you know it includes at least the fields you want to send along), you can have the reader Use String Fields From Header, that will create an input schema (using the header line) and assume all fields are strings. However the output schema would have the selected fields along with their types, and ConvertRecord will handle the "deletion" of the other fields, as well as any conversion from String to the desired data type for each of the selected fields.

回答2:

I think using regular expression is not the best solution: Here is how I should do:

First you need to explore the csv:
```
$handle = fopen("test.csv", "r")
```
Map through the data
```
$data = fgetcsv($handle, 1000, ",") 
```
Create New Header and Array from existed $data with wanted fields

Put new data into new csv.

$fp = fopen('file.csv', 'w');

foreach ($data as $fields) {
    fputcsv($fp, $fields);
}

fclose($fp);

来源：https://stackoverflow.com/questions/52340071/how-to-extract-and-route-only-specified-columns-from-a-csv-files-and-drop-all-ot

标签

regex

apache

etl

apache-nifi