how to import a “.csv ” data-file into the Redis database

霸气de小男生 提交于 2020-01-02 07:08:53

问题


How do I import a .csv data-file into the Redis database? The .csv file has "id, time, latitude, longitude" columns in it. Can you suggest to me the best possible way to import the CSV file and be able to perform spatial queries?


回答1:


This is a very broad question, because we don't know what data structure you want to have. What queries do you expect, etc. In order to solve your question you need:

  1. Write down expected queries. Write down expected partitions. Is this file your complete dataset?

  2. Write down your data structure. It will heavily depend on answers from p1.

  3. Pick any (scripting) language you are most comfortable with. Load your file, process it in CSV library, map to your data structure from p2, push to Redis. You can do the latter with client library or with redis-cli.

The if for example, you want to lay your data in sorted sets where your id is zset's key, timestamp is score and lat,lon is the payload, you can do this:

$ cat data.csv
id1,1528961481,45.0,45.0
id1,1528961482,45.1,45.1
id2,1528961483,50.0,50.0
id2,1528961484,50.1,50.0

cat data.csv | awk -F "," '{print $1" "$2" "$3" "$4}' | xargs -n4 sh -c 'redis-cli -p 6370 zadd $1 $2 "$3,$4"' sh

127.0.0.1:6370> zrange id2 0 -1
1) "50.0,50.0"
2) "50.1,50.0"



来源:https://stackoverflow.com/questions/50840707/how-to-import-a-csv-data-file-into-the-redis-database

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!