Get a string in text CSV file in C++

问题

I have a large CSV (~75 MB) of this kind:

1,3,4.1,5.4
-2,-4,-0.1,-11.3
...

And I store my data with this code (C style):

#include <iostream>
#include <cstdio>
#include <vector>

int main()
{
    int s;
    int x;
    float y;
    double z;

    std::vector<int> t;
    std::vector<int> u;
    std::vector<float> v;
    std::vector<double> w;

    if (std::FILE *f = std::fopen("data.csv", "r")) {
        while (std::fscanf(f, "%d,%d,%f,%lf", &s, &x, &y, &z) == 4) {
            t.push_back(s);
            u.push_back(x);
            v.push_back(y);
            w.push_back(z);
        }
    std::fclose(f);
    }

    return 0;
}

And it tooks me in this large CSV (~75MB):

real        0m3.195s
user        0m3.032s
sys         0m0.148s

It's so fast in C style!

Another way it's with this code (C++ style):

#include <iostream>
#include <fstream>
#include <vector>

int main()
{
    char c; // to eat the commas. Not eat spaces :-(
    int s;
    int x;
    float y;
    double z;

    std::vector<int> t;
    std::vector<int> u;
    std::vector<float> v;
    std::vector<double> w;

    std::ifstream file("data.csv");
    while (file >> s >> c >> x >> c >> y >> c >> z) {
        t.push_back(s);
        u.push_back(x);
        v.push_back(y);
        w.push_back(z);
    }

    return 0;
}

And it tooks me in this large CSV (~75MB):

real        0m4.766s
user        0m4.660s
sys         0m0.088s

C style is more fast!

I'd like to read a string in the first column (or in the second) and put into a vector of std::string.

I try many possibilites (char *, iostream, etc.) But I can't done in a fast and elegant way.

Examples of types of large CSV files (is there one easier to read than another?):

a.csv:

hi,3,4.1,5.4
hello,-4,-0.1,-11.3
...

b.csv:

hi 3 4.1 5.4
hello -4 -0.1 -11.3
...

c.csv:

"hi",3,4.1,5.4
"hello",-4,-0.1,-11.3
...

d.csv:

"hi" 3 4.1 5.4
"hello" -4 -0.1 -11.3
...

Thank you very much for the help! :)

回答1:

So you're looking for a more efficient way of doing this? Well, one thing you could do is consider whether or not you really need vectors. Depending on your usage, you may be better with some kind of linked list structure.

来源：https://stackoverflow.com/questions/21785538/get-a-string-in-text-csv-file-in-c

标签

c++

performance

csv

serialization