read input files, fastest way possible?

Question

I have numerous text files of data in the form of float numbers. I\'m looking for the fastest way to read them in C++. I can change the file to binary if that\'s the fastest

Answer 1

Another attention to compile mode. I have tried parsing a file with 1M lines. Debug mode consumed 50secs to parse data and append to my container. Release mode consumed at least ten times faster, about 4secs. The code below is to read the whole file before using istringstream to parse the data as 2D points (,).

vector  in_data;
string raw_data;

ifstream ifs;
ifs.open(_file_in.c_str(), ios::binary);
ifs.seekg(0, ios::end);
long length = ifs.tellg();
ifs.seekg(0, ios::beg);
char * buffer;
buffer = new char[length];
ifs.read(buffer, length);
raw_data = buffer;
ifs.close();
delete[]buffer;
cout << "Size: " << raw_data.length()/1024/1024.0 << "Mb" << endl;
istringstream _sstr(raw_data);
string _line;

while (getline(_sstr, _line)){
    istringstream _ss(_line);
    vector  record;
    //maybe using boost/Tokenizer is a good idea ...
    while (_ss)
    {
        string s;
        if (!getline(_ss, s, ',')) break;
        record.push_back(atof(s.c_str()));
    }
    in_data.push_back(record[0]);
}