Load binary file using fstream

Question

I'm trying to load binary file using fstream in the following way:

#include <iostream>
#include <fstream>
#include <iterator>
#include <vector>

using namespace std;

int main()
{
    basic_fstream<uint32_t> file( "somefile.dat", ios::in|ios::binary );

    vector<uint32_t> buffer;
    buffer.assign( istream_iterator<uint32_t, uint32_t>( file ), istream_iterator<uint32_t, uint32_t>() );

    cout << buffer.size() << endl;

    return 0;
}

But it doesn't work. In Ubuntu it crashed with std::bad_cast exception. In MSVC++ 2008 it just prints 0.

I know that I could use file.read to load file, but I want to use iterator and operator>> to load parts of the file. Is that possible? Why the code above doesn't work?

Answer 1

1. istream_iterator wants basic_istream as argument.
2. It is impossible to overload operator>> inside basic_istream class.
3. Defining global operator>> will lead to compile time conflicts with class member operator>>.
4. You could specialize basic_istream for type uint32_t. But for specialization you should rewrite all fuctionons of basic_istream class. Instead you could define dummy class x and specialize basic_istream for it as in the following code:

using namespace std;

struct x {};
namespace std {
template<class traits>
class basic_istream<x, traits> : public basic_ifstream<uint32_t>
{
public:
    explicit basic_istream<x, traits>(const wchar_t* _Filename, 
        ios_base::openmode _Mode, 
        int _Prot = (int)ios_base::_Openprot) : basic_ifstream<uint32_t>( _Filename, _Mode, _Prot ) {}

    basic_istream<x, traits>& operator>>(uint32_t& data)
    {
        read(&data, 1);
        return *this;
    }
};
} // namespace std 

int main() 
{
    basic_istream<x> file( "somefile.dat", ios::in|ios::binary );
    vector<uint32_t> buffer;
    buffer.assign( istream_iterator<uint32_t, x>( file ), istream_iterator<uint32_t, x>() );
    cout << buffer.size() << endl;
    return 0;
}

Answer 2

You can reload operator>> to read integers properly. Of course all it will do is read() 4 bytes. But that's what all other operators>> are eventually doing anyway.

Here is example (no error checking, assuming endianess is same as current compiler uses, etc)

std::istream& operator>>(std::istream& in, uint32_t& data)
{
    in.read(&data, sizeof(data));
    return in;
}

Tailor for your own flavor of integers (might have to read one byte at a time and shift assign them, look at the file in hex editor if you don't know byte order), add error checking, and you should be able to use your existing code.

EDIT: ah, and yes, make sure this shadows provided stl operator that reads integer -- might have to derive your own class from the stream you are using and use that instead of std::istream& in, just so compiler knows who to check first.

Answer 3

The main question probably is what you mean by "binary file". The ios::binary only makes sure that the istream object doesn't replace platform-specific newlines by '\n'. Nothing else. Is that enough for you?

An istream_iterator basically is just a fancy way to invoke operator>>. If you have real binary data in your stream, that will fail. Do you have real binary data in your file? Or are the integers stored as strings?

If you need to read real binary integers, what you need is either istream.read() or using the stream buffer object directly.

Answer 4

A different way to do the same as Alexey Malistov's answer:

#include <fstream>
#include <iterator>
#include <vector>
#include <iostream>

struct rint // this class will allow us to read binary
{
  // ctors & assignment op allows implicit construction from uint
  rint () {}
  rint (unsigned int v) : val(v) {}
  rint (rint const& r) : val(r.val) {}
  rint& operator= (rint const& r) { this->val = r.val; return *this; }
  rint& operator= (unsigned int r) { this->val = r; return *this; }

  unsigned int val;

  // implicit conversion to uint from rint
  operator unsigned int& ()
  {
    return this->val;
  }
  operator unsigned int const& () const
  {
    return this->val;
  }
};

// reads a uints worth of chars into an rint
std::istream& operator>> (std::istream& is, rint& li)
{
  is.read(reinterpret_cast<char*>(&li.val), 4);
  return is;
}

// writes a uints worth of chars out of an rint
std::ostream& operator<< (std::ostream& os, rint const& li)
{
  os.write(reinterpret_cast<const char*>(&li.val), 4);
  return os;
}

int main (int argc, char *argv[])
{
  std::vector<int> V;

  // make sure the file is opened binary & the istream-iterator is
  // instantiated with rint; then use the usual copy semantics
  std::ifstream file(argv[1], std::ios::binary | std::ios::in);
  std::istream_iterator<rint> iter(file), end;
  std::copy(iter, end, std::back_inserter(V));

  for (int i = 0; i < V.size(); ++i)
    std::cout << std::hex << "0x" << V[i] << std::endl;

  // this will reverse the binary file at the uint level (on x86 with
  // g++ this is 32-bits at a time)
  std::ofstream of(argv[2], std::ios::binary | std::ios::out);
  std::ostream_iterator<rint> oter(of);
  std::copy(V.rbegin(), V.rend(), oter);

  return 0;
}