How to hook up Boost serialization & iostreams to serialize & gzip an object to string?

Question

I've been using the Boost serialization library, which is actually pretty nice, and lets me make simple wrappers to save my serializable objects to strings, like so:

namespace bar = boost::archive;
namespace bio = boost::iostreams;

template <class T> inline std::string saveString(const T & o) {
 std::ostringstream oss;
 bar::binary_oarchive oa(oss);
 oa << o;
 return oss.str();
}
template <class T> inline void saveFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 bar::binary_oarchive oa(ofs);
 oa << o;
}
template <class T> inline void loadFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 bar::binary_iarchive ia(ifs);
 ia >> o;
}

The thing is, I just found the need to compress my serialized data, too, so I'm looking at doing that with the filters in boost::iostreams. I figured out how to do it successfully with files:

template <class T> inline void saveGZFile(const T & o, const char* fname) {
 std::ofstream ofs(fname, std::ios::out|std::ios::binary|std::ios::trunc);
 bio::filtering_streambuf<bio::output> out;
 out.push(boost::iostreams::gzip_compressor());
 out.push(ofs);
 bar::binary_oarchive oa(out);
 oa << o;
}
template <class T> inline void loadGZFile(T & o, const char* fname) {
 std::ifstream ifs(fname, std::ios::in|std::ios::binary);
 assert(ifs.good()); // XXX catch if file not found
 bio::filtering_streambuf<bio::input> in;
 in.push(bio::gzip_decompressor());
 in.push(ifs);
 bar::binary_iarchive ia(in);
 ia >> o;
}

But can't figure out how to save correctly to a compressed string. The problem is that I'm not flushing the chain of filters, but I've tried popping and syncing and nothing seems to work. Here's my broken code:

template <class T> inline std::string saveGZString(const T & o) {
 std::ostringstream oss;
 bio::filtering_streambuf<bio::output> out;
 out.push(bio::gzip_compressor());
 out.push(oss);
 bar::binary_oarchive oa(out);
 oa << o;
 // XXX out.pop() twice?  out.strict_sync()??  oss.flush()??
 return oss.str();
}

As a result some data gets stuck in the stream buffer somewhere, and I always end up with a a few complete blocks (16K or 32K) of compressed data when I know it should be 43K or so given the (valid) output I get from using my saveGZFile method. Apparently hooking up the ofstream closes and flushes properly, but hooking up the ostringstream doesn't.

Any help? (This is my first stackoverflow question — help me, guys, you're my only hope!)

Answer 1

Returning to this question, I realized I must've fixed it sometime last year (as I'm using saveGZString right now). Digging to see how I fixed it, it was pretty silly/simple:

namespace bar = boost::archive;
namespace bio = boost::iostreams;

template <typename T> inline std::string saveGZString(const T & o) {
        std::ostringstream oss;
        { 
                bio::filtering_stream<bio::output> f;
                f.push(bio::gzip_compressor());
                f.push(oss);
                bar::binary_oarchive oa(f);
                oa << o;
        } // gzip_compressor flushes when f goes out of scope
        return oss.str();
}

Just let the whole chain go out of scope and it works! Neat! Here's my loader for completeness:

template <typename T> inline void loadGZString(T & o, const std::string& s) {
        std::istringstream iss(s);
        bio::filtering_stream<bio::input> f;
        f.push(bio::gzip_decompressor());
        f.push(iss);
        bar::binary_iarchive ia(f);
        ia >> o;
}

Answer 2

I haven't run the code myself, but my best guess is to use out.strict_sync() which applies flush() to every filter/device in the pipeline. I can't seem to tell, though, if gzip_compressor is flushable. If it is not, then strict_sync() will return false, and sync() would be more appropriate.