Nested STL vector using way too much memory

Question

I have an STL vector My_Partition_Vector of Partition objects, defined as

struct Partition // the event log data structure
{
    in         


        

Answer 1

...This is a bit of a side conversation, but boost::multi_array was suggested as an alternative to the OP's use of nested vectors. My finding was that multi_array was using a similar amount of memory when applied to the OP's operating parameters.

I derived this code from the example at Boost.MultiArray. On my machine, this showed multi_array using about 10x more memory than ideally required assuming that the 16 bytes are arranged in a simple rectangular geometry.

To evaluate the memory usage, I checked the system monitor while the program was running and I compiled with

( export CXXFLAGS="-Wall -DNDEBUG -O3" ; make main && ./main )

Here's the code...

   #include 
   #include 
   #include "boost/multi_array.hpp"
   #include 
   #include 

   #define USE_CUSTOM_ARRAY 0 // compare memory usage of my custom array vs. boost::multi_array

   using std::cerr;
   using std::vector;

  #ifdef USE_CUSTOM_ARRAY
   template< typename T, int YSIZE, int XSIZE >
   class array_2D
      {
      std::tr1::array data;
   public:
      T & operator () ( int y, int x ) { return data[y*XSIZE+x]; } // preferred accessor (avoid pointers)
      T * operator [] ( int index ) { return &data[index*XSIZE]; } // alternative accessor (mimics boost::multi_array syntax)
      };
  #endif

int main ()
   {

   int COUNT = 1024*1024;

  #if USE_CUSTOM_ARRAY
   vector< array_2D > A( COUNT );
   typedef int index;
  #else
   typedef boost::multi_array array_type;
   typedef array_type::index index;
   vector A( COUNT, array_type(boost::extents[4][4]) );
  #endif

  // Assign values to the elements
  int values = 0;
  for ( int n=0; n