How do I write a range pipeline that uses temporary containers?

Question

I have a third-party function with this signature:

std::vector f(T t);

I also have an existing potentially infinite range (of the

Answer 1

The problem here of course is the whole idea of a view - a non-storing layered lazy evaluator. To keep up with this contract, views have to pass around references to range elements, and in general they can handle both rvalue and lvalue references.

Unfortunately in this specific case view::transform can only provide an rvalue reference as your function f(T t) returns a container by value, and view::join expects an lvalue as it tries to bind views (view::all) to inner containers.

Possible solutions will all introduce some kind of temporary storage somewhere into the pipeline. Here are the options I came up with:

• Create a version of view::all that can internally store a container passed by an rvalue reference (As suggested by Barry). From my point of view, this violates the "non-storing view" conception and also requires some painful template coding so I would suggest against this option.

• Use a temporary container for the whole intermediate state after the view::transform step. Can be done either by hand:

  auto rng1 = src | view::transform(f)
  vector> temp = rng1;
  auto rng = temp | view::join;
  

  Or using action::join. This would result in "premature evaluation", will not work with infinite src, will waste some memory, and overall has a completely different semantics from your original intention, so that is hardly a solution at all, but at least it complies with view class contracts.

• Wrap a temporary storage around the function you pass into view::transform. The simpliest example is

  const std::vector& f_store(const T& t)
  {
    static std::vector temp;
    temp = f(t);
    return temp;
  }
  

  and then pass f_store to the view::transform. As f_store returns an lvalue reference, view::join will not complain now.

  This of course is somewhat of a hack and will only work if you then streamline the whole range into some sink, like an output container. I believe it will withstand some straightforward transformations, like view::replace or more view::transforms, but anything more complex can try to access this temp storage in non-straightforward order.

  In that case other types of storage can be used, e.g. std::map will fix that problem and will still allow infinite src and lazy evaluation at the expense of some memory:

  const std::vector& fc(const T& t)
  {
      static std::map> smap;
      smap[t] = f(t);
      return smap[t];
  }
  

  If your f function is stateless, this std::map can also be used to potentially save some calls. This approach can possibly be improved further if there is a way to guarantee that an element will no longer be required and remove it from the std::map to conserve memory. This however depends on further steps of the pipeline and the evaluation.

As these 3 solutions pretty much cover all the places to introduce temporary storage between view::transform and view::join, I think these are all the options you have. I would suggest going with #3 as it will allow you to keep the overall semantics intact and it is quite simple to implement.