Deedle moving window stats calcuation with a dynamic condition and boundary.atending

问题

I am using a dynamic moving window to calculation simple stats on a series ordered on the date key. I want to be able to set the boundary at the end of the window. for example a timeseries with monthly moving average, the monthly is decided by a

(fun d1 d2 -> d1.addMonths(1) <= d2)

however the deedle series function

windowWhileInto cond f series

always uses the begin as the boundary. Therefore, it always creates produce a n datapoints series from the first data instance for the next n data points (n is decided by the fun above). i would like to have a n datapoints series from the nth data and look backwards into the past.

I also tried to use Series.Rev first to reverse the series but deedle think that series although in a reversed order is no longer ordered.

Is what i am looking for possible?

回答1:

If you look at the list of aggregation functions in the docs, you'll find a function aggregate that is a generalization of all the windowing & chunking functions and also takes a key selector.

This means that you can do something like this:

ts |> Series.aggregateInto
        (WindowWhile(fun d1 d2 -> d1.AddMonths(1) >= d2))  // Aggregation to perform
        (fun seg -> seg.Data.LastKey())                    // Key selector (use last)
        (fun ds -> OptionalValue(ds.Data))                 // Value selector

The function takes 3 parameters including key selector and a function that gets "data segment" (which has the window together with a flag whether it is complete or incomplete - e.g. at the end of windowing).

Sadly, this does not quite work here, because it will create a series with duplicate keys (and those are not supported by Deedle). The windows at the end of the chunk will all end with the same date and so you'll get duplicate keys (it actually runs, but you cannot do much with the series).

An ugly workaround is to remember the last chunk's end and return missing values once the end starts repeating:

let lastKey = ref None
let r = 
  ts |> Series.aggregateInto
      (WindowWhile(fun d1 d2 -> d1.AddMonths(1) >= d2)) (fun seg -> seg.Data.LastKey())
      (fun ds -> 
         match lastKey.Value, ds.Data.LastKey() with 
         | Some lk, clk when lk = clk -> OptionalValue.Missing
         | _, clk -> lastKey := Some clk; OptionalValue(ds.Data))
     |> Series.dropMissing

EDIT: I logged a GitHub issue for this.

来源：https://stackoverflow.com/questions/25103662/deedle-moving-window-stats-calcuation-with-a-dynamic-condition-and-boundary-aten

标签

window

deedle