Python: Sliding windowed mean, ignoring missing data

Question

I am currently trying to process an experimental timeseries dataset, which has missing values. I would like to calculate the sliding windowed mean of this dataset along time

Answer 1

You can do that using the rolling function of Pandas:

import numpy as np
import pandas as pd

#Construct sample data
n = 50
n_miss = 20
win_size = 3
data = np.random.random(n)
data[np.random.randint(0, n-1, n_miss)] = None

windowed_mean = pd.Series(data).rolling(window=win_size, min_periods=1).mean()

print(pd.DataFrame({'Data': data, 'Windowed mean': windowed_mean}) )

Output:

        Data  Windowed mean
0   0.589376       0.589376
1   0.639173       0.614274
2   0.343534       0.524027
3   0.250329       0.411012
4   0.911952       0.501938
5        NaN       0.581141
6   0.224964       0.568458
7        NaN       0.224964
8   0.508419       0.366692
9   0.215418       0.361918
10       NaN       0.361918
11  0.638118       0.426768
12  0.587478       0.612798
13  0.097037       0.440878
14  0.688689       0.457735
15  0.858593       0.548107
16  0.408903       0.652062
17  0.448993       0.572163
18       NaN       0.428948
19  0.877453       0.663223
20       NaN       0.877453
21       NaN       0.877453
22  0.021798       0.021798
23  0.482054       0.251926
24  0.092387       0.198746
25  0.251766       0.275402
26  0.093854       0.146002
27       NaN       0.172810
28       NaN       0.093854
29       NaN            NaN
30  0.965669       0.965669
31  0.695999       0.830834
32       NaN       0.830834
33       NaN       0.695999
34       NaN            NaN
35  0.613727       0.613727
36  0.837533       0.725630
37       NaN       0.725630
38  0.782295       0.809914
39       NaN       0.782295
40  0.777429       0.779862
41  0.401355       0.589392
42  0.491709       0.556831
43  0.127813       0.340292
44  0.781625       0.467049
45  0.960466       0.623301
46  0.637618       0.793236
47  0.651264       0.749782
48  0.154911       0.481264
49  0.159145       0.321773