series

I have a Pandas series sf: email email1@email.com [1.0, 0.0, 0.0] email2@email.com [2.0, 0.0, 0.0] email3@email.com [1.0, 0.0, 0.0] email4@email.com [4.0, 0.0, 0.0] email5@email.com [1.0, 0.0, 3.0] email6@email.com [1.0, 5.0, 0.0] And I would like to transform it to the following DataFrame: index | email | list _____________________________________________ 0 | email1@email.com | [1.0, 0.0, 0.0] 1 | email2@email.com | [2.0, 0.0, 0.0] 2 | email3@email.com | [1.0, 0.0, 0.0] 3 | email4@email.com | [4.0, 0.0, 0.0] 4 | email5@email.com | [1.0, 0.0, 3.0] 5 | email6@email.com | [1.0, 5.0, 0.0] I found

In Pandas, when I select a label that only has one entry in the index I get back a Series, but when I select an entry that has more then one entry I get back a data frame. Why is that? Is there a way to ensure I always get back a data frame? In [1]: import pandas as pd In [2]: df = pd.DataFrame(data=range(5), index=[1, 2, 3, 3, 3]) In [3]: type(df.loc[3]) Out[3]: pandas.core.frame.DataFrame In [4]: type(df.loc[1]) Out[4]: pandas.core.series.Series Granted that the behavior is inconsistent, but I think it's easy to imagine cases where this is convenient. Anyway, to get a DataFrame every time,

Lets say I have a dataframe like this A B 0 a b 1 c d 2 e f 3 g h 0,1,2,3 are times, a, c, e, g is one time series and b, d, f, h is another time series. I need to be able to add two columns to the orignal dataframe which is got by computing the differences of consecutive rows for certain columns. So i need something like this A B dA 0 a b (a-c) 1 c d (c-e) 2 e f (e-g) 3 g h Nan I saw something called diff on the dataframe/series but that does it slightly differently as in first element will become Nan. Use shift . df['dA'] = df['A'] - df['A'].shift(-1) You could use diff and pass -1 as the

I have two Series s1 and s2 with the same (non-consecutive) indices. How do I combine s1 and s2 to being two columns in a DataFrame and keep one of the indices as a third column? I think concat is a nice way to do this. If they are present it uses the name attributes of the Series as the columns (otherwise it simply numbers them): In [1]: s1 = pd.Series([1, 2], index=['A', 'B'], name='s1') In [2]: s2 = pd.Series([3, 4], index=['A', 'B'], name='s2') In [3]: pd.concat([s1, s2], axis=1) Out[3]: s1 s2 A 1 3 B 2 4 In [4]: pd.concat([s1, s2], axis=1).reset_index() Out[4]: index s1 s2 0 A 1 3 1 B 2 4

In Python generally, membership of a hashable collection is best tested via set . We know this because the use of hashing gives us O(1) lookup complexity versus O(n) for list or np.ndarray . In Pandas, I often have to check for membership in very large collections. I presumed that the same would apply, i.e. checking each item of a series for membership in a set is more efficient than using list or np.ndarray . However, this doesn't seem to be the case: import numpy as np import pandas as pd np.random.seed(0) x_set = {i for i in range(100000)} x_arr = np.array(list(x_set)) x_list = list(x_set)