chained-assignment

How to find the line that is generating a Pandas SettingWithCopyWarning?

由 风格不统一 提交于 2020-05-24 21:07:27
问题 I have a large block of code that is, at some point somewhere, generating a setting with copy warning in pandas (this problem). I know how to fix the problem, but I can't find what line number it is! Is there a way to back out the line number (apart from brute force methods like debug-stepping or putting in multiple prints)? The only output I get is the below, which doesn't go up the stack to my code: C:\Anaconda3\lib\site-packages\pandas\core\frame.py:2302: SettingWithCopyWarning: A value is

Pandas: SettingWithCopyWarning, trying to understand how to write the code better, not just whether to ignore the warning

由 喜欢而已 提交于 2019-12-23 23:06:59
问题 I am trying to change all date values in a spreadsheet's Date column where the year is earlier than 1900, to today's date, so I have a slice. EDIT: previous lines of code: df=pd.read_excel(filename)#,usecols=['NAME','DATE','EMAIL'] #regex to remove weird characters df['DATE'] = df['DATE'].str.replace(r'[^a-zA-Z0-9\._/-]', '') df['DATE'] = pd.to_datetime(df['DATE']) sample row in dataframe: name, date, email [u'Public, Jane Q.\xa0' u'01/01/2016\xa0' u'jqpublic@email.com\xa0'] This line of code

Type hints and chained assignment and multiple assignments

由 喜欢而已 提交于 2019-12-22 04:13:10
问题 I guess these two questions are related, so I'll post them together: 1.- Is it possible to put type hint in chained assignments? These two attempts failed: >>> def foo(a:int): ... b: int = c:int = a File "<stdin>", line 2 b: int = c:int = a ^ SyntaxError: invalid syntax >>> def foo(a:int): ... b = c:int = a File "<stdin>", line 2 b = c:int = a ^ SyntaxError: invalid syntax 2.- Is it possible to put type hint in multiple assignments? These were my attempts: >>> from typing import Tuple >>> def

Python: Pandas Dataframe how to multiply entire column with a scalar

由 前提是你 提交于 2019-12-17 21:54:43
问题 How do I multiply each element of a given column of my dataframe with a scalar? (I have tried looking on SO, but cannot seem to find the right solution) Doing something like: df['quantity'] *= -1 # trying to multiply each row's quantity column with -1 gives me a warning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead Note: If possible, I do not want to be iterating over the dataframe and do something like this...as I

Extracting specific selected columns to new DataFrame as a copy

由 ▼魔方 西西 提交于 2019-12-17 17:24:08
问题 I have a pandas DataFrame with 4 columns and I want to create a new DataFrame that only has three of the columns. This question is similar to: Extracting specific columns from a data frame but for pandas not R. The following code does not work, raises an error, and is certainly not the pandasnic way to do it. import pandas as pd old = pd.DataFrame({'A' : [4,5], 'B' : [10,20], 'C' : [100,50], 'D' : [-30,-50]}) new = pd.DataFrame(zip(old.A, old.C, old.D)) # raises TypeError: data argument can't

Action with pandas SettingWithCopyWarning

由 為{幸葍}努か 提交于 2019-12-17 06:14:53
问题 I try to delete some column and convert some value in column with df2.drop(df2.columns[[0, 1, 3]], axis=1, inplace=True) df2['date'] = df2['date'].map(lambda x: str(x)[1:]) df2['date'] = df2['date'].str.replace(':', ' ', 1) df2['date'] = pd.to_datetime(df2['date']) and to all this string I get df2.drop(df2.columns[[0, 1, 3]], axis=1, inplace=True) C:/Users/����� �����������/Desktop/projects/youtube_log/filter.py:11: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from

python performance problems using loops with big tables

由 百般思念 提交于 2019-12-10 22:24:04
问题 I am using python and multiple libaries like pandas and scipy to prepare data so I can start deeper analysis. For the preparation purpose I am for instance creating new columns with the difference of two dates. My code is providing the expected results but is really slow so I cannot use it for a table with like 80K rows. The run time would take ca. 80 minutes for the table just for this simple operation. The problem is definitely related with my writing operation: tableContent[6]['p_test

Type hints and chained assignment and multiple assignments

由 ￡可爱￡侵袭症＋ 提交于 2019-12-05 06:52:16
I guess these two questions are related, so I'll post them together: 1.- Is it possible to put type hint in chained assignments? These two attempts failed: >>> def foo(a:int): ... b: int = c:int = a File "<stdin>", line 2 b: int = c:int = a ^ SyntaxError: invalid syntax >>> def foo(a:int): ... b = c:int = a File "<stdin>", line 2 b = c:int = a ^ SyntaxError: invalid syntax 2.- Is it possible to put type hint in multiple assignments? These were my attempts: >>> from typing import Tuple >>> def bar(a: Tuple[int]): ... b: int, c:int = a File "<stdin>", line 2 b: int, c:int = a ^ SyntaxError:

df.loc causes a SettingWithCopyWarning warning message

由 青春壹個敷衍的年華 提交于 2019-11-30 16:57:12
问题 The following line of my code causes a warning : import pandas as pd s = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) s.loc[-1] = [5,np.nan,np.nan,6] grouped = s.groupby(['A']) for key_m, group_m in grouped: group_m.loc[-1] = [10,np.nan,np.nan,10] C:\Anaconda3\lib\site-packages\ipykernel\__main__.py:10: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame According to the documentation this is the recommended way of doing, so

Python: Pandas Dataframe how to multiply entire column with a scalar

由 ♀尐吖头ヾ 提交于 2019-11-28 17:46:00
How do I multiply each element of a given column of my dataframe with a scalar? (I have tried looking on SO, but cannot seem to find the right solution) Doing something like: df['quantity'] *= -1 # trying to multiply each row's quantity column with -1 gives me a warning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead Note: If possible, I do not want to be iterating over the dataframe and do something like this...as I think any standard math operation on an entire column should be possible w/o having to write a loop:
订阅 chained-assignment