How can I retrieve data from a QTableWidget to Dataframe?

Question

I have a QTableWidget in editable mode in which user puts in integer input , how can I generate a list of data entered in this table so as to perform operations on it , here is my manual code for that:

def dataframe_generation_from_table(self,table):
    number_of_rows = table.rowCount()
    number_of_columns = table.columnCount()

    tmp_df = pd.DataFrame({ 'Date' : [] , str(self.final_lvl_of_analysis) :[],  'Value': []}) 

    for i in range(0,number_of_rows):
        for j in range(0,number_of_columns):
            tmp_item = table.item(i,j)
            tmp_df2 = pd.DataFrame( { 'Date' : [pd.to_datetime(table.horizontalHeaderItem(j).data())] , str(self.final_lvl_of_analysis) :[ str(table.verticalHeaderItem(i).data())], 'Value': [float(tmp_item.data(0))]})
            print tmp_df2
            tmp_df.update(tmp_df2, join = 'left', overwrite = False)

    return tmp_df

Also , I am using the following code for QTableWidget generation:

    self.pd_table = QtGui.QTableWidget(self.groupBox_19)
    self.pd_table.setObjectName(_fromUtf8("pd_table"))
    self.pd_table.setColumnCount(0)
    self.pd_table.setRowCount(0)

My specs are : pandas 0.18.1 , PyQt 4 and Python 2.7

Answer 1

I think you're overcomplicating it a little with the updates/joins. The simplest approach is to create the full-size DataFrame first (filled with NaN) and then assign the data to this:

def dataframe_generation_from_table(self,table):
    number_of_rows = table.rowCount()
    number_of_columns = table.columnCount()

    tmp_df = pd.DataFrame( 
                columns=['Date', str(self.final_lvl_of_analysis), 'Value'], # Fill columnets
                index=range(number_of_rows) # Fill rows
                ) 

    for i in range(number_of_rows):
        for j in range(number_of_columns):
            tmp_df.ix[i, j] = table.item(i, j).data()

    return tmp_df

The above code assigns data to it's location by the numerical index, so position 1,1 in the QtTableWidget will end up at 1,1 in the DataFrame. This way you don't need to worry about the column headers when moving data. If you want to change the column names you can do that when creating the DataFrame, changing the values passed into the columns= parameter.

If you want to change a column to DateTime format, you should be able to do this in a single operation after the loop with:

tmp_df['Date'] = pd.to_datetime( tmp_df['Date'] )

Answer 2

The change from .data() to .text() eliminated the ValueError.

def saveFile(self):
    df = pd.DataFrame()
    savePath = QtGui.QFileDialog.getSaveFileName(None, "Blood Hound", 
        "Testing.csv", "CSV files (*.csv)")        
    rows = self.tableWidget.rowCount()
    columns = self.tableWidget.columnCount()        

    for i in range(rows):            
        for j in range(columns):                
            df.loc[i, j] = str(self.tableWidget.item(i, j).text())              
    df.to_csv((savePath), header = None, index = 0)

Answer 3

# creates a new df from qtables dimensions,
# copies qtable (data & headers) to the df and returns the df
@staticmethod
def write_qtable_to_df(table):
    col_count = table.columnCount()
    row_count = table.rowCount()
    headers = [str(table.horizontalHeaderItem(i).text()) for i in range(col_count)]

    # df indexing is slow, so use lists
    df_list = []
    for row in range(row_count):
        df_list2 = []
        for col in range(col_count):
            table_item = table.item(row,col)
            df_list2.append('' if table_item is None else str(table_item.text()))
        df_list.append(df_list2)

    df = pandas.DataFrame(df_list, columns=headers)

    return df