How to set/get pandas.DataFrame to/from Redis?

Question

After setting a DataFrame to redis, then getting it back, redis returns a string and I can\'t figure out a way to convert this str to a DataFrame.

How can I do these

Answer 1

For caching a dataframe use this.

import pyarrow as pa

def cache_df(alias,df):

    pool = redis.ConnectionPool(host='host', port='port', db='db')
    cur = redis.Redis(connection_pool=pool)
    context = pa.default_serialization_context()
    df_compressed =  context.serialize(df).to_buffer().to_pybytes()

    res = cur.set(alias,df_compressed)
    if res == True:
        print('df cached')

For fetching the cached dataframe use this.

def get_cached_df(alias):

    pool = redis.ConnectionPool(host='host',port='port', db='db') 
    cur = redis.Redis(connection_pool=pool)
    context = pa.default_serialization_context()
    all_keys = [key.decode("utf-8") for key in cur.keys()]

    if alias in all_keys:   
        result = cur.get(alias)

        dataframe = pd.DataFrame.from_dict(context.deserialize(result))

        return dataframe

    return None