How do I convert from a dask dataframe to a list of futures?

问题

I have a dask dataframe like the following:

import dask.dataframe as dd
df = dd.read_csv('s3://...')

How do I get a list of futures from this dataframe?

回答1:

You can use the the .to_delayed method to convert from a dask dataframe to a list of dask.delayed objects

L = df.to_delayed()

You can then convert these delayed objects into dask futures using the client.compute method.

from dask.distributed import Client
client = Client()

futures = client.compute(L)

来源：https://stackoverflow.com/questions/48796323/how-do-i-convert-from-a-dask-dataframe-to-a-list-of-futures