How to see progress of Dask Compute task?

Question

I would like to see a progressbar on Jupyternotebook while i\'m running a compute task using Dask, I\'m counting all values of \"id\" column from a large csv file +4GB, so a

Answer 1

If you're using the single machine scheduler then do this:

from dask.diagnostics import ProgressBar
ProgressBar().register()

http://dask.pydata.org/en/latest/diagnostics-local.html

If you're using the distributed scheduler then do this:

from dask.distributed import progress

result = df.id.count.persist()
progress(result)

Or just use the dashboard

http://dask.pydata.org/en/latest/diagnostics-distributed.html