Shutdown dask workers from client or scheduler

Question

In the API, there is a way to restart all workers and to shutdown the client completely, but I see no way to stop all workers while keeping the client unchanged. Is there a way to do this that I cannot find or is it a feature that doesn't exist ?

Answer 1

This seems like a feature that does not exist, but is nevertheless doable using the current code. You can use run_on_scheduler to interact with the methods of the scheduler itself.

workers = list(c.scheduler_info()['workers'])
c.run_on_scheduler(lambda dask_scheduler=None: 
    dask_scheduler.retire_workers(workers, close_workers=True))

where c is a Client, and we call retire_workers to gracefully ask each worker to exit.

There are probably other ways to achieve this. Note that the scheduler remains running in this case - it was not clear from the question if that is desired or not.