Federated learning : convert my own image dataset into tff simulation Clientdata

Question

here is the code of my federated learning test

from __future__ import absolute_import, division, print_function
import os
import collections
import warnings
         


        

Answer 1

Thanks for your interest in TFF!

Generally, TFF is designed to ingest tf.data.Dataset objects, so the example above needs a little extra preprocessing.

Good news is, there is an existing tutorial showing an example of doing this. In the above, something like the following should work:

ds = tf.data.Dataset.from_generator(
    img_gen.flow_from_directory, args=[<your_directory>], 
    output_types=<your_types>, 
    output_shapes=<your_shapes>
)

Generally, one can think about the ClientData object as being a fancy dict mapping client ids to tf.data.Datasets. ClientData itself is an abstract class, and so can't be directly instantiated, and classmethods are provided to construct real instantiations of ClientData. One such classmethod that should work here would be tff.simulation.ClientData.from_clients_and_fn. Here, if you pass a list of client_ids and a function which returns the appropriate dataset when given a client id, you will have your hands on a fully functional ClientData.

I think here, an approach for defining the function you may use is to construct a Python dict which maps client IDs to tf.data.Dataset objects--you could then define a function which takes a client id, looks up the dataset in the dict, and returns the dataset.

Hopefully this helps!