How to split a whole tf.data.Dataset into images and labels?

问题

I'm trying to split a separate distinct dataset into images and its labels to check my model against it.

I got the idea of just equating images, labels to the created dataset from here, but I get this error:

2020-12-04 15:27:39.801157: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at whole_file_read_ops.cc:116 : Unknown: NewRandomAccessFile failed to Create/Open: D:\test_dataset\: Access is denied.
; Input/output error
Traceback (most recent call last):
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\eager\context.py", line 2102, in execution_mode
    yield
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 758, in _next_internal
    output_shapes=self._flat_output_shapes)
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2610, in iterator_get_next
    _ops.raise_from_not_ok_status(e, name)
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\framework\ops.py", line 6843, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.UnknownError: NewRandomAccessFile failed to Create/Open: D:\test_dataset : Access is denied.
; Input/output error
     [[{{node ReadFile}}]] [Op:IteratorGetNext]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/projects/test/main.py", line 51, in <module>
    test_model(dir_model_disease, dir_dataset_disease)
  File "D:/projects/test/main.py", line 41, in test_model
    images, labels = dataset
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 736, in __next__
    return self.next()
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 772, in next
    return self._next_internal()
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 764, in _next_internal
    return structure.from_compatible_tensor_list(self._element_spec, ret)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\eager\context.py", line 2105, in execution_mode
    executor_new.wait()
  File "D:\projects\venv\test\lib\site-packages\tensorflow\python\eager\executor.py", line 67, in wait
    pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
tensorflow.python.framework.errors_impl.UnknownError: NewRandomAccessFile failed to Create/Open: D:\test_dataset : Access is denied.
; Input/output error
     [[{{node ReadFile}}]]

My code:

import glob2
import os

import tensorflow as tf

def load_image(file_path):
    img = tf.io.read_file(file_path)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.convert_image_dtype(img, tf.float32)
    img = tf.image.resize(img, size=(224, 224))
    label = tf.strings.split(file_path, os.sep)[0]
    label = tf.cast(tf.equal(label, 'class_a'), tf.int32)
    return img, label


def load_model(dir_model):
    model = tf.keras.models.load_model(dir_model)
    model.summary()
    return model


def create_dataset(dir_dataset):
    files = glob2.glob(dir_dataset)
    dataset = tf.data.Dataset.from_tensor_slices(files).map(load_image).batch(4)
    return dataset


def test_model(dir_model, dir_dataset):
    model = load_model(dir_model)
    dataset = create_dataset(dir_dataset)
    images, labels = dataset
    print(labels)
    results = model.evaluate(images, labels, batch_size=8)
    print("test loss: {} test accuracy: {}".format(results[0], results[1]))
    for image in images:
        prediction = model.predict(image)
        print(prediction)


if __name__ == '__main__':
    test_model(dir_model, dir_dataset)

This method of creating a dataset this way (where subdirectory name is the label of the images inside) is from my previous question's answer.

My end goal would be just to use predict on and evaluate my model using this dataset. How would I go around this?

Some things I've tried:

• use placeholder variables to fit how other examples get the x and y of the dataset:

(__, images), (__, labels) = dataset

Same error.

• use dataset as is in evaluate function:

results = models.evaluate(dataset, batch_size=8)

Error:

ValueError: Shapes (None, 1) and (None, 29) are incompatible

• print dataset

<BatchDataset shapes: ((None, 224, 224, 3), (None,)), types: (tf.float32, tf.int32)>

来源：https://stackoverflow.com/questions/65139622/how-to-split-a-whole-tf-data-dataset-into-images-and-labels