Retrain image detection with MobileNet

这一生的挚爱 提交于 2019-11-27 06:16:25

问题


Several ways of retraining MobileNet for use with Tensorflow.js have failed for me. Is there any way to use a retrained model with Tensorflow.js?

Both using the modern, hub-based tutorial, as well as using retrain.py seem to fail.

  • Convert output of retrain.py to tensorflow.js
  • Error converting keras model to tfjs: duplicate weight name Variable

as well as some other open questions

  • Retrain an Image Classifier in tensorflow js
  • Loading of mobilenet v2 works, but pretrained mobilenet v2 fails
  • Can't convert TensorFlow saved model to tfjs_layers_model webmodel

The top two other questions show the code that failed in both instances, both are unsolved.

The aim is to load the mobilenet, retrain using custom data, and use it in Tensorflow.js. Following both tutorials seem to fail. Could this be done inside node.js? Is there another way? Where did I make mistakes (or is the software unable to use retrained models)? How can this work?

EDITs: latest github issue and one more question


回答1:


I encountered the same problem and it seems that we use the wrong method. There are loadGraphModel for TF converted models and loadLayersModel for Keras ones my comment about the issue




回答2:


The retrain.py python script does not generate a saved model, it actually generates a frozen graph model. That is why you cannot convert it using the tfjs 1.x converter. You need to use tfjs 0.8.5 pip to convert. Also, the output node name is different from the mobilenet model graph, it is 'final_result' for retrained graph.

To convert it you need to use the tensorflowjs 0.8.5 pip:

  • use virtualenv to create an empty env.
  • pip install tensorflowjs==0.8.5
  • run the converter
tensorflowjs_converter \
  --input_format=tf_frozen_model \
  --output_node_names='final_result' \
  --output_json=true /tmp/output_graph.pb \ /tmp/web_model

This should give you something like the following:

ls /tmp/web_model/
group1-shard10of21  group1-shard14of21  group1-shard18of21  group1-shard21of21  group1-shard5of21  group1-shard9of21
group1-shard11of21  group1-shard15of21  group1-shard19of21  group1-shard2of21   group1-shard6of21  model.json
group1-shard12of21  group1-shard16of21  group1-shard1of21   group1-shard3of21   group1-shard7of21
group1-shard13of21  group1-shard17of21  group1-shard20of21  group1-shard4of21   group1-shard8of21



回答3:


To use the latest TFjs:

python retrain.py --tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/2 \
    --image_dir /tmp/flower_photos --saved_model_dir /tmp/saved_retrained_model
tensorflowjs_converter --input_format=tf_saved_model \
    --output_format=tfjs_graph_model \
    --saved_model_tags=serve \
    /tmp/saved_retrained_model/ /tmp/converted_model/

creates a model.json file. Command described in https://github.com/tensorflow/tfjs-converter#step-1-converting-a-savedmodel-keras-h5-tfkeras-savedmodel-or-tensorflow-hub-module-to-a-web-friendly-format.

Yet, loading the model with tf.loadLayersModel("file:///tmp/web_model/model.json") failed with

'className' and 'config' must set.




回答4:


Maybe somebody can modify retain.py to support mobileV2 use my way. The original retrain.py link. This link is Google's GitHub code, not my link.

I changed retrain.py, the below is my git diff:

diff --git a/scripts/retrain.py b/scripts/retrain.py
index 5fa9b0f..02a4f9a 100644
--- a/scripts/retrain.py
+++ b/scripts/retrain.py
@@ -1,3 +1,5 @@
+# -*- coding: utf-8 -*-
+
 # Copyright 2015 The TensorFlow Authors. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -112,6 +114,13 @@ from tensorflow.python.framework import graph_util
 from tensorflow.python.framework import tensor_shape
 from tensorflow.python.platform import gfile
 from tensorflow.python.util import compat
+from tensorflow import saved_model as sm
+from tensorflow.python.saved_model import builder as saved_model_builder
+from tensorflow.python.saved_model import signature_constants
+from tensorflow.python.saved_model import signature_def_utils
+from tensorflow.python.saved_model import tag_constants
+from tensorflow.python.saved_model import utils as saved_model_utils
+

 FLAGS = None

@@ -319,6 +328,7 @@ def maybe_download_and_extract(data_url):
   Args:
     data_url: Web location of the tar file containing the pretrained model.
   """
+  print(FLAGS.model_dir)
   dest_directory = FLAGS.model_dir
   if not os.path.exists(dest_directory):
     os.makedirs(dest_directory)
@@ -827,6 +837,7 @@ def save_graph_to_file(sess, graph, graph_file_name):
       sess, graph.as_graph_def(), [FLAGS.final_tensor_name])
   with gfile.FastGFile(graph_file_name, 'wb') as f:
     f.write(output_graph_def.SerializeToString())
+
   return


@@ -971,6 +982,7 @@ def main(_):

   # Prepare necessary directories  that can be used during training
   prepare_file_system()
+  sigs = {}

   # Gather information about the model architecture we'll be using.
   model_info = create_model_info(FLAGS.architecture)
@@ -1002,6 +1014,9 @@ def main(_):
       FLAGS.random_brightness)

   with tf.Session(graph=graph) as sess:
+    serialized_tf_example = tf.placeholder(tf.string, name='tf_example')
+    feature_configs = {'x': tf.FixedLenFeature(shape=[784], dtype=tf.float32),}
+    tf_example = tf.parse_example(serialized_tf_example, feature_configs)
     # Set up the image decoding sub-graph.
     jpeg_data_tensor, decoded_image_tensor = add_jpeg_decoding(
         model_info['input_width'], model_info['input_height'],
@@ -1133,6 +1148,73 @@ def main(_):
                           (test_filename,
                            list(image_lists.keys())[predictions[i]]))

+    """
+    # analyze SignatureDef protobuf
+    SignatureDef_d = graph.signature_def
+    SignatureDef = SignatureDef_d[sm.signature_constants.CLASSIFY_INPUTS]
+
+    # three TensorInfo protobuf
+    X_TensorInfo = SignatureDef.inputs['input_1']
+    scale_TensorInfo = SignatureDef.inputs['input_2']
+    y_TensorInfo = SignatureDef.outputs['output']
+
+    # Tensor details
+    # .get_tensor_from_tensor_info() to get default graph 
+    X = sm.utils.get_tensor_from_tensor_info(X_TensorInfo, sess.graph)
+    scale = sm.utils.get_tensor_from_tensor_info(scale_TensorInfo, sess.graph)
+    y = sm.utils.get_tensor_from_tensor_info(y_TensorInfo, sess.graph)
+    """
+
+    """
+    output_graph_def = graph_util.convert_variables_to_constants(
+      sess, graph.as_graph_def(), [FLAGS.final_tensor_name])
+
+    X_TensorInfo = sm.utils.build_tensor_info(bottleneck_input)
+    scale_TensorInfo = sm.utils.build_tensor_info(ground_truth_input)
+    y_TensorInfo = sm.utils.build_tensor_info(output_graph_def)
+
+    # build SignatureDef protobuf
+    SignatureDef = sm.signature_def_utils.build_signature_def(
+                                inputs={'input_1': X_TensorInfo, 'input_2': scale_TensorInfo},
+                                outputs={'output': y_TensorInfo},
+                                method_name='what'
+    )
+    """
+
+    #graph = tf.get_default_graph()
+    tensors_per_node = [node.values() for node in graph.get_operations()]
+    tensor_names = [tensor.name for tensors in tensors_per_node for tensor in tensors]
+    print(tensor_names)
+
+    export_dir = './tf_files/savemode'
+    builder = saved_model_builder.SavedModelBuilder(export_dir)
+
+    # name="" is important to ensure we don't get spurious prefixing
+    graph_def = tf.GraphDef()
+    tf.import_graph_def(graph_def, name="")
+    g = tf.get_default_graph()
+    inp1 = g.get_tensor_by_name("input:0")
+    inp2 = g.get_tensor_by_name("input_1/BottleneckInputPlaceholder:0")
+    inp3 = g.get_tensor_by_name("input_1/GroundTruthInput:0")
+    out = g.get_tensor_by_name("accuracy_1:0")
+
+    sigs[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY] = \
+        tf.saved_model.signature_def_utils.predict_signature_def(
+            {'input_1': inp1, 'input_2': inp3}, {"output": out})
+
+    builder.add_meta_graph_and_variables(sess,
+                                         tags=[tag_constants.SERVING],
+                                         signature_def_map=sigs)
+
+    """
+    builder.add_meta_graph_and_variables(
+            sess=sess,
+            tags=[tag_constants.SERVING],
+            signature_def_map={sm.signature_constants.CLASSIFY_INPUTS: SignatureDef})
+    """
+
+    builder.save()
+
     # Write out the trained graph and labels with the weights stored as
     # constants.
     save_graph_to_file(sess, graph, FLAGS.output_graph)

Using my diff, I can generate Tensorflow Served model. And then I use the command to convert TensorFlow served model to Tfjs model.

tensorflowjs_converter \
    --input_format=tf_saved_model \
    --output_format=tfjs_graph_model \
    ./tf_files/savemode \
    ./tf_files/js_model

Still unsupported Ops for lasted Tensorflow JS version.

I just make a video here to explain why we cannot convert Tensorflow frozen model to Tensorflow JS model, tells how to find the input Tensor and Output Tensor. The running steps and result, finally, give unsupported Ops ScalarSummary and the reason.

Now that I cannot change the Mobilenet Model to Tensorflow JS model, so my workaround is using Python tensorflow and flask library on Server side, user upload the image to server and then return the result.



来源:https://stackoverflow.com/questions/55849309/retrain-image-detection-with-mobilenet

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!