How can I test a .tflite model to prove that it behaves as the original model using the same Test Data?

Question

I have generated a .tflite model based on a trained model, I would like to test that the tfilte model gives the same results as the original model.

Giving both the same test data and obtaining the same result.

Answer 1

You may use TensorFlow Lite Python interpreter to test your tflite model.

It allows you to feed input data in python shell and read the output directly like you are just using a normal tensorflow model.

I have answered this question here.

And you can read this TensorFlow lite official guide for detailed information.

I also found a very good visualization tool which can load .tflite file directly to let you inspect your model architecture and model weights.

Answer 2

There is a tflite_diff_example_test in the TensorFlow code base. It generates random data and feed the same data into TensorFlow & TensorFlow lite, then compare if the difference is within a small threshold.

You can to checkout TensorFlow code from Github, and run it with bazel:

bazel run //tensorflow/contrib/lite/testing:tflite_diff_example_test

then you'll see what arguments you need to pass.

Answer 3

In addition to the answer given by @miaout17, to debug / understand your tflite model (which is the spirit of the question), you can

• use --dump_graphviz to visualize the graph as explained here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/cmdline_examples.md#using---dump_graphviz
• use flatc to generate a python api and then parse the model via that api https://google.github.io/flatbuffers/flatbuffers_guide_use_python.html
• generate json from the tflite file using flatc and print it out
• Use the tflite visualizer: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/tools/visualize.py (currently has a bug)