问题

I was wondering if there is a way to know the list of inputs and outputs for a particular node in tflite? I know that I can get input/outputs details, but this does not allow me to reconstruct the computation process that happens inside an Interpreter. So what I do is:

interpreter = tf.lite.Interpreter(model_path=model_path)
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
interpreter.get_tensor_details()

The last 3 commands basically give me dictionaries which don't seem to have the necessary information.

So I was wondering if there is way to know where each nodes outputs goes? Surely Interpreter knows this somehow. Can we? Thanks.

回答1:

The mechanism of TF-Lite makes the whole process of inspecting the graph and getting the intermediate values of inner nodes a bit tricky. The get_tensor(...) method suggested by the other answer does not work.

How to visualize TF-Lite inference graph?

TensorFlow Lite models can be visualized using the visualize.py script in the TensorFlow Lite repository. You just need to:

Clone the TensorFlow repository

Run the visualize.py script with bazel:

bazel run //tensorflow/lite/tools:visualize \
     model.tflite \
     visualized_model.html

Does the nodes in my TF model have a equivalent one in TF-Lite?

NO! In fact, TF-Lite can modify your graph so that it become more optimal. Here are some words about it from the TF-Lite documentation:

A number of TensorFlow operations can be processed by TensorFlow Lite even though they have no direct equivalent. This is the case for operations that can be simply removed from the graph (tf.identity), replaced by tensors (tf.placeholder), or fused into more complex operations (tf.nn.bias_add). Even some supported operations may sometimes be removed through one of these processes.

Moreover, the TF-Lite API currently doesn't allow to get node correspondence; it's hard to interpret the inner format of TF-Lite. So, you can't get the intermediate outputs for any nodes you want, even without the one more issue below...

Can I get intermediate values of some TF-Lite nodes?

NO! Here, I will explain why get_tensor(...) wouldn't work in TF-Lite. Suppose in the inner representation, the graph contains of 3 tensors, together with some dense operations (nodes) in-between (you can think of tensor1 as input and tensor3 as output of your model). During inference of this particular graph, TF-Lite only needs 2 buffers, let's show how.

First, use tensor1 to compute tensor2 by applying dense operation. This only requires 2 buffers to store the values:

           dense              dense
[tensor1] -------> [tensor2] -------> [tensor3]
 ^^^^^^^            ^^^^^^^
 bufferA            bufferB

Second, use the value of tensor2 stored in bufferB to compute tensor3... but wait! We don't need bufferA anymore, so let's use it to store the value of tensor3:

           dense              dense
[tensor1] -------> [tensor2] -------> [tensor3]
                    ^^^^^^^            ^^^^^^^
                    bufferB            bufferA

Now is the tricky part. The "output value" of tensor1 will still point to bufferA, which now holds the values of tensor3. So if you call get_tensor(...) for the 1st tensor, you'll get incorrect values. The documentation of this method even states:

This function cannot be used to read intermediate results.

How to get around this?

Easy but limited way. You can specify the names of the nodes, output tensors of which you want to get the values of during conversion:
```
tflite_convert \
    -- # other options of your model
    --output_arrays="output_node,intermediate/node/n1,intermediate/node/n2"
```
Hard but flexible way. You can compile TF-Lite with Bazel (using this instruction). Then you can actually inject some logging code to Interpreter::Invoke() in the file tensorflow/lite/interpreter.cc. An ugly hack, but it works.

回答2:

tf.lite.Interpreter provides a API to peek into the model on a tensor level (https://www.tensorflow.org/api_docs/python/tf/lite/Interpreter#get_tensor). So basically you need to know the tensor indexes, for both input(s) and output(s), of the node and then call "get_tensor()" at each inference.

来源：https://stackoverflow.com/questions/56885007/visualize-tflite-graph-and-get-intermediate-values-of-a-particular-node

标签

tensorflow