tensorflow

How implement Batch Norm with SWA in Tensorflow?

由 偶尔善良 提交于 2021-02-11 06:13:06
问题 I am using Stochastic Weight Averaging (SWA) with Batch Normalization layers in Tensorflow 2.2. For Batch Norm I use tf.keras.layers.BatchNormalization . For SWA I use my own code to average the weights (I wrote my code before tfa.optimizers.SWA appeared). I have read in multiple sources that if using batch norm and SWA we must run a forward pass to make certain data (running mean and st dev of activation weights and/or momentum values?) available to the batch norm layers. What I do not

How do I connect two keras models into one model?

由 て烟熏妆下的殇ゞ 提交于 2021-02-11 05:55:39
问题 Let's say I have a ResNet50 model and I wish to connect the output layer of this model to the input layer of a VGG model. This is the ResNet model and the output tensor of ResNet50: img_shape = (164, 164, 3) resnet50_model = ResNet50(include_top=False, input_shape=img_shape, weights = None) print(resnet50_model.output.shape) I get the output: TensorShape([Dimension(None), Dimension(6), Dimension(6), Dimension(2048)]) Now I want a new layer where I reshape this output tensor to (64,64,18) Then

How do I connect two keras models into one model?

由 穿精又带淫゛_ 提交于 2021-02-11 05:55:17
问题 Let's say I have a ResNet50 model and I wish to connect the output layer of this model to the input layer of a VGG model. This is the ResNet model and the output tensor of ResNet50: img_shape = (164, 164, 3) resnet50_model = ResNet50(include_top=False, input_shape=img_shape, weights = None) print(resnet50_model.output.shape) I get the output: TensorShape([Dimension(None), Dimension(6), Dimension(6), Dimension(2048)]) Now I want a new layer where I reshape this output tensor to (64,64,18) Then

Tensorflow variable initialization inside loss function

由 左心房为你撑大大i 提交于 2021-02-11 04:52:39
问题 I have an object detection model implemented in tensorflow.keras (version 1.15). I am trying to implement a modified (hybrid) loss function in my model. Basically I need a few variables defined in my loss function because I am processing the y_true and y_pred provided in my classification loss function (a focal loss to be exact). So, I naturally resided in implementing my ops inside the loss function. I have defined a WrapperClass to initialize my variables: class LossWrapper(object): def _

Tensorflow variable initialization inside loss function

由 拥有回忆 提交于 2021-02-11 04:52:28
问题 I have an object detection model implemented in tensorflow.keras (version 1.15). I am trying to implement a modified (hybrid) loss function in my model. Basically I need a few variables defined in my loss function because I am processing the y_true and y_pred provided in my classification loss function (a focal loss to be exact). So, I naturally resided in implementing my ops inside the loss function. I have defined a WrapperClass to initialize my variables: class LossWrapper(object): def _

TensorFlow与PyTorch对比

由 六月ゝ 毕业季﹏ 提交于 2021-02-11 03:15:52
用TensorFlow还是PyTorch？从TensorFlow开始学起还是PyTorch？在一年前，这个问题毫无争议，当然是TensorFlow. 但时过境迁，现在的情况大不一样了，下面就来分析对比一下这两个主流框架。 首先看一下最近的统计数据，下图的数据是从arxiv论文中统计得到的。上面的黄线是TensorFlow的使用比例，下面的红线是PyTorch的使用比例，可以看出，最近的数据两者已经不差上下，甚至红线PyTorch在2019.6要略胜一筹。 右边的条形图是从1月到6月的累计数据，TensorFlow占比还是要略高一点，但是23%的增长率明显要低于PyTorch的194%.也就是说，现在TensorFlow和PyTorch在学术界的 使用率上来说已经不差上下了 。 来源：https://www.oreilly.com/ideas/one-simple-graphic-researchers-love-pytorch-and-tensorflow 决定一个框架的使用率有哪些因素呢？我总结有以下四个方面： 易用性 速度 算子数量 开源模型 第一是 易用性 。PyTorch自从2018年12月PyTorch 1.0 stable版本正式发布以来，只有半年多的时间，能迅速增长易用性功不可没。PyTorch的易用性一方面是debug简单，可以直接设置断点查看各个tensor的值

Tensorflow @tf.function - Cannot get session inside Tensorflow graph function

由 两盒软妹~` 提交于 2021-02-11 02:50:30
问题 I'm trying to use the @tf.function directive with the Keras functional API, to create a TF graph in the training step of a simple neural network. I'm using Tensorflow v 2.1.0 installed with Python 3.7. However I obtain the runtime error as in title and I would appreciate any hint to understand the reason of that. The code is the following. import tensorflow as tf import numpy as np # import the CIFAR10 dataset and normalise the feature distributions (train_images, train_labels), (test_images,

Tensorflow @tf.function - Cannot get session inside Tensorflow graph function

由 无人久伴 提交于 2021-02-11 02:49:36
问题 I'm trying to use the @tf.function directive with the Keras functional API, to create a TF graph in the training step of a simple neural network. I'm using Tensorflow v 2.1.0 installed with Python 3.7. However I obtain the runtime error as in title and I would appreciate any hint to understand the reason of that. The code is the following. import tensorflow as tf import numpy as np # import the CIFAR10 dataset and normalise the feature distributions (train_images, train_labels), (test_images,

Tensorflow @tf.function - Cannot get session inside Tensorflow graph function

由 巧了我就是萌 提交于 2021-02-11 02:47:59
问题 I'm trying to use the @tf.function directive with the Keras functional API, to create a TF graph in the training step of a simple neural network. I'm using Tensorflow v 2.1.0 installed with Python 3.7. However I obtain the runtime error as in title and I would appreciate any hint to understand the reason of that. The code is the following. import tensorflow as tf import numpy as np # import the CIFAR10 dataset and normalise the feature distributions (train_images, train_labels), (test_images,

tensorflow 模型权重导出

由 亡梦爱人 提交于 2021-02-11 02:35:28
tensorflow在保存权重模型时多使用tf.train.Saver().save 函数进行权重保存，保存的ckpt文件无法直接打开，不利于将模型权重导入到其他框架使用（如Caffe、Keras等）。 好在tensorflow提供了相关函数 tf.train.NewCheckpointReader 可以对ckpt文件进行权重查看，因此可以通过该函数进行数据导出。 1 import tensorflow as tf 2 import h5py 3 4 cpktLogFileName = r ' ./checkpoint/checkpoint ' # cpkt 文件路径 5 with open(cpktLogFileName, ' r ' ) as f: 6 # 权重节点往往会保留多个epoch的数据，此处获取最后的权重数据 7 cpktFileName = f.readline().split( ' " ' )[1 ] 8 9 h5FileName = r ' ./model/net_classification.h5 ' 10 11 reader = tf.train.NewCheckpointReader(cpktFileName) 12 f = h5py.File(h5FileName, ' w ' ) 13 t_g = None 14 for key in sorted
订阅 tensorflow