tensor

why do we “pack” the sequences in pytorch?

怎甘沉沦 提交于 2019-11-27 00:01:45
问题 I was trying to replicate How to use packing for variable-length sequence inputs for rnn but I guess I first need to understand why we need to "pack" the sequence. I understand why we need to "pad" them but why is "packing" ( through pack_padded_sequence ) necessary? Any high-level explanation would be appreciated! 回答1: I have stumbled upon this problem too and below is what I figured out. When training RNN (LSTM or GRU or vanilla-RNN), it is difficult to batch the variable length sequences.

使用PyTorch创建神经网络

China☆狼群 提交于 2019-11-26 19:24:11
2019年年初,ApacheCN组织志愿者翻译了PyTorch1.0版本中文文档( github地址 ),同时也获得了PyTorch官方授权,我相信已经有许多人在 中文文档官网 上看到了。不过目前 校对 还缺人手,希望大家踊跃参与。之前一段时间我们和PyTorch的有关负责人Bruce Lin一直在进行邮件交流。在之后适当的时候,我们会组织志愿者进行其他有关PyTorch的项目,欢迎大家加入我们,关注我们。更希望我们的一系列工作能够对大家有所帮助。 译者: bat67 校对者: FontTian 可以使用 torch.nn 包来构建神经网络. 我们已经介绍了 autograd , nn 包则依赖于 autograd 包来定义模型并对它们求导。一个 nn.Module 包含各个层和一个 forward(input) 方法,该方法返回 output 。 例如,下面这个神经网络可以对数字进行分类: 这是一个简单的前馈神经网络(feed-forward network)。它接受一个输入,然后将它送入下一层,一层接一层的传递,最后给出输出。 一个神经网络的典型训练过程如下: 定义包含一些可学习参数(或者叫权重)的神经网络 在输入数据集上迭代 通过网络处理输入 计算损失(输出和正确答案的距离) 将梯度反向传播给网络的参数 更新网络的权重,一般使用一个简单的规则: weight = weight

Keras input explanation: input_shape, units, batch_size, dim, etc

橙三吉。 提交于 2019-11-26 17:59:42
For any Keras layer ( Layer class), can someone explain how to understand the difference between input_shape , units , dim , etc.? For example the doc says units specify the output shape of a layer. In the image of the neural net below hidden layer1 has 4 units. Does this directly translate to the units attribute of the Layer object? Or does units in Keras equal the shape of every weight in the hidden layer times the number of units? In short how does one understand/visualize the attributes of the model - in particular the layers - with the image below? Units: The amount of "neurons", or

How to understand the term `tensor` in TensorFlow?

家住魔仙堡 提交于 2019-11-26 17:56:30
问题 I am new to TensorFlow. While I am reading the existing documentation, I found the term tensor really confusing. Because of it, I need to clarify the following questions: What is the relationship between tensor and Variable , tensor vs. tf.constant , 'tensor' vs. tf.placeholder ? Are they all types of tensors? 回答1: TensorFlow doesn't have first-class Tensor objects, meaning that there are no notion of Tensor in the underlying graph that's executed by the runtime. Instead the graph consists of

Understanding tensordot

て烟熏妆下的殇ゞ 提交于 2019-11-26 11:21:01
After I learned how to use einsum , I am now trying to understand how np.tensordot works. However, I am a little bit lost especially regarding the various possibilities for the parameter axes . To understand it, as I have never practiced tensor calculus, I use the following example: A = np.random.randint(2, size=(2, 3, 5)) B = np.random.randint(2, size=(3, 2, 4)) In this case, what are the different possible np.tensordot and how would you compute it manually? The idea with tensordot is pretty simple - We input the arrays and the respective axes along which the sum-reductions are intended. The

Tensorflow基础杂记

故事扮演 提交于 2019-11-26 07:20:58
测试是否安装成功 jupyter里 import tensorflow as tf tf . __version__ 按Ctrl+Enter执行 '1.2.1' 即为成功。 import tensorflow as tf # 创建一个常量运算, 将作为一个节点加入到默认计算图中 hello = tf . constant ( "Hello, World!" ) # 创建一个TF对话 sess = tf . Session ( ) # 运行并获得结果 print ( sess . run ( hello ) ) b'Hello, World!' 'b’表示Bytes literals(字节文字) 计算图 TensorFlow = Tensor + Flow Tensor张量 数据结构:多维数组 Flow流 计算模型:张量之间通过计算而转换的过程 TensorFlow是一个通过计算图的形式表述计算的编程系统,每一个计算都是计算图上的一个节点,节点之间的边描述了计算之间的关系。 TensorFlow有两种边: 常规边(实线):代表数据依赖关系。一个节点的运算输出成为另一个节点的输入,两个节点之间有tensor流动(值传递)。 特殊边(虚线):不携带值,表示两个节点之间的控制相关性。比如,happens-before关系,源节点必须在目的节点执行前完成执行。 张量 import

Keras input explanation: input_shape, units, batch_size, dim, etc

情到浓时终转凉″ 提交于 2019-11-26 04:28:59
问题 For any Keras layer ( Layer class), can someone explain how to understand the difference between input_shape , units , dim , etc.? For example the doc says units specify the output shape of a layer. In the image of the neural net below hidden layer1 has 4 units. Does this directly translate to the units attribute of the Layer object? Or does units in Keras equal the shape of every weight in the hidden layer times the number of units? In short how does one understand/visualize the attributes

DCGAN及其TensorFlow源码

本小妞迷上赌 提交于 2019-11-26 04:00:31
上一节我们提到G和D由多层感知机定义。深度学习中对图像处理应用最好的模型是CNN,那么如何把CNN与GAN结合?DCGAN是这方面最好的尝试之一。源码: https://github.com/Newmu/dcgan_code 。DCGAN论文作者用theano实现的,他还放上了其他人实现的版本,本文主要讨论tensorflow版本。 TensorFlow版本的源码: https://github.com/carpedm20/DCGAN-tensorflow DCGAN把上述的G和D换成了两个卷积神经网络(CNN)。但不是直接换就可以了,DCGAN对卷积神经网络的结构做了一些改变,以提高样本的质量和收敛的速度,这些改变有: 取消所有pooling层。G网络中使用转置卷积(transposed convolutional layer)进行上采样,D网络中用加入strided的卷积代替pooling。 在D和G中均使用batch normalization 去掉FC层,使网络变为全卷积网络 G网络中使用ReLU作为激活函数,最后一层使用tanh D网络中使用LeakyReLU作为激活函数 这些改变在代码中都可以看到。DCGAN论文中提到对CNN结构有三点重要的改变: Allconvolutional net (Springenberg et al., 2014) 全卷积网络 判别模型D

AttributeError:'Tensor' object has no attribute '_keras_history'

那年仲夏 提交于 2019-11-26 03:38:00
当你的model各层的维度都对上了,最后马上就要输出了,忽然来这个错,我的心里是崩溃的。那么具体的原因是什么呢? 具体而言,就是我们在用keras训练的模型过程中出了叛徒啦!!! (我卡卡西早已看穿一切) 废话不多说,这个问题的主要原因是我们使用了tensorflow的一些函数导致的,比如我们可能会用tf.XXXX函数做一些处理。而tensorflow的函数处理过后的tensor叫做tf.tensor,keras处理过后的tensor叫做keras.tensor。在网上查到的一个比较详细的回答如下 : 解决方法人家给提了三个,第一个就是说你别整tf的函数,直接用keras.backend的函数,出来的就是咱自己人:keras.tensor。 第二个方法说的是我们可以用Lambda层把tf的函数包住,这样出来的tensor还是keras.tensor,因为Lambda层就是keras.layers中的层,而这个层就是处理这种不需要参数,只需要数据变化的事物的,所以我们有问题尽管找到。第三个方法,我还查了一下,哦!Designate(指定)keras function,哎和第一个差不多,就是告你别用tensorflow的函数。 我在这里就不介绍第一个和第三个了,因为我们也不知道你用的是什么函数,是吧,我就只说Lambda层的方法。 例如,我们想要使用 from keras import

Understanding tensordot

自古美人都是妖i 提交于 2019-11-26 01:59:30
问题 After I learned how to use einsum , I am now trying to understand how np.tensordot works. However, I am a little bit lost especially regarding the various possibilities for the parameter axes . To understand it, as I have never practiced tensor calculus, I use the following example: A = np.random.randint(2, size=(2, 3, 5)) B = np.random.randint(2, size=(3, 2, 4)) In this case, what are the different possible np.tensordot and how would you compute it manually? 回答1: The idea with tensordot is