tensorflow

大型分布式存储方案MinIO介绍，看完你就懂了！

由 点点圈 提交于 2021-01-29 09:23:39
1、MinIO是什么？ 官方解释：MinIO 是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小，从几kb到最大5T不等。 MinIO是一个非常轻量的服务,可以很简单的和其他应用的结合，类似 NodeJS, Redis 或者 MySQL。 2、MinIO有哪些优势？ 2.1 开发文档全面 MinIO作为一款基于Golang 编程语言开发的一款高性能的分布式式存储方案的开源项目，有十分完善的官方文档。。 官网文档地址：https://docs.min.io/cn/ 2.2 高性能 MinIO号称是目前速度最快的对象存储服务器。在标准硬件上，对象存储的读/写速度最高可以高达183 GB/s和171 GB/s。对象存储可以作为主存储层，用来处理Spark、Presto、TensorFlow、H2O.ai等各种复杂工作负载以及成为Hadoop HDFS的替代品。 MinIO用作云原生应用程序的主要存储，和传统对象存储相比，云原生应用程序需要更高的吞吐量和更低的延迟。而这些都是MinIO能够达成的性能指标。 2.3 支持全面 目前MinIO支持市面主流的开发语言并且可以通过SDK快速集成快速集成使用。 2.4 AWS

tensoflow-trained ssd model not working after converting to tensorflow-lite for raspi

由 梦想的初衷 提交于 2021-01-29 09:22:43
问题 System information Laptop: Linux Ubuntu Tensorflow 1.15.0 Raspi: Raspberry Pi 4 tflite-runtime 2.5.0 tensorflow-estimator 1.14.0 Coral Edge TPU Hello everybody, I am stuck at getting my trained model running on raspi. I trained ssd_mobilenet_v2_coco model from tensorflow 1 modelzoo with my own custom dataset on google cloud with this config file where I did few changes: model { ssd { num_classes: 3 image_resizer { fixed_shape_resizer { height: 720 width: 1280 } } feature_extractor { type:

Merging Two Trained Networks for Inferencing Sequentially

由 ε祈祈猫儿з 提交于 2021-01-29 09:21:37
问题 I am trying to merge two trained neural networks. I have two trained Keras model files A and B. Model A is for image super-resolution and model B is for image colorization. I am trying to merge two trained networks so that I can inference SR+colorization faster. (I am not willing to use a single network to accomplish both SR and colorization tasks. I need to use two different networks for SR and colorization tasks.) Any tips on how to merge two Keras neural networks? 回答1: As long a the shape

understanding tensorflow binary image classification results

由 ﹥＞﹥吖頭↗ 提交于 2021-01-29 09:01:08
问题 For one of my first attempts at using Tensor flow I've followed the Binary Image Classification tutorial https://www.tensorflow.org/tutorials/keras/text_classification_with_hub#evaluate_the_model. I was able to follow the tutorial fine, but then I wanted to try to inspect the results more closely, namely I wanted to see what predictions the model made for each item in the test data set. In short, I wanted to see what "label" (1 or 0) it would predict applies to a given movie review. So I

How to compute number of weights of CNN?

由 孤街醉人 提交于 2021-01-29 08:55:01
问题 How can we compute number of weights considering a convolutional neural network that is used to classify images into two classes : INPUT: 100x100 gray-scale images. LAYER 1: Convolutional layer with 60 7x7 convolutional filters (stride=1, valid padding). LAYER 2: Convolutional layer with 100 5x5 convolutional filters (stride=1, valid padding). LAYER 3: A max pooling layer that down-samples Layer 2 by a factor of 4 (e.g., from 500x500 to 250x250) LAYER 4: Dense layer with 250 units LAYER 5:

Why tensorflow is slower with GPU instead of CPU?

由 假如想象 提交于 2021-01-29 08:54:48
问题 This is a really simple neural network: n_pts = 500000 np.random.seed(0) Xa = np.array([np.random.normal(13, 2, n_pts), np.random.normal(12, 2, n_pts)]).T Xb = np.array([np.random.normal(8, 2, n_pts), np.random.normal(6, 2, n_pts)]).T X = np.vstack((Xa, Xb)) y = np.matrix(np.append(np.zeros(n_pts), np.ones(n_pts))).T # Create a new Keras model model = Sequential() model.add(Dense(units=1, input_shape=(2,), activation='sigmoid')) adam = Adam(lr=0.1) model.compile(adam, loss='binary

How to compute the complexity of machine learning models

由 核能气质少年 提交于 2021-01-29 08:45:49
问题 I am working on comparison of deep learning models with application in Vehicular network communication security. I want to know how I can compute the complexity of these models to know the performance of my proposed ones. I am making use of tensorflow 回答1: You can compare the complexity of two deep networks with respect to space and time. Regarding space complexity: Number of parameters in your model -> this is directly proportional to the amount of memory consumed by your model. Regarding

how to enable GPU visible for ML runtime environment on databricks?

由 隐身守侯 提交于 2021-01-29 08:32:10
问题 I am trying to run some TensorFlow (2.2) example code on databricks/GPU (p2.xlarge) with environment as: 6.6 ML, spark 2.4.5, GPU, Scala 2.11 Keras version : 2.2.5 nvidia-smi NVIDIA-SMI 440.64.00 Driver Version: 440.64.00 CUDA Version: 10.2 I have checked https://docs.databricks.com/applications/deep-learning/single-node-training/tensorflow.html#install-tensorflow-22-on-databricks-runtime-66-ml&language-GPU But, I do not want to run the shell commands every time the databricks GPU clusters is

Intent classification with large number of intent classes

由 眉间皱痕 提交于 2021-01-29 08:21:46
问题 I am working on a data set of approximately 3000 questions and I want to perform intent classification. The data set is not labelled yet , but from the business perspective, there's a requirement of identifying approximately 80 various intent classes . Let's assume my training data has approximately equal number of each classes and is not majorly skewed towards some of the classes. I am intending to convert the text to word2vec or Glove and then feed into my classifier. I am familiar with

How to random_crop an unlabeled tensorflow Dataset? ValueError: Dimensions must be equal, but are 4 and 3

由 六眼飞鱼酱① 提交于 2021-01-29 08:12:51
问题 I am trying to augment (random crop) images while loading them using a tensorflow Dataset. I am getting this error when I call the method tf.image.random_crop in the mapped function: ValueError: Dimensions must be equal, but are 4 and 3 for '{{node random_crop/GreaterEqual}} = GreaterEqual[T=DT_INT32](random_crop/Shape, random_crop/size)' with input shapes: [4], [3]. In order to reproduce the error, just place some png images in the directory: ./img/class0/ Then run this code: import os
订阅 tensorflow