数据集准备及数据预处理_1.准备数据集

最新推荐文章于 2024-12-04 23:19:39 发布

weixin_26752765

最新推荐文章于 2024-12-04 23:19:39 发布

阅读量1.5k

点赞数

文章标签：机器学习 python 人工智能

原文链接：https://medium.com/@fabulousjeong/updating-old-tensorflow-codes-to-new-tensorflow-2-0-style-ed4dca5f42f

版权

本文聚焦于机器学习项目中的关键步骤——数据集准备和预处理。内容涵盖从获取数据到清洗、转换，为TensorFlow 2.0代码更新提供指导。

摘要由CSDN通过智能技术生成

数据集准备及数据预处理

In this post I will introduce an example of how to upgrade legacy tensorflow codes to new style. tensorflow 2.0 recommends using Keras which is a well known high level API. Their sequential style approach makes your code more simple. And the script would be worked line by line, so we can easily understand the working process.

在这篇文章中，我将介绍一个如何将旧版tensorflow代码升级为新样式的示例。 tensorflow 2.0建议使用Keras，这是众所周知的高级API。他们的顺序样式方法使您的代码更简单。而且脚本将逐行工作，因此我们可以轻松地了解工作过程。

Let me introduce a guide by my BEGAN repository

让我介绍一下BEGAN存储库中的指南

1.准备数据集 (1. Prepare a dataset)

For preparing the dataset, at the previous version, I made the ImageIterator class which provides preprocessing and iterator. Obviously, it is not quite hard, cos we can find a kind guide on the homepage for that. But it looks cumbersome. In tensorflow 2.0 you don’t have to make that kind of class, we just use ImageDataGenerator which is a predefined API for data generation. We can set parameters for preprocessing and data augmentation when we create ImageDataGenerator. I gave some parameters for normalization and horizontal flipping as data augmentation as shown in below.

为了准备数据集，在以前的版本中，我制作了ImageIterator 类，其提供预处理和迭代器。显然，这并不难，因为我们可以在主页上找到一种类似的指南。但这看起来很麻烦。在tensorflow 2.0中，您不必创建此类，我们只使用ImageDataGenerator ，这是用于数据生成的预定义API。创建ImageDataGenerator时，可以设置用于预处理和数据扩充的参数。我提供了一些参数进行标准化和水平翻转，作为数据增强，如下所示。

2.设计一个网络。 (2. Design a network.)

The began network consists of a decoder and an encoder. In the previous repository, to implement a Convolution Layer, I used 3 low level functions as shown in below.

初始网络由解码器和编码器组成。在先前的存储库中，为了实现卷积层，我使用了3个低级函数，如下所示。

And I also predefined some parameters about layer weights.

我还预定义了一些有关图层权重的参数。

So, someone used to define a custom function for reducing code lines and better readability.

因此，曾经有人定义了一个自定义函数以减少代码行并提高可读性。

But, with Keras, I only needed just a line.

但是，有了Keras，我只需要一条线。

The high level API includes parameters on how to define activation and bias. So we don’t have to make any custom function. We just use predefined Keras APIs. I could design the decoder network using about 20 lines. Awesome!!

高级API包含有关如何定义激活和偏差的参数。因此，我们不必进行任何自定义功能。我们只使用预定义的Keras API。我可以使用大约20条线设计解码器网络。太棒了!!

Before training the network we have to make a model which takes inputs and returns outputs. Keras provides an API for that. Look at the codes to feel how simple it is.

在训练网络之前，我们必须先建立一个模型，该模型接受输入并返回输出。 Keras为此提供了一个API。查看代码以了解它有多简单。

It is interesting I didn’t write any lines for defining variable scope like previous codes. It works automatically on the Keras API.

有趣的是，我没有写任何行来定义变量范围，如先前的代码。它可以在Keras API上自动运行。

3.训练和测试模型 (3. Train and Test a Model)

3.1 Train

3.1火车

What I really love about Keras is I don’t have to use sess.run anymore. I think that the old style process makes codes quite complex. Look at the flow, we can easily follow these lines how this is gonna work.

我真正喜欢Keras的地方是我不再需要使用sess.run 。我认为旧样式的流程使代码变得相当复杂。看一下流程，我们可以很容易地遵循以下原则。

The model only had been trained for 5 hours on a Nvidia 1080TI. But the generator could show quite impressive outputs.

该模型仅在Nvidia 1080TI上训练了5个小时。但是生成器可能会显示出令人印象深刻的输出。

3.2 Test

3.2测试

Look at the super tiny test codes. There’s no excuse for not updating your legacy repository.

看一下超小的测试代码。没有任何理由不更新旧版存储库。

I also updated my DCGAN repository. Visit the repository, switch branches and check updated codes. I derived help from a tutorial that introduced the tensorflow homepage. I would like to recommend you read the tutorial too.

我还更新了DCGAN存储库。访问存储库，切换分支并检查更新的代码。我从介绍tensorflow主页的教程中获得了帮助。我也建议您阅读本教程。