ai css 线条粗细_如何训练AI将您的设计模型转换为HTML和CSS

ai css 线条粗细

by Emil Wallner

埃米尔·沃尔纳(Emil Wallner)

如何训练AI将您的设计模型转换为HTML和CSS (How you can train an AI to convert your design mockups into HTML and CSS)

Within three years, deep learning will change front-end development. It will increase prototyping speed and lower the barrier for building software.

三年之内，深度学习将改变前端开发。 它将提高原型制作速度，并降低构建软件的障碍。

The field took off last year when Tony Beltramelli introduced the pix2code paper and Airbnb launched sketch2code.

去年，当Tony Beltramelli提出pix2code纸，而Airbnb推出了sketch2code时，这一领域开始兴起 。

Currently, the largest barrier to automating front-end development is computing power. However, we can use current deep learning algorithms, along with synthesized training data, to start exploring artificial front-end automation right now.

当前，自动化前端开发的最大障碍是计算能力。 但是，我们可以使用当前的深度学习算法以及综合的训练数据来立即开始探索人工前端自动化。

In this post, we’ll teach a neural network how to code a basic a HTML and CSS website based on a picture of a design mockup. Here’s a quick overview of the process:

在这篇文章中，我们将教一个神经网络如何基于设计模型的图片编写一个基本HTML和CSS网站。 以下是该过程的快速概述：

1)将设计图像提供给经过训练的神经网络 (1) Give a design image to the trained neural network)

2)神经网络将图像转换为HTML标记 (2) The neural network converts the image into HTML markup)

3)渲染输出 (3) Rendered output)

We’ll build the neural network in three iterations.

我们将通过三个迭代来构建神经网络。

First, we’ll make a bare minimum version to get a hang of the moving parts. The second version, HTML, will focus on automating all the steps and explaining the neural network layers. In the final version, Bootstrap, we’ll create a model that can generalize and explore the LSTM layer.

首先，我们将制作一个最低限度的最低版本，以掌握更多的活动部件。 第二个版本HTML将着重于自动化所有步骤并解释神经网络层。 在最终版本Bootstrap中，我们将创建一个可以概括和探索LSTM层的模型。

All the code is prepared on GitHub and FloydHub in Jupyter notebooks. All the FloydHub notebooks are inside the floydhub directory and the local equivalents are under local.

所有代码均在Jupyter笔记本中的GitHub和FloydHub上准备。 所有FloydHub笔记本都在floydhub目录中，而本地等效项在local下。

The models are based on Beltramelli‘s pix2code paper and Jason Brownlee’s image caption tutorials. The code is written in Python and Keras, a framework on top of TensorFlow.

这些模型基于Beltramelli的pix2code纸和Jason Brownlee的图像说明教程 。 该代码是用Python和Keras(TensorFlow之上的框架)编写的。

If you’re new to deep learning, I’d recommend getting a feel for Python, backpropagation, and convolutional neural networks. My three earlier posts on FloydHub’s blog will get you started:

如果您是深度学习的新手，建议您对Python，反向传播和卷积神经网络有所了解。 我在FloydHub博客上发表的前三篇文章将帮助您入门：

• My First Weekend Of Deep Learning

  我的深度学习的第一个周末

• Coding The History Of Deep Learning

  编码深度学习的历史

• Colorizing B&W Photos with Neural Networks

  使用神经网络为黑白照片着色

核心逻辑 (Core Logic)

Let’s recap our goal. We want to build a neural network that will generate HTML/CSS markup that corresponds to a screenshot.

让我们回顾一下我们的目标。 我们要构建一个神经网络，该网络将生成与屏幕截图相对应HTML / CSS标记。

When you train the neural network, you give it several screenshots with matching HTML.

训练神经网络时，您会给它几个具有匹配HTML的屏幕截图。

It learns by predicting all the matching HTML markup tags one by one. When it predicts the next markup tag, it receives the screenshot as well as all the correct markup tags until that point.

它通过逐一预测所有匹配HTML标记来学习。 当它预测下一个标记标签时，它将接收屏幕截图以及所有正确的标记标签，直到该点为止。

Here is a simple training data example in a Google Sheet.

这是Google表格中的一个简单的训练数据示例 。

Creating a model that predicts word by word is the most common approach today. There are other approaches, but that’s the method we’ll use throughout this tutorial.

创建一个可以逐词预测的模型是当今最常见的方法。 还有其他方法 ，但这就是我们在本教程中将使用的方法。

Notice that for each prediction it gets the same screenshot. So if it has to predict 20 words, it will get the same design mockup twenty times. For now, don’t worry about how the neural network works. Focus on grasping the input and output of the neural network.

注意，对于每个预测，它都会获得相同的屏幕截图。 因此，如果必须预测20个单词，它将获得20次相同的设计样机。 现在，不必担心神经网络如何工作。 专注于掌握神经网络的输入和输出。

Let’s focus on the previous markup. Say we train the network to predict the sentence “I can code.” When it receives “I,” then it predicts “can.” Next time it will receive “I can” and predict “code.” It receives all the previous words and only has to predict the next word.

让我们集中于上一个标记。 假设我们训练网络以预测句子“我可以编码”。 当它收到“ I”时，它将预测“可以”。 下次它将收到“我可以”并预测“代码”。 它接收所有先前的单词，而只需要预测下一个单词。

The neural network creates features from the data. The network builds features to link the input data with the output data. It has to create representations to understand what is in each screenshot, the HTML syntax, that it has predicted. This builds the knowledge to predict the next tag.

神经网络根据数据创建特征。 网络构建了将输入数据与输出数据链接的功能。 它必须创建表示形式以了解每个屏幕快照中所预测的内容(HTML语法)。 这将积累知识以预测下一个标签。

When you want to use the trained model for real-world usage, it’s similar to when you train the model. The text is generated one by one with the same screenshot each time. Instead of feeding it with the correct HTML tags, it receives the markup it has generated so far. Then, it predicts the next markup tag. The prediction is initiated with a “start tag” and stops when it predicts an “end tag” or reaches a max limit. Here’s another example in a Google Sheet.

当您想将训练后的模型用于实际使用时，与训练模型时相似。 每次使用相同的屏幕截图一一生成文本。 它没有提供正确HTML标记，而是接收到目前为止生成的标记。 然后，它预测下一个标记标签。 预测以“开始标签”开始，并在预测“结束标签”或达到最大限制时停止。 这是Google表格中的另一个示例。

“ Hello World”版本 (“Hello World” Version)

Let’s build a “hello world” version. We’ll feed a neural network a screenshot with a website displaying “Hello World!” and teach it to generate the markup.

让我们构建一个“ hello world”版本。 我们将使用显示“ Hello World！”的网站为神经网络提供屏幕截图。 并教它生成标记。

First, the neural network maps the design mockup into a list of pixel values. From 0–255 in three channels — red, blue, and green.

首先，神经网络将设计模型映射到像素值列表中。 从0到255在三个通道中-红色，蓝色和绿色。

To represent the markup in a way that the neural network understands, I use one hot encoding. Thus, the sentence “I can code” could be mapped like the below.

为了以神经网络理解的方式表示标记，我使用一种热编码 。 因此，句子“我可以编码”可以像下面这样映射。

In the above graphic, we include the start and end tag. These tags are cues for when the network starts its predictions and when to stop.

在上图中，我们包含了开始和结束标记。 这些标记是网络何时开始其预测以及何时停止的线索。

For the input data, we will use sentences, starting with the first word and then adding each word one by one. The output data is always one word.

对于输入数据，我们将使用句子，从第一个单词开始，然后将每个单词一个接一个地添加。 输出数据始终为一个字。

Sentences follow the same logic as words. They also need the same input length. Instead of being capped by the vocabulary, they are bound by maximum sentence length. If it’s shorter than the maximum length, you fill it up with empty words, a word with just zeros.

句子的逻辑与单词相同。 它们还需要相同的输入长度。 它们不受最大词汇量的限制，而不受词汇量的限制。 如果它小于最大长度，则将其填充为空单词，即只有零的单词。

As you see, words are printed from right to left. This forces each word to change position for each training round. This allows the model to learn the sequence instead of memorizing the position of each word.

如您所见，单词是从右到左打印的。 这迫使每个单词改变每个训练回合的位置。 这使模型可以学习顺序，而不必记住每个单词的位置。

In the below graphic there are four predictions. Each row is one prediction. To the left are the images represented in their three color channels: red, green and blue and the previous words. Outside of the brackets are the predictions one by one, ending with a red square to mark the end.

在下图中，有四个预测。 每一行都是一个预测。 左侧是以三个颜色通道表示的图像：红色，绿色和蓝色以及前一个单词。 括号之外是一个接一个的预测，以红色正方形标记结束。

#Length of longest sentencemax_caption_len = 3#Size of vocabulary vocab_size = 3

# Load one screenshot for each word and turn them into digits images = []for i in range(2):    images.append(img_to_array(load_img('screenshot.jpg', target_size=(224, 224))))images = np.array(images, dtype=float)# Preprocess input for the VGG16 modelimages = preprocess_input(images)

#Turn start tokens into one-hot encodinghtml_input = np.array(            [[[0., 0., 0.], #start             [0., 0., 0.],             [1., 0., 0.]],             [[0., 0., 0.], #start <HTML>Hello World!</HTML>             [1., 0., 0.],             [0., 1., 0.]]])

#Turn next word into one-hot encodingnext_words = np.array(            [[0., 1., 0.], # <HTML>Hello World!</HTML>             [0., 0., 1.]]) # end

# Load the VGG16 model trained on imagenet and output the classification featureVGG = VGG16(weights='imagenet', include_top=True)# Extract the features from the imagefeatures = VGG.predict(images)

#Load the feature to the network, apply a dense layer, and repeat the vectorvgg_feature = Input(shape=(1000,))vgg_feature_dense = Dense(5)(vgg_feature)vgg_feature_repeat = RepeatVector(max_caption_len)(vgg_feature_dense)# Extract information from the input seqence language_input = Input(shape=(vocab_size, vocab_size))language_model = LSTM(5, return_sequences=True)(language_input)

# Concatenate the information from the image and the inputdecoder = concatenate([vgg_feature_repeat, language_model])# Extract information from the concatenated outputdecoder = LSTM(5, return_sequences=False)(decoder)# Predict which word comes nextdecoder_output = Dense(vocab_size, activation='softmax')(decoder)# Compile and run the neural networkmodel = Model(inputs=[vgg_feature, language_input], outputs=decoder_output)model.compile(loss='categorical_crossentropy', optimizer='rmsprop')

# Train the neural networkmodel.fit([features, html_input], next_words, batch_size=2, shuffle=False, epochs=1000)

In the hello world version, we use three tokens: start, <HTML><center><H1>Hello World!<;/H1&gt;</center></HTML> and end. A token can be anything. It can be a character, word, or sentence. Character versions require a smaller vocabulary but constrain the neural network. Word level tokens tend to perform best.

在hello world版本中，我们使用三个标记： start ，