tensorflow学习：使用tensorflow实现word embedding

最新推荐文章于 2024-08-03 08:00:00 发布

qoopqpqp

最新推荐文章于 2024-08-03 08:00:00 发布

阅读量6.7k

点赞数

分类专栏：深度学习 tensorflow 文章标签： tensorflow 神经网络

本文链接：https://blog.csdn.net/qoopqpqp/article/details/76037334

版权

本文档介绍如何使用TensorFlow实现简单的Word Embedding，主要基于word2vec的skip-gram模型，通过下载英文语料库进行训练，并利用t-SNE进行可视化。虽然作者承认理解不深，但分享了初步的学习成果。

摘要由CSDN通过智能技术生成

今天学习使用tensorflow实现word embedding，下面的例子来自于tensorflow的官方文档，实现word embedding本身是比较复杂的，下文只用比较简单的方法实现，限于本人刚入门，水平有限，有些细节还没有彻底搞明白，还需要再花些时间研究，现把今天的研究成果记下供日后继续完善。

文本参考：https://liusida.github.io/2016/11/14/study-embeddings/ 感谢作者的努力与奉献

程序功能：

程序下载一份英文语料库，然后使用word2vec+skip-gram实现word的embedding，最后使用t-SNE选取部分点降维可视化。

# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Basic word2vec example."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import collections
import math
import os
import random
import zipfile

import numpy as np
from six.moves import urllib
from six.moves import xrange  # pylint: disable=redefined-builtin
import tensorflow as tf

# Step 1: Download the data.
url = 'http://mattmahoney.net/dc/'


def maybe_download(filename, expected_bytes):
  """下载数据集，文件中没有标点，全部小写，词与词之间用空格隔开."""
  if not os.path.exists(filename):
    filename, _ = urllib.request.urlretrieve(url + filename, filename)
  statinfo = os.stat(filename)
  if statinfo.st_size == expected_bytes:
    print('Found and verified', filename)
  else:
    print(statinfo.st_size)
    raise Exception(
        'Failed to verify ' + filename + '. Can you get to it with a browser?')
  return filename

filename = maybe_download('text8.zip', 31344016)


# Read the data into a list of strings.
def read_data(filename):
  """Extract the first file enclosed in a zip file as a list of words."""
  with zipfile.ZipFile(filename) as f:
    data = tf.compat.as_str(f.read(f.namelist()[0])).split()
  return data

#读取压缩包中第一个文件的全部内容
vocabulary = read_data(filename)
print('Data size', len(vocabulary))
print ('vocabulary:', vocabulary[:10])

# Step 2: Build the dictionary and replace rare words with UNK token.
vocabulary_size = 50000


def build_dataset(words, n_words):
  """Process raw inputs into a dataset."""
  count = [['UNK', -1]]
  count.extend(collections.Counter(words).most_common(n_words - 1))
  dictionary = dict()
  for word, _ in count:
    dictionary[word] = len(dictionary)
  data = list()
  unk_count = 0
  for word in words:
    if word in dictionary:
      index = dictionary[word]
    else:
      index = 0  # dictionary['UNK']
      unk_count += 1
    data.append(index)
  count[0][1] = unk_count
  reversed_dictionary = dict(zip(dictionary.values(), dictionary.keys()))
  return data, count, dictionary, reversed_dictionary

#data：把原文中的word转化成ID后的串
#count：[word, freq]存储的是word和word对应的出现次数
#dictionary：词到ID的对应关系
#reverse_dictionary：ID到词的对应关系
data, count, dictionary, reverse_dictionary = build_dataset(vocabulary,
                                                            vocabulary_size)
del vocabulary  # Hint to reduce memory.
print(