Tesnsorflow2.0 kears 读取数据

Tensorflow 2.0 的keras库中，提供了读取训练数据集的接口，可以读取
boston_housing,cifar10,cifa100,fashion_mnist,imdb,mnist和reuters等数据集。

这些数据的访问都是通过load_data()实现的。由于网络的原因，下载经常不成功。分析一下Google提供的源码可以知道load_data的实现并不复杂。学习Tensorflow的同学可以自己下载源数据文件后，自己编写一个load_data函数就可以正常读取训练集了。
下面的代码是mnist的load_data：

@keras_export('keras.datasets.mnist.load_data')
def load_data(path='mnist.npz'):
  """Loads the MNIST dataset.
  Arguments:
      path: path where to cache the dataset locally
          (relative to ~/.keras/datasets).
  Returns:
      Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
  License:
      Yann LeCun and Corinna Cortes hold the copyright of MNIST dataset,
      which is a derivative work from original NIST datasets.
      MNIST dataset is made available under the terms of the
      [Creative Commons Attribution-Share Alike 3.0 license.](
      https://creativecommons.org/licenses/by-sa/3.0/)
  """
  origin_folder = 'https://storage.googleapis.com/tensorflow/tf-keras-datasets/'
  path = get_file(
      path,
      origin=origin_folder + 'mnist.npz',
      file_hash=
      '731c5ac602752760c8e48fbffcf8c3b850d9dc2a2aedcf2cc48468fc17b673d1')
  with np.load(path, allow_pickle=True) as f:
    x_train, y_train = f['x_train'], f['y_train']
    x_test, y_test = f['x_test'], f['y_test']

    return (x_train, y_train), (x_test, y_test)

主要的功能文件下载是使用了get_file。我们只需要修改一下get_file 函数的cache_folder和cache_subdir参数就可以实现本地mnist数据集读取了。

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

def load_data(path='mnist.npz'):
  origin_folder = 'https://storage.googleapis.com/tensorflow/tf-keras-datasets/'
  path = tf.keras.utils.get_file(
      path,
      origin=origin_folder + 'mnist.npz',
      cache_dir='DataSet/',
      cache_subdir=""
      )
  with np.load(path, allow_pickle=True) as f:
    x_train, y_train = f['x_train'], f['y_train']
    x_test, y_test = f['x_test'], f['y_test']

    return (x_train, y_train), (x_test, y_test)
(x_train,y_train),(x_test,y_test)=load_data(path='mnist.npz')
print(x_train.shape,y_train.shape,x_test.shape,y_test.shape)

get_file 的功能主要是下载文件、保存文件以及从cache中读取缓存文件。我们已经下载好了mnist.npz，就可以直接从缓存中读取文件，不用下载了。但是origin还是要提供正确的url格式，因为get_file会检查。
cache_dir这个参数设置为mnist.npz所在的目录。cache_subdir设置为空。
下面的代码展示了文件中的内容：

index=89
img = x_train[index]
plt.imshow(img)
print(y_train[index])