Tensorflow官网例程学习记录 图片的四种显示方式 数据加强

官网网址：
https://www.tensorflow.org/tutorials/images/cnn

最简例子

model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)), #定义了维数
tf.keras.layers.Dense(128, activation=‘relu’),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10)
])

或者
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation=‘relu’, input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.summary()

For each example the model returns a vector of “logits” or “log-odds” scores, one for each class.
predictions = model(x_train[:1]).numpy()
predictions

array([[ 0.48826253, -1.3999053 , 0.09991729, -0.45614815, -0.02467804,
0.04654375, 0.60656685, -0.00573175, -0.8574921 , 0.18740411]],
dtype=float32)

The tf.nn.softmax function converts these logits to “probabilities” for each class:
tf.nn.softmax(predictions).numpy()

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

This loss is equal to the negative log probability of the true class: It is zero if the model is sure of the correct class.
This untrained model gives probabilities close to random (1/10 for each class), so the initial loss should be close to -tf.log(1/10) ~= 2.3.

loss_fn(y_train[:1], predictions).numpy()
2.2656865

第二种数据读取方式

mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

#Add a channels dimension
x_train = x_train[…, tf.newaxis].astype(“float32”)
x_test = x_test[…, tf.newaxis].astype(“float32”)

Use tf.data to batch and shuffle the dataset:
train_ds = tf.data.Dataset.from_tensor_slices(
(x_train, y_train)).shuffle(10000).batch(32)

第二种模型定义方式

class MyModel(Model):
def init(self):
super(MyModel, self).init()
self.conv1 = Conv2D(32, 3, activation=‘relu’)
self.flatten = Flatten()
self.d1 = Dense(128, activation=‘relu’)
self.d2 = Dense(10)

def call(self, x):
x = self.conv1(x)
x = self.flatten(x)
x = self.d1(x)
return self.d2(x)

#Create an instance of the model
model = MyModel()

Use tf.GradientTape to train the model

交叉熵
loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
求均值，用于求loss
train_loss = tf.keras.metrics.Mean(name=‘train_loss’)
准确度应输入两个参数
train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name=‘train_accuracy’)

loss = loss_object(labels, predictions)
gradients = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))

train_loss(loss)
train_accuracy(labels, predictions)

计算方法

gradients = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))

均值提供了一种方法result来显示结果
template = ‘Epoch {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}’
print(template.format(epoch + 1, train_loss.result(), train_accuracy.result() * 100）

绘图方式

numpy形式 用专用命令集读取， plt.imshow直显
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()

张量类型图片显示，.numpy().astype(“uint8”)，image/255
plt像素值可在0-1或整数

下面语句取出的是张量
image_batch, label_batch = next(iter(train_dataset)
for images, labels in train_dataset.take(1):

下面语句读出的的图片类型PIL.Image
keras.preprocessing.image.load_img
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()

张量转化为numpy(), 可显示
plt.imshow(images[i].numpy().astype(“uint8”))
或者矢量/255显示
plt.imshow(augmented_image[0]/255）
张量，甚至是一维通道转化为PIL.Image
plt.imshow(tf.keras.preprocessing.image.array_to_img(display_list[i]))

PIL.Image转化为张量
keras.preprocessing.image.img_to_array
增加维数之后可以参加张量模型计算
tf.expand_dims(img_array, 0)

数据集用take取出图形张量以显示
class_names = train_dataset.class_names
plt.figure(figsize=(10, 10))
for images, labels in train_dataset.take(1):
for i in range(9):
ax = plt.subplot(3, 3, i + 1)
plt.imshow(images[i].numpy().astype(“uint8”))
plt.title(class_names[labels[i]])
plt.axis(“off”)
plt.title(class_names[labels[i]])
plt.axis(“off”)

图片名称直接显示 PIL.Image.open
PIL.Image.open 读的是图片路径， plt.imshow读的是numpy矩阵，还有张量转化plt.imshow(images[i].numpy().astype(“uint8”))
print(np.min(first_image), np.max(first_image))

cv2读入 ，两种方式输出plt.imshow cv2.imshow

import cv2
import matplotlib.pyplot as plt
 
 #读入彩色图片
messi=cv2.imread('messi.jpg',1)
 
#使用matplotlib.pyplot的imshow显示图片
plt.imshow(messi),plt.title('messi_plt')
plt.xticks([]),plt.yticks([]) #隐藏坐标轴
plt.show()
 
##调换r、b通道，生成rgb顺序的图片并显示
b,g,r=cv2.split(messi) #通道的拆分
messi_rgb=cv2.merge((r,g,b)) #通道的融合
plt.imshow(messi_rgb),plt.title('messi_rgb_plt')
plt.xticks([]),plt.yticks([])
plt.show()
 
#使用opencv的imshow显示图片
cv2.imshow('messi_cv',messi)
cv2.waitKey(0)
cv2.destroyAllWindows()

网络文件下载

形成列表文件
image_count = len(list(data_dir.glob(’/.jpg’)))
print(image_count)

取训练数据

取出，分出train和validation子集
It’s good practice to use a validation split when developing your model. We will use 80% of the images for training, and 20% for validation.
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
data_dir,
validation_split=0.2,
subset=“training”,
seed=123,
image_size=(img_height, img_width),
batch_size=batch_size)

class_names = train_ds.class_names

或者用cardinality取出数量
validation_dataset = image_dataset_from_directory(validation_dir,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE)
val_batches = tf.data.experimental.cardinality(validation_dataset)
test_dataset = validation_dataset.take(val_batches // 5)
validation_dataset = validation_dataset.skip(val_batches // 5)

直接取出train
train_dataset = image_dataset_from_directory(train_dir,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE)

加速存取

AUTOTUNE = tf.data.experimental.AUTOTUNE
train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE)
val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)

预处理归一化

直接调用 preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input
或者
model = Sequential([
data_augmentation,
layers.experimental.preprocessing.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
layers.Conv2D(16, 3, padding=‘same’, activation=‘relu’),
])

data_augmentation = keras.Sequential(
[ layers.experimental.preprocessing.RandomFlip(“horizontal”, input_shape=(img_height, img_width, 3)),
layers.experimental.preprocessing.RandomRotation(0.1),
layers.experimental.preprocessing.RandomZoom(0.1),
])

迁移学习

做迁移训练，只训练Dense时，设置
When you set layer.trainable = False, the BatchNormalization layer will run in inference mode, and will not update its mean and variance statistics.

迁移训练即使解冻，也要 BatchNormalization training = False
When you unfreeze a model that contains BatchNormalization layers in order to do fine-tuning, you should keep the BatchNormalization layers in inference mode by passing training = False when calling the base model. Otherwise, the updates applied to the non-trainable weights will destroy what the model the model has learned.

Most common is to use a moving average of mean and std for your batch normalization as used by Keras for example (https://github.com/keras-team/keras/blob/master/keras/layers/normalization.py). If you just turn it off the network will perform worse on the same data, due to changes in how the images are processed.

This is done by storing the average mean and the average std of all the batches used during training the network. Then in inference this moving average is used for normalization.

#Create the base model from the pre-trained model MobileNet V2
IMG_SHAPE = IMG_SIZE + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights=‘imagenet’)

acc = history.history['accuracy']
val_acc = history.history['val_accuracy']

loss = history.history['loss']
val_loss = history.history['val_loss']

plt.figure(figsize=(8, 8))
plt.subplot(2, 1, 1)
plt.plot(acc, label='Training Accuracy')
plt.plot(val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.ylabel('Accuracy')
plt.ylim([min(plt.ylim()),1])
plt.title('Training and Validation Accuracy')

plt.subplot(2, 1, 2)
plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.ylabel('Cross Entropy')
plt.ylim([0,1.0])
plt.title('Training and Validation Loss')
plt.xlabel('epoch')
plt.show()

Note: This should only be attempted after you have trained the top-level classifier with the pre-trained model set to non-trainable. If you add a randomly initialized classifier on top of a pre-trained model and attempt to train all layers jointly, the magnitude of the gradient updates will be too large (due to the random weights from the classifier) and your pre-trained model will forget what it has learned.

acc列表累加的的方法：
acc = history.history[‘accuracy’]

acc += history_fine.history['accuracy']
val_acc += history_fine.history['val_accuracy']

loss += history_fine.history['loss']
val_loss += history_fine.history['val_loss']

plt.figure(figsize=(8, 8))
plt.subplot(2, 1, 1)
plt.plot(acc, label='Training Accuracy')
plt.plot(val_acc, label='Validation Accuracy')
plt.ylim([0.8, 1])
plt.plot([initial_epochs-1,initial_epochs-1],
          plt.ylim(), label='Start Fine Tuning')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(2, 1, 2)
plt.plot(loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.ylim([0, 1.0])
plt.plot([initial_epochs-1,initial_epochs-1],
         plt.ylim(), label='Start Fine Tuning')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.xlabel('epoch')
plt.show()

#Retrieve a batch of images from the test set
image_batch, label_batch = test_dataset.as_numpy_iterator().next()
predictions = model.predict_on_batch(image_batch).flatten()

#Apply a sigmoid since our model returns logits

predictions = tf.nn.sigmoid(predictions)
predictions = tf.where(predictions < 0.5, 0, 1)

print('Predictions:\n', predictions.numpy())
print('Labels:\n', label_batch)

plt.figure(figsize=(10, 10))
for i in range(9):
  ax = plt.subplot(3, 3, i + 1)
  plt.imshow(image_batch[i].astype("uint8"))
  plt.title(class_names[predictions[i]])
  plt.axis("off")

注意二分类或多分类时使用sigmoid激活函数

PIL用np.array直接转换成np

读文件分割标号
imagenet_labels = np.array(open(labels_path).read().splitlines())

单图片预测
IMAGE_SHAPE = (224, 224)
classifier = tf.keras.Sequential([hub.KerasLayer(classifier_url, input_shape=IMAGE_SHAPE+(3,))])
grace_hopper = tf.keras.utils.get_file(‘image.jpg’,‘https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg’)
grace_hopper = Image.open(grace_hopper).resize(IMAGE_SHAPE)
ult = classifier.predict(grace_hopper[np.newaxis, …])

批量预测
data_root = tf.keras.utils.get_file(
‘flower_photos’,‘https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz’, untar=True)
image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1/255)
image_data = image_generator.flow_from_directory(data_root, target_size=IMAGE_SHAPE)

classifier and headless model url

方式一 Download the classifier
https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/2

方式二 Download the headless model
feature_extractor_url https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/2

方式三 tf.keras.application

inputs = tf.keras.Input(shape=(160, 160, 3))
x = data_augmentation(inputs)
x = preprocess_input(x)
x = base_model(x, training=False)
x = global_average_layer(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = prediction_layer(x)
model = tf.keras.Model(inputs, outputs)

Attach a classification head
Now wrap the hub layer in a tf.keras.Sequential model, and add a new classification layer.

classifier = tf.keras.Sequential([
hub.KerasLayer(classifier_url, input_shape=IMAGE_SHAPE+(3,))
])

model = tf.keras.Sequential([
feature_extractor_layer,
layers.Dense(image_data.num_classes)
])

steps_per_epoch = np.ceil(image_data.samples/image_data.batch_size)

lambda
You’re using a lambda expression (or anonymous function), to sort your list of tuples based on a certain key. pair[1] indicates that you are sorting with a key of the elements in the index position of 1 in each tuple (the strings). Sorting with strings sorts by alphabetical order, which results in the output you are seeing.

Were you to use the first element in each tuple as the sorting key for instance (pair[0]), you would then sort in increasing numerical order:

pairs = [(1, ‘one’), (2, ‘two’), (3, ‘three’), (4, ‘four’)]
pairs.sort(key=lambda pair: pair[0])
pairs
[(1, ‘one’), (2, ‘two’), (3, ‘three’), (4, ‘four’)]

{}将现在的时间代入方式

model.predict_on_batch() model.predict() 区别
The difference lies in when you pass as x data that is larger than one batch.
predict will go through all the data, batch by batch, predicting labels. It thus internally does the splitting in batches and feeding one batch at a time.
predict_on_batch, on the other hand, assumes that the data you pass in is exactly one batch and thus feeds it to the network. It won’t try to split it (which, depending on your setup, might prove problematic for your GPU memory if the array is very big)

Data augmentation

一种转字符形式
num_classes = metadata.features[‘label’].num_classes
get_label_name = metadata.features[‘label’].int2str

next只取下一个值

取字符串方法
get_label_name = metadata.features[‘label’].int2str
print(get_label_name(4))
image, label = next(iter(train_ds))
print(label)
print(label.numpy())
_ = plt.imshow(image)
_ = plt.title(get_label_name(label))

数据加强

resize_and_rescale = tf.keras.Sequential([
layers.experimental.preprocessing.Resizing(IMG_SIZE, IMG_SIZE),
layers.experimental.preprocessing.Rescaling(1./255)
])

data_augmentation = tf.keras.Sequential([
layers.experimental.preprocessing.RandomFlip(“horizontal_and_vertical”),
layers.experimental.preprocessing.RandomRotation(0.2),
])

方式一
model = tf.keras.Sequential([
resize_and_rescale,
data_augmentation,
layers.Conv2D(16, 3, padding=‘same’, activation=‘relu’),
layers.MaxPooling2D(),

Rest of your model

])

方式二
batch_size = 32
AUTOTUNE = tf.data.experimental.AUTOTUNE

def prepare(ds, shuffle=False, augment=False):

Resize and rescale all datasets

ds = ds.map(lambda x, y: (resize_and_rescale(x), y),
num_parallel_calls=AUTOTUNE)

if shuffle:
ds = ds.shuffle(1000)

Batch all datasets

ds = ds.batch(batch_size)

Use data augmentation only on the training set

if augment:
ds = ds.map(lambda x, y: (data_augmentation(x, training=True), y),
num_parallel_calls=AUTOTUNE)

Use buffered prefecting on all datasets

return ds.prefetch(buffer_size=AUTOTUNE)

train_ds = prepare(train_ds, shuffle=True, augment=True)
val_ds = prepare(val_ds)
test_ds = prepare(test_ds)

create a layers.Lambda layer

加层方法
random_invert = random_invert()

加子类方法
implement a custom layer by subclassing
above layers.preprocessing utilities

tf.data and tf.image

def visualize(original, augmented):
fig = plt.figure()
plt.subplot(1,2,1)
plt.title(‘Original image’)
plt.imshow(original)

plt.subplot(1,2,2)
plt.title(‘Augmented image’)
plt.imshow(augmented)

flipped = tf.image.flip_left_right(image)
visualize(image, flipped)

grayscaled = tf.image.rgb_to_grayscale(image)
visualize(image, tf.squeeze(grayscaled))
_ = plt.colorbar()

saturated = tf.image.adjust_saturation(image, 3)
visualize(image, saturated)

bright = tf.image.adjust_brightness(image, 0.4)
visualize(image, bright)

cropped = tf.image.central_crop(image, central_fraction=0.5)
visualize(image,cropped)

rotated = tf.image.rot90(image)
visualize(image, rotated)

def resize_and_rescale(image, label):
image = tf.cast(image, tf.float32)
image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE])
image = (image / 255.0)
return image, label

def augment(image,label):
image, label = resize_and_rescale(image, label)

Add 6 pixels of padding

image = tf.image.resize_with_crop_or_pad(image, IMG_SIZE + 6, IMG_SIZE + 6)

Random crop back to the original size

image = tf.image.random_crop(image, size=[IMG_SIZE, IMG_SIZE, 3])
image = tf.image.random_brightness(image, max_delta=0.5) # Random brightness
image = tf.clip_by_value(image, 0, 1)
return image, label

Segmentation

for image, mask in train.take(1):
sample_image, sample_mask = image, mask
#print(type(sample_image))
display([sample_image, sample_mask])

sample_image张量转class 'PIL.Image.Image’
必须用到 plt.imshow(tf.keras.preprocessing.image.array_to_img(display_list[i]))
不能用 plt.imshow(display_list[i].numpy())， 因为
对mask单维图像,plt.imshow不能处理 TypeError: Invalid shape (128, 128, 1) for image data

取出模型中间层，并使之成为模型输出
base_model = tf.keras.applications.MobileNetV2(input_shape=[128, 128, 3], include_top=False)
#Use the activations of these layers
layer_names = [
‘block_1_expand_relu’, # 64x64
‘block_3_expand_relu’, # 32x32
‘block_6_expand_relu’, # 16x16
‘block_13_expand_relu’, # 8x8
‘block_16_project’, # 4x4
]
layers = [base_model.get_layer(name).output for name in layer_names]

#Create the feature extraction model
down_stack = tf.keras.Model(inputs=base_model.input, outputs=layers)
down_stack.trainable = False

上采样和上卷积
up_stack = [
pix2pix.upsample(512, 3), # 4x4 -> 8x8
pix2pix.upsample(256, 3), # 8x8 -> 16x16
pix2pix.upsample(128, 3), # 16x16 -> 32x32
pix2pix.upsample(64, 3), # 32x32 -> 64x64
]

list[:-1] 取除最后一位外的所有成员
list[-1] 取最后一位成员

def unet_model(output_channels):
inputs = tf.keras.layers.Input(shape=[128, 128, 3])
x = inputs

Downsampling through the model

skips = down_stack(x)
取最后输出，作为up的输入
x = skips[-1]
除最后一层外，各层反转
skips = reversed(skips[:-1])

Upsampling and establishing the skip connections

for up, skip in zip(up_stack, skips):
x = up(x)
concat = tf.keras.layers.Concatenate()
x = concat([x, skip])

This is the last layer of the model

last = tf.keras.layers.Conv2DTranspose(
output_channels, 3, strides=2,
padding=‘same’) #64x64 -> 128x128

x = last(x)
return tf.keras.Model(inputs=inputs, outputs=x)

def create_mask(pred_mask):
print(pred_mask.shape)
(1, 128, 128, 3)
pred_mask = tf.argmax(pred_mask, axis=-1)
缩纬 (1, 128, 128)
print(pred_mask.shape)
扩纬 (1, 128, 128, tf.newaxis)
pred_mask = pred_mask[…, tf.newaxis]
恢复模型需要的纬数
return pred_mask[0]

数据集方法

dataset, info = tfds.load(‘oxford_iiit_pet:3..’, with_info=True)
TRAIN_LENGTH = info.splits[‘train’].num_examples
BATCH_SIZE = 64
BUFFER_SIZE = 1000
STEPS_PER_EPOCH = TRAIN_LENGTH // BATCH_SIZE
train = dataset[‘train’].map(load_image_train, num_parallel_calls=tf.data.experimental.AUTOTUNE)
test = dataset[‘test’].map(load_image_test)

欣赏下面的点和线及颜色

注意第二行两个索引取出np中的数字
plt.imshow(train_images[i], cmap=plt.cm.binary)
plt.xlabel(class_names[train_labels[i][0]])
plt.show(）

print(image[0].shape)
print(np.max(augmented_image[0],1).shape) 每行取最大值的那个点的三像素
两种显示方式

np.max(a，1)
np.argmax(a， 1)
tf.argmax(pred_mask, axis=-1)

pix2pix
https://www.tensorflow.org/tutorials/generative/pix2pix
(···条消息)TensorFlow 2.0 U-Net图像分割教程_XiaoMeow的博客-CSDN博客 https://blog.csdn.net/XiaoMeow/article/details/100587879
(···条消息)深度学习中的concatenate使用_暮日落流年的博客-CSDN博客 https://blog.csdn.net/alxe_made/article/details/80506051?utm_source=blogxgwz1