昇思MindSpore学习笔记6-01LLM原理和实践--FCN图像语义分割

最新推荐文章于 2024-09-14 21:53:48 发布
最新推荐文章于 2024-09-14 21:53:48 发布
阅读量905 收藏 19
点赞数 16
文章标签： 学习 笔记 深度学习
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/muren/article/details/140248786
版权

摘要：

        记录MindSpore AI框架使用FCN全卷积网络理解图像进行图像语议分割的过程、步骤和方法。包括环境准备、下载数据集、数据集加载和预处理、构建网络、训练准备、模型训练、模型评估、模型推理等。

一、概念

1.语义分割

图像语义分割

semantic segmentation

        图像处理

        机器视觉

                图像理解

        AI领域重要分支

        应用

                人脸识别

                物体检测

                医学影像

                卫星图像分析

                自动驾驶感知

        目的

                图像每个像素点分类

                输出与输入大小相同的图像

                输出图像的每个像素对应了输入图像每个像素的类别

        图像领域语义

                图像的内容

                对图片意思的理解

实例

2.FCN全卷积网络

Fully Convolutional Networks

图像语义分割框架

        2015年UC Berkeley提出

        端到端(end to end)像素级(pixel level)预测全卷积网络

全卷积神经网络主要使用三种技术：

1.卷积化Convolutional

VGG-16

        FCN的backbone

        输入224*224RGB图像

                固定大小的输入

                丢弃了空间坐标

                产生非空间输出

        输出1000个预测值

卷积层

        输出二维矩阵

        生成输入图片映射的heatmap

2.上采样Upsample

卷积过程

        卷积操作

        池化操作

特征图尺寸变小

上采样操作

        得到原图大小的稠密图像预测

双线性插值参数

初始化上采样逆卷积参数

反向传播学习非线性上采样

3.跳跃结构Skip Layer

将深层的全局信息与浅层的局部信息相结合

                             底层stride 32的预测FCN-32s    2倍上采样

融合（相加）  pool4层stride 16的预测FCN-16s    2倍上采样

融合（相加）  pool3层stride 8的预测FCN-8s

特点:

(1)不含全连接层(fc)的全卷积(fully conv)网络，可适应任意尺寸输入。

(2)增大数据尺寸的反卷积(deconv)层，能够输出精细的结果。

(3)结合不同深度层结果的跳级(skip)结构，同时确保鲁棒性和精确性。

二、环境准备

%%capture captured_output
# 实验环境已经预装了mindspore==2.2.14，如需更换mindspore版本，可更改下面mindspore的版本号
!pip uninstall mindspore -y
!pip install -i https://pypi.mirrors.ustc.edu.cn/simple mindspore==2.2.14
# 查看当前 mindspore 版本
!pip show mindspore

输出：

Name: mindspore
Version: 2.2.14
Summary: MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
Home-page: https://www.mindspore.cn
Author: The MindSpore Authors
Author-email: contact@mindspore.cn
License: Apache 2.0
Location: /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages
Requires: asttokens, astunparse, numpy, packaging, pillow, protobuf, psutil, scipy
Required-by:

三、数据处理

1.下载数据集

from download import download
​
url = "https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/dataset_fcn8s.tar"
​
download(url, "./dataset", kind="tar", replace=True)

输出：

Creating data folder...
Downloading data from https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/dataset_fcn8s.tar (537.2 MB)

file_sizes: 100%|█████████████████████████████| 563M/563M [00:03<00:00, 160MB/s]
Extracting tar file...
Successfully downloaded / unzipped to ./dataset
'./dataset'

2.数据预处理

PASCAL VOC 2012数据集图像分辨率不一致

        标准化处理

3.数据加载

混合PASCAL VOC 2012数据集、SDB数据集

import numpy as np
import cv2
import mindspore.dataset as ds
​
class SegDataset:
    def __init__(self,
                 image_mean,
                 image_std,
                 data_file='',
                 batch_size=32,
                 crop_size=512,
                 max_scale=2.0,
                 min_scale=0.5,
                 ignore_label=255,
                 num_classes=21,
                 num_readers=2,
                 num_parallel_calls=4):
​
        self.data_file = data_file
        self.batch_size = batch_size
        self.crop_size = crop_size
        self.image_mean = np.array(image_mean, dtype=np.float32)
        self.image_std = np.array(image_std, dtype=np.float32)
        self.max_scale = max_scale
        self.min_scale = min_scale
        self.ignore_label = ignore_label
        self.num_classes = num_classes
        self.num_readers = num_readers
        self.num_parallel_calls = num_parallel_calls
        max_scale > min_scale
​
    def preprocess_dataset(self, image, label):
        image_out = cv2.imdecode(np.frombuffer(image, dtype=np.uint8), cv2.IMREAD_COLOR)
        label_out = cv2.imdecode(np.frombuffer(label, dtype=np.uint8), cv2.IMREAD_GRAYSCALE)
        sc = np.random.uniform(self.min_scale, self.max_scale)
        new_h, new_w = int(sc * image_out.shape[0]), int(sc * image_out.shape[1])
        image_out = cv2.resize(image_out, (new_w, new_h), interpolation=cv2.INTER_CUBIC)
        label_out = cv2.resize(label_out, (new_w, new_h), interpolation=cv2.INTER_NEAREST)
​
        image_out = (image_out - self.image_mean) / self.image_std
        out_h, out_w = max(new_h, self.crop_size), max(new_w, self.crop_size)
        pad_h, pad_w = out_h - new_h, out_w - new_w
        if pad_h > 0 or pad_w > 0:
            image_out = cv2.copyMakeBorder(image_out, 0, pad_h, 0, pad_w, cv2.BORDER_CONSTANT, value=0)
            label_out = cv2.copyMakeBorder(label_out, 0, pad_h, 0, pad_w, cv2.BORDER_CONSTANT, value=self.ignore_label)
        offset_h = np.random.randint(0, out_h - self.crop_size + 1)
        offset_w = np.random.randint(0, out_w - self.crop_size + 1)
        image_out = image_out[offset_h: offset_h + self.crop_size, offset_w: offset_w + self.crop_size, :]
        label_out = label_out[offset_h: offset_h + self.crop_size, offset_w: offset_w+self.crop_size]
        if np.random.uniform(0.0, 1.0) > 0.5:
            image_out = image_out[:, ::-1, :]
            label_out = label_out[:, ::-1]
        image_out = image_out.transpose((2, 0, 1))
        image_out = image_out.copy()
        label_out = label_out.copy()
        label_out = label_out.astype("int32")
        return image_out, label_out
​
    def get_dataset(self):
        ds.config.set_numa_enable(True)
        dataset = ds.MindDataset(self.data_file, columns_list=["data", "label"],
                                 shuffle=True, num_parallel_workers=self.num_readers)
        transforms_list = self.preprocess_dataset
        dataset = dataset.map(operations=transforms_list, input_columns=["data", "label"],
                              output_columns=["data", "label"],
                              num_parallel_workers=self.num_parallel_calls)
        dataset = dataset.shuffle(buffer_size=self.batch_size * 10)
        dataset = dataset.batch(self.batch_size, drop_remainder=True)
        return dataset
​
​
# 定义创建数据集的参数
IMAGE_MEAN = [103.53, 116.28, 123.675]
IMAGE_STD = [57.375, 57.120, 58.395]
DATA_FILE = "dataset/dataset_fcn8s/mindname.mindrecord"
​
# 定义模型训练参数
train_batch_size = 4
crop_size = 512
min_scale = 0.5
max_scale = 2.0
ignore_label = 255
num_classes = 21
​
# 实例化Dataset
dataset = SegDataset(image_mean=IMAGE_MEAN,
                     image_std=IMAGE_STD,
                     data_file=DATA_FILE,
                     batch_size=train_batch_size,
                     crop_size=crop_size,
                     max_scale=max_scale,
                     min_scale=min_scale,
                     ignore_label=ignore_label,
                     num_classes=num_classes,
                     num_readers=2,
                     num_parallel_calls=4)
​
dataset = dataset.get_dataset()

4.训练集可视化

import numpy as np
import matplotlib.pyplot as plt
​
plt.figure(figsize=(16, 8))
​
# 对训练集中的数据进行展示
for i in range(1, 9):
    plt.subplot(2, 4, i)
    show_data = next(dataset.create_dict_iterator())
    show_images = show_data["data"].asnumpy()
    show_images = np.clip(show_images, 0, 1)
# 将图片转换HWC格式后进行展示
    plt.imshow(show_images[0].transpose(1, 2, 0))
    plt.axis("off")
    plt.subplots_adjust(wspace=0.05, hspace=0)
plt.show()

输出：

四、网络构建

FCN网络流程

        输入图像image

        pool1池化

                尺寸变为原始尺寸的1/2

        pool2池化

                尺寸变为原始尺寸的1/4

        pool3池化

                尺寸变为原始尺寸的1/8

        pool4池化

                尺寸变为原始尺寸的1/16

        pool5池化

                尺寸变为原始尺寸的1/32

        conv6-7卷积

                输出尺寸原图的1/32

        FCN-32s

                反卷积扩大到原始尺寸

        FCN-16s

                融合

                        conv7反卷积尺寸扩大两倍至原图的1/16

                        pool4特征图

                反卷积扩大到原始尺寸

        FCN-8s

                融合

                        conv7反卷积尺寸扩大4倍

                        pool4特征图反卷积扩大2倍

                        pool3特征图

                反卷积扩大到原始尺寸

构建FCN-8s网络代码：

import mindspore.nn as nn
​
class FCN8s(nn.Cell):
    def __init__(self, n_class):
        super().__init__()
        self.n_class = n_class
        self.conv1 = nn.SequentialCell(
            nn.Conv2d(in_channels=3, out_channels=64,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(64),
            nn.ReLU(),
            nn.Conv2d(in_channels=64, out_channels=64,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(64),
            nn.ReLU()
        )
        self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv2 = nn.SequentialCell(
            nn.Conv2d(in_channels=64, out_channels=128,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(128),
            nn.ReLU(),
            nn.Conv2d(in_channels=128, out_channels=128,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(128),
            nn.ReLU()
        )
        self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv3 = nn.SequentialCell(
            nn.Conv2d(in_channels=128, out_channels=256,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(256),
            nn.ReLU(),
            nn.Conv2d(in_channels=256, out_channels=256,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(256),
            nn.ReLU(),
            nn.Conv2d(in_channels=256, out_channels=256,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(256),
            nn.ReLU()
        )
        self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv4 = nn.SequentialCell(
            nn.Conv2d(in_channels=256, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU(),
            nn.Conv2d(in_channels=512, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU(),
            nn.Conv2d(in_channels=512, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU()
        )
        self.pool4 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv5 = nn.SequentialCell(
            nn.Conv2d(in_channels=512, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU(),
            nn.Conv2d(in_channels=512, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU(),
            nn.Conv2d(in_channels=512, out_channels=512,
                      kernel_size=3, weight_init='xavier_uniform'),
            nn.BatchNorm2d(512),
            nn.ReLU()
        )
        self.pool5 = nn.MaxPool2d(kernel_size=2, stride=2)
        self.conv6 = nn.SequentialCell(
            nn.Conv2d(in_channels=512, out_channels=4096,
                      kernel_size=7, weight_init='xavier_uniform'),
            nn.BatchNorm2d(4096),
            nn.ReLU(),
        )
        self.conv7 = nn.SequentialCell(
            nn.Conv2d(in_channels=4096, out_channels=4096,
                      kernel_size=1, weight_init='xavier_uniform'),
            nn.BatchNorm2d(4096),
            nn.ReLU(),
        )
        self.score_fr = nn.Conv2d(in_channels=4096, out_channels=self.n_class,
                                  kernel_size=1, weight_init='xavier_uniform')
        self.upscore2 = nn.Conv2dTranspose(in_channels=self.n_class, out_channels=self.n_class,
                                           kernel_size=4, stride=2, weight_init='xavier_uniform')
        self.score_pool4 = nn.Conv2d(in_channels=512, out_channels=self.n_class,
                                     kernel_size=1, weight_init='xavier_uniform')
        self.upscore_pool4 = nn.Conv2dTranspose(in_channels=self.n_class, out_channels=self.n_class,
                                                kernel_size=4, stride=2, weight_init='xavier_uniform')
        self.score_pool3 = nn.Conv2d(in_channels=256, out_channels=self.n_class,
                                     kernel_size=1, weight_init='xavier_uniform')
        self.upscore8 = nn.Conv2dTranspose(in_channels=self.n_class, out_channels=self.n_class,
                                           kernel_size=16, stride=8, weight_init='xavier_uniform')
​
    def construct(self, x):
        x1 = self.conv1(x)
        p1 = self.pool1(x1)
        x2 = self.conv2(p1)
        p2 = self.pool2(x2)
        x3 = self.conv3(p2)
        p3 = self.pool3(x3)
        x4 = self.conv4(p3)
        p4 = self.pool4(x4)
        x5 = self.conv5(p4)
        p5 = self.pool5(x5)
        x6 = self.conv6(p5)
        x7 = self.conv7(x6)
        sf = self.score_fr(x7)
        u2 = self.upscore2(sf)
        s4 = self.score_pool4(p4)
        f4 = s4 + u2
        u4 = self.upscore_pool4(f4)
        s3 = self.score_pool3(p3)
        f3 = s3 + u4
        out = self.upscore8(f3)
        return out

五、训练准备

1.导入VGG-16部分预训练权重

from download import download
from mindspore import load_checkpoint, load_param_into_net
​
url = "https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/fcn8s_vgg16_pretrain.ckpt"
download(url, "fcn8s_vgg16_pretrain.ckpt", replace=True)
def load_vgg16():
    ckpt_vgg16 = "fcn8s_vgg16_pretrain.ckpt"
    param_vgg = load_checkpoint(ckpt_vgg16)
    load_param_into_net(net, param_vgg)

输出：

Downloading data from https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/fcn8s_vgg16_pretrain.ckpt (513.2 MB)

file_sizes: 100%|█████████████████████████████| 538M/538M [00:03<00:00, 179MB/s]
Successfully downloaded file to fcn8s_vgg16_pretrain.ckpt

2.损失函数

交叉熵损失函数

mindspore.nn.CrossEntropyLoss()

计算FCN网络输出与mask之间的交叉熵损失

3.自定义评价指标 Metrics

用于评估模型效果

设共有 K+1个类

        从L_0 到L_{ki}

        其中包含一个空类或背景

P_{ij}表示本属于i类但被预测为j类的像素数量

P_{ii}表示真正的数量

P_{ij}P_{ji}则分别被解释为假正和假负

Pixel Accuracy

PA像素精度

        标记正确的像素占总像素的比例。

PA=\frac{\sum_{i=0}^{k}P_{ii}}{\sum_{i=0}^{k}\sum_{j=0}^{k}P_{ij}}

Mean Pixel Accuracy

MPA均像素精度

计算每个类内正确分类像素数的比例

求所有类的平均

MPA=\frac{1}{K+1}\sum \sum_{i=0}^{k}\frac{P_{ii}}{\sum_{j=0}^{k}P_{ij}}

Mean Intersection over Union

MloU均交并比

        语义分割的标准度量

                计算两个集合的交集和并集之比

                        交集为真实值（ground truth)

                        并集为预测值（predicted segmentation）

                两者之比：正真数 (intersection) /（真正+假负+假正（并集））

                在每个类上计算loU

                平均

MIoU=\frac{1}{K+1} \sum_{i=0}^{k}\frac{p_{ii}}{\sum_{j=0}^{k}p_{ij}+{\sum_{j=0}^{k}p_{ji}}-p_{ii}}

Frequency Weighted Intersection over Union

FWIoU频权交井比

根据每个类出现的频率设置权重

FWIoU=\frac{1}{\sum_{i=0}^{k}\sum_{j=0}^{k}p_{ij}} \sum_{i=0}^{k}\frac{p_{ii}}{\sum_{j=0}^{k}p_{ij}+{\sum_{j=0}^{k}p_{ji}}-p_{ii}}

import numpy as np
import mindspore as ms
import mindspore.nn as nn
import mindspore.train as train
​
class PixelAccuracy(train.Metric):
    def __init__(self, num_class=21):
        super(PixelAccuracy, self).__init__()
        self.num_class = num_class
​
    def _generate_matrix(self, gt_image, pre_image):
        mask = (gt_image >= 0) & (gt_image < self.num_class)
        label = self.num_class * gt_image[mask].astype('int') + pre_image[mask]
        count = np.bincount(label, minlength=self.num_class**2)
        confusion_matrix = count.reshape(self.num_class, self.num_class)
        return confusion_matrix
​
    def clear(self):
        self.confusion_matrix = np.zeros((self.num_class,) * 2)
​
    def update(self, *inputs):
        y_pred = inputs[0].asnumpy().argmax(axis=1)
        y = inputs[1].asnumpy().reshape(4, 512, 512)
        self.confusion_matrix += self._generate_matrix(y, y_pred)
​
    def eval(self):
        pixel_accuracy = np.diag(self.confusion_matrix).sum() / self.confusion_matrix.sum()
        return pixel_accuracy
​
​
class PixelAccuracyClass(train.Metric):
    def __init__(self, num_class=21):
        super(PixelAccuracyClass, self).__init__()
        self.num_class = num_class
​
    def _generate_matrix(self, gt_image, pre_image):
        mask = (gt_image >= 0) & (gt_image < self.num_class)
        label = self.num_class * gt_image[mask].astype('int') + pre_image[mask]
        count = np.bincount(label, minlength=self.num_class**2)
        confusion_matrix = count.reshape(self.num_class, self.num_class)
        return confusion_matrix
​
    def update(self, *inputs):
        y_pred = inputs[0].asnumpy().argmax(axis=1)
        y = inputs[1].asnumpy().reshape(4, 512, 512)
        self.confusion_matrix += self._generate_matrix(y, y_pred)
​
    def clear(self):
        self.confusion_matrix = np.zeros((self.num_class,) * 2)
​
    def eval(self):
        mean_pixel_accuracy = np.diag(self.confusion_matrix) / self.confusion_matrix.sum(axis=1)
        mean_pixel_accuracy = np.nanmean(mean_pixel_accuracy)
        return mean_pixel_accuracy
​
​
class MeanIntersectionOverUnion(train.Metric):
    def __init__(self, num_class=21):
        super(MeanIntersectionOverUnion, self).__init__()
        self.num_class = num_class
​
    def _generate_matrix(self, gt_image, pre_image):
        mask = (gt_image >= 0) & (gt_image < self.num_class)
        label = self.num_class * gt_image[mask].astype('int') + pre_image[mask]
        count = np.bincount(label, minlength=self.num_class**2)
        confusion_matrix = count.reshape(self.num_class, self.num_class)
        return confusion_matrix
​
    def update(self, *inputs):
        y_pred = inputs[0].asnumpy().argmax(axis=1)
        y = inputs[1].asnumpy().reshape(4, 512, 512)
        self.confusion_matrix += self._generate_matrix(y, y_pred)
​
    def clear(self):
        self.confusion_matrix = np.zeros((self.num_class,) * 2)
​
    def eval(self):
        mean_iou = np.diag(self.confusion_matrix) / (
            np.sum(self.confusion_matrix, axis=1) + np.sum(self.confusion_matrix, axis=0) -
            np.diag(self.confusion_matrix))
        mean_iou = np.nanmean(mean_iou)
        return mean_iou
​
​
class FrequencyWeightedIntersectionOverUnion(train.Metric):
    def __init__(self, num_class=21):
        super(FrequencyWeightedIntersectionOverUnion, self).__init__()
        self.num_class = num_class
​
    def _generate_matrix(self, gt_image, pre_image):
        mask = (gt_image >= 0) & (gt_image < self.num_class)
        label = self.num_class * gt_image[mask].astype('int') + pre_image[mask]
        count = np.bincount(label, minlength=self.num_class**2)
        confusion_matrix = count.reshape(self.num_class, self.num_class)
        return confusion_matrix
​
    def update(self, *inputs):
        y_pred = inputs[0].asnumpy().argmax(axis=1)
        y = inputs[1].asnumpy().reshape(4, 512, 512)
        self.confusion_matrix += self._generate_matrix(y, y_pred)
​
    def clear(self):
        self.confusion_matrix = np.zeros((self.num_class,) * 2)
​
    def eval(self):
        freq = np.sum(self.confusion_matrix, axis=1) / np.sum(self.confusion_matrix)
        iu = np.diag(self.confusion_matrix) / (
            np.sum(self.confusion_matrix, axis=1) + np.sum(self.confusion_matrix, axis=0) -
            np.diag(self.confusion_matrix))
​
        frequency_weighted_iou = (freq[freq > 0] * iu[freq > 0]).sum()
        return frequency_weighted_iou

六、模型训练

导入VGG-16预训练参数

实例化损失函数、优化器

Model接口编译网络

训练FCN-8s网络

import mindspore
from mindspore import Tensor
import mindspore.nn as nn
from mindspore.train import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor, Model
​
device_target = "Ascend"
mindspore.set_context(mode=mindspore.PYNATIVE_MODE, device_target=device_target)
​
train_batch_size = 4
num_classes = 21
# 初始化模型结构
net = FCN8s(n_class=21)
# 导入vgg16预训练参数
load_vgg16()
# 计算学习率
min_lr = 0.0005
base_lr = 0.05
train_epochs = 1
iters_per_epoch = dataset.get_dataset_size()
total_step = iters_per_epoch * train_epochs
​
lr_scheduler = mindspore.nn.cosine_decay_lr(min_lr,
                                            base_lr,
                                            total_step,
                                            iters_per_epoch,
                                            decay_epoch=2)
lr = Tensor(lr_scheduler[-1])
​
# 定义损失函数
loss = nn.CrossEntropyLoss(ignore_index=255)
# 定义优化器
optimizer = nn.Momentum(params=net.trainable_params(), learning_rate=lr, momentum=0.9, weight_decay=0.0001)
# 定义loss_scale
scale_factor = 4
scale_window = 3000
loss_scale_manager = ms.amp.DynamicLossScaleManager(scale_factor, scale_window)
# 初始化模型
if device_target == "Ascend":
    model = Model(net, loss_fn=loss, optimizer=optimizer, loss_scale_manager=loss_scale_manager, metrics={"pixel accuracy": PixelAccuracy(), "mean pixel accuracy": PixelAccuracyClass(), "mean IoU": MeanIntersectionOverUnion(), "frequency weighted IoU": FrequencyWeightedIntersectionOverUnion()})
else:
    model = Model(net, loss_fn=loss, optimizer=optimizer, metrics={"pixel accuracy": PixelAccuracy(), "mean pixel accuracy": PixelAccuracyClass(), "mean IoU": MeanIntersectionOverUnion(), "frequency weighted IoU": FrequencyWeightedIntersectionOverUnion()})
​
# 设置ckpt文件保存的参数
time_callback = TimeMonitor(data_size=iters_per_epoch)
loss_callback = LossMonitor()
callbacks = [time_callback, loss_callback]
save_steps = 330
keep_checkpoint_max = 5
config_ckpt = CheckpointConfig(save_checkpoint_steps=10,
                               keep_checkpoint_max=keep_checkpoint_max)
ckpt_callback = ModelCheckpoint(prefix="FCN8s",
                                directory="./ckpt",
                                config=config_ckpt)
callbacks.append(ckpt_callback)
model.train(train_epochs, dataset, callbacks=callbacks)

输出：

epoch: 1 step: 1, loss is 3.0504844
epoch: 1 step: 2, loss is 3.017057
epoch: 1 step: 3, loss is 2.9523003
epoch: 1 step: 4, loss is 2.9488814
epoch: 1 step: 5, loss is 2.666231
epoch: 1 step: 6, loss is 2.7145326
epoch: 1 step: 7, loss is 1.796408
epoch: 1 step: 8, loss is 1.5167583
epoch: 1 step: 9, loss is 1.6862022
epoch: 1 step: 10, loss is 2.4622822
......
epoch: 1 step: 1141, loss is 1.70966
epoch: 1 step: 1142, loss is 1.434751
epoch: 1 step: 1143, loss is 2.406475
Train epoch time: 762889.258 ms, per step time: 667.445 ms

七、模型评估

IMAGE_MEAN = [103.53, 116.28, 123.675]
IMAGE_STD = [57.375, 57.120, 58.395]
DATA_FILE = "dataset/dataset_fcn8s/mindname.mindrecord"
​
# 下载已训练好的权重文件
url = "https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/FCN8s.ckpt"
download(url, "FCN8s.ckpt", replace=True)
net = FCN8s(n_class=num_classes)
​
ckpt_file = "FCN8s.ckpt"
param_dict = load_checkpoint(ckpt_file)
load_param_into_net(net, param_dict)
​
if device_target == "Ascend":
    model = Model(net, loss_fn=loss, optimizer=optimizer, loss_scale_manager=loss_scale_manager, metrics={"pixel accuracy": PixelAccuracy(), "mean pixel accuracy": PixelAccuracyClass(), "mean IoU": MeanIntersectionOverUnion(), "frequency weighted IoU": FrequencyWeightedIntersectionOverUnion()})
else:
    model = Model(net, loss_fn=loss, optimizer=optimizer, metrics={"pixel accuracy": PixelAccuracy(), "mean pixel accuracy": PixelAccuracyClass(), "mean IoU": MeanIntersectionOverUnion(), "frequency weighted IoU": FrequencyWeightedIntersectionOverUnion()})
​
# 实例化Dataset
dataset = SegDataset(image_mean=IMAGE_MEAN,
                     image_std=IMAGE_STD,
                     data_file=DATA_FILE,
                     batch_size=train_batch_size,
                     crop_size=crop_size,
                     max_scale=max_scale,
                     min_scale=min_scale,
                     ignore_label=ignore_label,
                     num_classes=num_classes,
                     num_readers=2,
                     num_parallel_calls=4)
dataset_eval = dataset.get_dataset()
model.eval(dataset_eval)

输出：

Downloading data from https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/FCN8s.ckpt (1.00 GB)

file_sizes: 100%|██████████████████████████| 1.08G/1.08G [00:10<00:00, 99.7MB/s]
Successfully downloaded file to FCN8s.ckpt
/
{'pixel accuracy': 0.9734831394168291,
 'mean pixel accuracy': 0.9423324801371116,
 'mean IoU': 0.8961453779807752,
 'frequency weighted IoU': 0.9488883312345654}

八、模型推理

使用训练的网络对模型推理结果进行展示。

import cv2
import matplotlib.pyplot as plt
​
net = FCN8s(n_class=num_classes)
# 设置超参
ckpt_file = "FCN8s.ckpt"
param_dict = load_checkpoint(ckpt_file)
load_param_into_net(net, param_dict)
eval_batch_size = 4
img_lst = []
mask_lst = []
res_lst = []
# 推理效果展示(上方为输入图片，下方为推理效果图片)
plt.figure(figsize=(8, 5))
show_data = next(dataset_eval.create_dict_iterator())
show_images = show_data["data"].asnumpy()
mask_images = show_data["label"].reshape([4, 512, 512])
show_images = np.clip(show_images, 0, 1)
for i in range(eval_batch_size):
    img_lst.append(show_images[i])
    mask_lst.append(mask_images[i])
res = net(show_data["data"]).asnumpy().argmax(axis=1)
for i in range(eval_batch_size):
    plt.subplot(2, 4, i + 1)
    plt.imshow(img_lst[i].transpose(1, 2, 0))
    plt.axis("off")
    plt.subplots_adjust(wspace=0.05, hspace=0.02)
    plt.subplot(2, 4, i + 5)
    plt.imshow(res[i])
    plt.axis("off")
    plt.subplots_adjust(wspace=0.05, hspace=0.02)
plt.show()

输出：

九、总结

FCN

        使用全卷积层

        通过学习让图片实现端到端分割。

        优点：

                输入接受任意大小的图像

                高效，避免了由于使用像素块而带来的重复存储和计算卷积的问题。

        待改进之处：

                结果不够精细。比较模糊和平滑，边界处细节不敏感。

                像素分类，没有考虑像素之间的关系（如不连续性和相似性）

                忽略空间规整（spatial regularization）步骤，缺乏空间一致性。

muren
关注
【4月组队学习-llm-universe笔记】 0基础小白 RAG应用开发实践笔记
05-22
附LangChain自定义Embedding封装讲解
算法部署-使用TensorRT-LLM部署大模型-附详细优化+分析流程教程-优质大模型部署项目实战.zip
06-05
算法部署-使用TensorRT-LLM部署大模型-附详细优化+分析流程教程-优质大模型部署项目实战.zip 算法部署-使用TensorRT-LLM部署大模型-附详细优化+分析流程教程-优质大模型部署项目实战.zip算法部署-使用TensorRT-...
昇思25天学习打卡营第17天|LLM原理和实践-基于MindSpore的GPT2文本摘要
Kirito_UGO的博客
07-05 438
使得开始训练的几个epoches或者一些steps内学习率较小,在预热的小学习率下，模型可以慢慢趋于稳定,等模型相对稳定后再选择预先设置的学习率进行训练,使得模型收敛速度变得更快，模型效果更佳。
【MindSpore学习打卡】应用实践-LLM原理和实践-基于MindSpore实现BERT对话情绪识别
weixin_43427267的博客
07-10 2075
BERT（Bidirectional Encoder Representations from Transformers）是一种基于Transformer的双向编码器表征模型。它主要通过两种预训练任务来捕捉词语和句子级别的表征：Masked Language Model（MLM）和Next Sentence Prediction（NSP）。：随机将语料库中15%的单词进行掩码操作，模型需要预测这些被掩码的单词。：模型需要预测两个句子之间是否存在顺序关系。
一文掌握文本语义分割：从朴素切分、Cross-Segment到阿里SeqModel
结构之法 算法之道
01-04 1万+
之所以写本文，源于以下两点详见，但该系统也有个可选项，可以选择达摩院开源的语义分割模型：nlp_bert_document-segmentation_chinese-base考虑到在RAG中，embedding和文档语义分割、段落分割都是绕不开的关键点，故本文重点梳理下各类典型的语义分割模型。
MLC-LLM框架的安卓应用部署实战
horsetaill的博客
03-17 3673
MLC-LLM框架的安卓应用实战，版本0.1.0
ChatGPT技术原理解析：从RL之PPO算法、RLHF到GPT4、instructGPT
热门推荐
结构之法 算法之道
01-15 19万+
本篇ChatGPT笔记会全力做到，通俗易懂且循序渐进(尽最大努力让每一个初学者哪怕是文科生都能没有障碍的读懂每一字一句、每一个概念、每一个公式) 一方面，对于想了解ChatGPT背后原理和如何发展而来的，逐一阐述从GPT/GPT2/GPT3到强化学习、PPO算法，最后再到instructGPT、ChatGPT、SeqGAN 且本文之前，99%的文章都不会把PPO算法从头推到尾，本文会把PPO从零推到尾，按照“RL-策略梯度-重要性采样(重要性权重)-TRPO(增加信任区域和KL散度约束)-PPO”的顺序逐步
欢迎学习面向开发者的大模型手册 - LLM Cookbook
叶庭云 成为自己的光
03-18 2451
欢迎学习面向开发者的大模型手册 - LLM Cookbook
[LLM]大模型训练DeepSpeed(一)-原理介绍
摩登都市天空---专栏
05-17 3万+
DeepSpeed是由Microsoft提供的分布式训练工具，旨在支持更大规模的模型和提供更多的优化策略和工具。与其他框架相比，DeepSpeed支持更大规模的模型和提供更多的优化策略和工具。其中，主要优势在于支持更大规模的模型、提供了更多的优化策略和工具（例如 ZeRO 和 Offload 等）
大模型推理框架Vllm和TensorRT-LLM在ChatGLM2-6B模型的推理速度对比
HUSTHY的博客
04-24 4590
这篇博客就专门对语言大模型推理框架Vllm和TensorRT-LLM在ChatGLM2-6B模型上的推理速度和效果进行一个对比。主要的内容分为三块，第一块简单介绍一下vllm和TensorRT-llM框架的特色和基本技术点，由于篇幅的原因关于技术的原理就不做多的介绍(每个技术点都可以拧出来写一篇文章，工作量非常大)；第二块内容就是介绍一下环境安装和重要的API，并且提供一个web推理服务；最后一块内容就是展示一下具体的推理案例，在推理速度和推理质量上进行对比
【LLM多模态】Qwen-VL模型架构和训练流程
发现问题，并解决问题，批判性思维
04-25 5519
Qwen-VL模型的整体网络架构包括以下三个组件： 大型语言模型（Large Language Model）： - 作用：作为Qwen-VL模型的基础组件，这个部分采用了一个大型语言模型，其初始权重来自于预训练的Qwen-7B模型。 - 来源：Qwen-VL的大型语言模型使用了来自Qwen-7B模型的预训练权重。 视觉编码器（Visual Encoder）： - 作用：视觉编码器采用了Vision Transformer（ViT）架构，用于处理输入图像并生成一组图像特征。在训练和推理过程中，将输入图像调
LLM - Qwen-72B LoRA 训练与推理实战
BITDDD小栈
02-05 6562
LLM Qwen-72B 简介与 LoRA 微调、推理。
大模型部署-使用Triton+TensorRT-LLM部署ChatGLM3-6B大模型-附项目源码+流程教程-优质项目实战
07-07
大模型部署_使用Triton+TensorRT-LLM部署ChatGLM3-6B大模型_附项目源码+流程教程_优质项目实战
算法部署-使用TensorRT-LLM部署通义千问Qwen-7B大模型-附详细优化+分析流程教程-优质大模型部署项目实战.zip
03-06
算法部署_使用TensorRT-LLM部署通义千问Qwen-7B大模型_附详细优化+分析流程教程_优质大模型部署项目实战
UEFI学习笔记（五）：EDK II PCD的概念、类型、使用
qq_44189622的博客
09-11 811
如果在BIOS里面有一些模块是binary方式集成进来的而这些binary又需要用到PCD（用于Binary Release），那么这些Binary集成的要用到的PCD就必须要设置为。PCD的值存在memory里面，下次启动时，上次更改的值丢失了，每次启动都是从default值开始。是存在VPD空间的（在FLASH上，只读），一般是出厂配置。如果platform是从源码build出来的，没有binary在里面的时候，PCD用的都是。作用域在一个模块中（模块级的），可以在Binary Level进行修改。
【学习笔记】手写Tomcat 二
LearnTech_123的博客
09-11 1046
在昨天的基础上，再进一步优化，能够响应HTML文件，图片等静态资源。
计算几何学习
网安小白
09-12 427
学习计算几何过程中对经典算法的记录
看Threejs好玩示例，学习创新与技术(二)
最新发布
htsitr2的专栏
09-14 227
它的主要特点是确保生成的点之间保持一定的最小距离，从而避免点的聚集。这种方法在计算机图形学、游戏开发、物理模拟等领域中非常有用，尤其是在需要生成自然分布的对象（如树木、石头等）时。如果不进行裁剪，那么效果如下（初步的锯齿感是因为矩形内采样的结果）。为啥这么复杂呢，为啥不开始存储每个方块的UV呢？下面代码的vTexCoords（=savedModelMatrix，已修改）记录方块所在的位置，vWorldPosition表示方块中像素的偏移。在上面那副图中，聪明的你一定初步猜到了每个方块的纹理是怎么得到的。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

muren

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值