CV党福音：YOLOv8实现实例分割（二）之训练过程

彭祥.

已于 2024-08-17 13:28:22 修改

阅读量1.2k

点赞数 26

分类专栏： YOLO系列文章标签： YOLO

于 2024-08-16 13:25:15 首次发布

本文链接：https://blog.csdn.net/pengxiang1998/article/details/141125655

版权

YOLO系列专栏收录该内容

40 篇文章 12 订阅

订阅专栏

在上一篇博客中，我们已经了解了YOLOv8实例分割的基本流程，本章则是对数据集、以及训练过程等进行进一步的学习。

文章目录

训练整体流程

训练模型的整体流程如下，可以看到其与预测时相比，多了损失计算等过程，那么，接下来我将从模型前向传播与损失计算两个过程来梳理训练过程。

在这里插入图片描述

语义分割与实例分割

与语义分割不同，实例分割一般是首先使用目标检测算法找到图像中的不同实例对象，然后对每个实例对象进行语义分割。
在YOLOv8中则是在原本检测头的基础上增加了分割头，两者共同作用。

总的来说，语义分割和实例分割在目标定义、输出结果和处理方式上有明显的区别。语义分割关注的是将图像划分为不同的语义区域，而实例分割则更进一步，不仅要进行语义分割，还要区分同一类别的不同实例。这种差异使得两者在计算机视觉任务中具有各自独特的应用场景。

在实际应用中，了解何时使用语义分割或实例分割是至关重要的。例如，在自动驾驶系统中，语义分割可能用于识别道路标志和行人，而实例分割则可能用于跟踪和区分不同的车辆。同样地，在医学影像分析中，语义分割可用于识别病变区域，而实例分割可用于精确区分不同的肿瘤实例。

数据集

在本次训练中，博主使用的是coco-8这个小数据集进行实验，该数据集内只有8张图像：

在这里插入图片描述

我们以第一张图像为例，其标签标注如下：

25 0.397235 0.750734 0.337422 0.750734 0.341996 0.749031 0.250021 0.700625 0.194802 0.691984 0.190229 0.688516 0.0751975 0.598625 0.059106 0.600359 0.0728898 0.596906 0.00852391 0.481078 0.02 0.465531 0.0130977 0.351437 0 0.351437 0.02 0.342813 0.0798129 0.228719 0.109709 0.223531 0.231601 0.138828 0.222391 0.123266 0.247734 0.130187 0.424823 0.095625 0.422516 0.0783125 0.434012 0.0904219 0.629522 0.109438 0.629522 0.0990625 0.636424 0.111172 0.797401 0.188938 0.806611 0.183781 0.797401 0.194141 0.900915 0.303031 0.914699 0.301297 0.900915 0.311688 0.94 0.429234 0.951518 0.429234 0.937734 0.437859 0.903222 0.555422 0.907838 0.569266 0.891726 0.560609 0.822703 0.631484 0.790499 0.512203 0.753721 0.498375 0.710021 0.484547 0.716923 0.399844 0.716923 0.34625 0.691622 0.313422 0.677817 0.308219 0.661726 0.268469 0.599626 0.251188 0.55131 0.246 0.509917 0.254656 0.477713 0.268469 0.431705 0.304766 0.422516 0.391188 0.459335 0.462063 0.482328 0.481078 0.482328 0.588266 0.463909 0.586547 0.468503 0.498375 0.404096 0.505281 0.378815 0.529484 0.355821 0.614203 0.397235 0.750734 0.459335 0.724828 0.480021 0.731734 0.477713 0.643578 0.470811 0.6505 0.459335 0.724828
0 0.717838 0.344813 0.723846 0.421563 0.738877 0.48025 0.783929 0.498313 0.822973 0.579578 0.925094 0.884313 0.985156 0.963313 0.994158 0.981375 0.898046 0.981375 0.83499 0.909141 0.783929 0.746609 0.780915 0.827875 0.804948 0.938484 0.825967 0.981375 0.6758 0.974609 0.45052 0.972344 0.45052 0.947516 0.52262 0.827875 0.465551 0.827875 0.435509 0.825625 0.420499 0.7895 0.402474 0.757906 0.348399 0.586344 0.372432 0.523141 0.423493 0.507344 0.477568 0.489281 0.435509 0.435109 0.402474 0.387703 0.432516 0.295141 0.465551 0.274828 0.492578 0.263547 0.53763 0.254516 0.612723 0.263547 0.645759 0.272578 0.678794 0.308688

按行划分目标，第一个数字为目标类别，后面每两个数值为一个坐标（即标注的关键点的坐标）

 类别  （x1,y1)  (x2,y2)  (x3,y3)   .....

第一张图像的检测与分割结果如下：

在这里插入图片描述

开启训练

开启训练代码如下：

from ultralytics import YOLO
if __name__=="__main__":
    # Load a model
    model = YOLO("yolov8n-seg.yaml").load("yolov8n-seg.pt")  # build from YAML and transfer weights
    # Train the model
    results = model.train(data="coco8-seg.yaml", epochs=100, imgsz=640)

训练结果如下：

在这里插入图片描述

从上一个博文中我们得知YOLOv8实例分割的结果中包含检测结果，因此，其损失函数共有三个，分别是目标分类损失、预测框回归损失以及mask分类损失。

由于YOLOv8需要完成多个模型的训练，因此其一层层的封装文件较多，我们需要慢慢的寻找。
其使用到的代码多在下面的文件夹中。

在这里插入图片描述

首先是跳转到train方法中，判断是否开启分布式训练，由于博主是在笔记本上进行的，故执行else中的方法：

self._do_train(world_size)

此时，在调用trian时，便已经完成了train的相关初始化了，传入的参数如下（参数太多，博主删了一部分）：

task=segment
mode=train
model=yolov8n-seg.yaml
data=D:\chat\programs\yolo\ultralytics\ultralytics\cfg\datasets\coco8-seg.yaml
epochs=100
batch=16
imgsz=640
save=True
workers=8
project=None
name=train15
exist_ok=False
pretrained=True
optimizer=auto
verbose=True
seed=0
deterministic=True
close_mosaic=10
resume=False
fraction=1.0
mask_ratio=4
dropout=0.0
val=True
split=val
save_json=False
save_hybrid=False
iou=0.7
max_det=300
vid_stride=1
stream_buffer=False
visualize=False
augment=False
agnostic_nms=False
classes=None
retina_masks=False
line_width=None
format=torchscript
keras=False
optimize=False
int8=False
dynamic=False
simplify=False
opset=None
workspace=4
nms=False
lr0=0.01
lrf=0.01
momentum=0.937
weight_decay=0.0005
warmup_epochs=3.0
warmup_momentum=0.8
warmup_bias_lr=0.1
box=7.5
cls=0.5
dfl=1.5
pose=12.0
kobj=1.0
label_smoothing=0.0
nbs=64
hsv_h=0.015
hsv_s=0.7
hsv_v=0.4
degrees=0.0
translate=0.1
scale=0.5
tracker=botsort.yaml
save_dir=runs\segment\train15

此时的world_size=1,随后进入do_train方法中，该方法位于\ultralytics\engine\trainer.py中，这也是最终执行训练的地方。

def _do_train(self, world_size=1):
        """Train completed, evaluate and plot if specified by arguments."""
        if world_size > 1:
            self._setup_ddp(world_size)
        self._setup_train(world_size)

        nb = len(self.train_loader)  # number of batches
        nw = max(round(self.args.warmup_epochs * nb), 100) if self.args.warmup_epochs > 0 else -1  # warmup iterations
        last_opt_step = -1
        self.epoch_time = None
        self.epoch_time_start = time.time()
        self.train_time_start = time.time()
        self.run_callbacks("on_train_start")
        LOGGER.info(
            f'Image sizes {self.args.imgsz} train, {self.args.imgsz} val\n'
            f'Using {self.train_loader.num_workers * (world_size or 1)} dataloader workers\n'
            f"Logging results to {colorstr('bold', self.save_dir)}\n"
            f'Starting training for ' + (f"{self.args.time} hours..." if self.args.time else f"{self.epochs} epochs...")
        )
        if self.args.close_mosaic:
            base_idx = (self.epochs - self.args.close_mosaic) * nb
            self.plot_idx.extend([base_idx, base_idx + 1, base_idx + 2])
        epoch = self.start_epoch
        self.optimizer.zero_grad()  # zero any resumed gradients to ensure stability on train start
        while True:
            self.epoch = epoch
            self.run_callbacks("on_train_epoch_start")
            with warnings.catch_warnings():
                warnings.simplefilter("ignore")  # suppress 'Detected lr_scheduler.step() before optimizer.step()'
                self.scheduler.step()

            self.model.train()
            if RANK != -1:
                self.train_loader.sampler.set_epoch(epoch)
            pbar = enumerate(self.train_loader)
            # Update dataloader attributes (optional)
            if epoch == (self.epochs - self.args.close_mosaic):
                self._close_dataloader_mosaic()
                self.train_loader.reset()

            if RANK in {-1, 0}:
                LOGGER.info(self.progress_string())
                pbar = TQDM(enumerate(self.train_loader), total=nb)
            self.tloss = None
            for i, batch in pbar:
                self.run_callbacks("on_train_batch_start")
                # Warmup
                ni = i + nb * epoch
                if ni <= nw:
                    xi = [0, nw]  # x interp
                    self.accumulate = max(1, int(np.interp(ni, xi, [1, self.args.nbs / self.batch_size]).round()))
                    for j, x in enumerate(self.optimizer.param_groups):
                        # Bias lr falls from 0.1 to lr0, all other lrs rise from 0.0 to lr0
                        x["lr"] = np.interp(
                            ni, xi, [self.args.warmup_bias_lr if j == 0 else 0.0, x["initial_lr"] * self.lf(epoch)]
                        )
                        if "momentum" in x:
                            x["momentum"] = np.interp(ni, xi, [self.args.warmup_momentum, self.args.momentum])

                # Forward
                with autocast(self.amp):
                    batch = self.preprocess_batch(batch)
                    self.loss, self.loss_items = self.model(batch)
                    if RANK != -1:
                        self.loss *= world_size
                    self.tloss = (
                        (self.tloss * i + self.loss_items) / (i + 1) if self.tloss is not None else self.loss_items
                    )

                # Backward
                self.scaler.scale(self.loss).backward()

                # Optimize - https://pytorch.org/docs/master/notes/amp_examples.html
                if ni - last_opt_step >= self.accumulate:
                    self.optimizer_step()
                    last_opt_step = ni

                    # Timed stopping
                    if self.args.time:
                        self.stop = (time.time() - self.train_time_start) > (self.args.time * 3600)
                        if RANK != -1:  # if DDP training
                            broadcast_list = [self.stop if RANK == 0 else None]
                            dist.broadcast_object_list(broadcast_list, 0)  # broadcast 'stop' to all ranks
                            self.stop = broadcast_list[0]
                        if self.stop:  # training time exceeded
                            break

                # Log
                mem = f"{torch.cuda.memory_reserved() / 1E9 if torch.cuda.is_available() else 0:.3g}G"  # (GB)
                loss_len = self.tloss.shape[0] if len(self.tloss.shape) else 1
                losses = self.tloss if loss_len > 1 else torch.unsqueeze(self.tloss, 0)
                if RANK in {-1, 0}:
                    pbar.set_description(
                        ("%11s" * 2 + "%11.4g" * (2 + loss_len))
                        % (f"{epoch + 1}/{self.epochs}", mem, *losses, batch["cls"].shape[0], batch["img"].shape[-1])
                    )
                    self.run_callbacks("on_batch_end")
                    if self.args.plots and ni in self.plot_idx:
                        self.plot_training_samples(batch, ni)

                self.run_callbacks("on_train_batch_end")

            self.lr = {f"lr/pg{ir}": x["lr"] for ir, x in enumerate(self.optimizer.param_groups)}  # for loggers
            self.run_callbacks("on_train_epoch_end")
            if RANK in {-1, 0}:
                final_epoch = epoch + 1 >= self.epochs
                self.ema.update_attr(self.model, include=["yaml", "nc", "args", "names", "stride", "class_weights"])

                # Validation
                if self.args.val or final_epoch or self.stopper.possible_stop or self.stop:
                    self.metrics, self.fitness = self.validate()
                self.save_metrics(metrics={**self.label_loss_items(self.tloss), **self.metrics, **self.lr})
                self.stop |= self.stopper(epoch + 1, self.fitness) or final_epoch
                if self.args.time:
                    self.stop |= (time.time() - self.train_time_start) > (self.args.time * 3600)

                # Save model
                if self.args.save or final_epoch:
                    self.save_model()
                    self.run_callbacks("on_model_save")

            # Scheduler
            t = time.time()
            self.epoch_time = t - self.epoch_time_start
            self.epoch_time_start = t
            if self.args.time:
                mean_epoch_time = (t - self.train_time_start) / (epoch - self.start_epoch + 1)
                self.epochs = self.args.epochs = math.ceil(self.args.time * 3600 / mean_epoch_time)
                self._setup_scheduler()
                self.scheduler.last_epoch = self.epoch  # do not move
                self.stop |= epoch >= self.epochs  # stop if exceeded epochs
            self.run_callbacks("on_fit_epoch_end")
            gc.collect()
            torch.cuda.empty_cache()  # clear GPU memory at end of epoch, may help reduce CUDA out of memory errors

            # Early Stopping
            if RANK != -1:  # if DDP training
                broadcast_list = [self.stop if RANK == 0 else None]
                dist.broadcast_object_list(broadcast_list, 0)  # broadcast 'stop' to all ranks
                self.stop = broadcast_list[0]
            if self.stop:
                break  # must break all DDP ranks
            epoch += 1

        if RANK in {-1, 0}:
            # Do final val with best.pt
            LOGGER.info(
                f"\n{epoch - self.start_epoch + 1} epochs completed in "
                f"{(time.time() - self.train_time_start) / 3600:.3f} hours."
            )
            self.final_eval()
            if self.args.plots:
                self.plot_metrics()
            self.run_callbacks("on_train_end")
        gc.collect()
        torch.cuda.empty_cache()
        self.run_callbacks("teardown")

训练Debug

那么，下面我们通过DeBug来梳理一下模型的训练过程吧

首先，我们明确一下损失函数有以下几个：其中box与dfl都是用于边界框回归的，seg则是分割损失，用于训练mask的，而cls则是类别损失

('box_loss', 'seg_loss', 'cls_loss', 'dfl_loss')

开启训练模式，需要注意的是，我们虽然设计batch=16，但训练集中只有4张图像，因此batch=4。

self.model.train()

数据封装

通过TQDM将训练器封装为进度条的形式，并将其输入到模型中进行前向传播

pbar = TQDM(enumerate(self.train_loader), total=nb)
for i, batch in pbar:
	batch = self.preprocess_batch(batch)#前处理(这个前处理没有啥操作，直接返回batch)
	self.loss, self.loss_items = self.model(batch)#前向传播

前处理操作，其实啥也没有操作。

 def preprocess_batch(self, batch):
        return batch

在这里插入图片描述
我们可以看到传入到模型中的batch数据如下，其直接计算了损失：

self.loss, self.loss_items = self.model(batch)

损失函数初始化

这里博主刚开始看到时并不明白，这个model该去哪里DeBug，因为SegmentationModel中并没有定义forward方法，后来想到model本质是SegmentationModel，而SegmentationModel又是继承于DetectionModel，而DetectionModel则继承于BaseModel，最终在BaseModel（\ultralytics\nn\tasks.py））中找到了对应的前向传播方法：

代码如下，其判断传入的x是否是dict类型，如果是dict类型，即说明是训练集与验证集的数据（因为其包含图像之外的其他信息，如标注信息等），否则的话，x应该只有一个数据，即图像，则执行预测。

class BaseModel(nn.Module):
    def forward(self, x, *args, **kwargs):
        if isinstance(x, dict):  # for cases of training and validating while training.
            return self.loss(x, *args, **kwargs)
        return self.predict(x, *args, **kwargs)

x即为刚刚传入的batch，类型为dict，故计算损失，self.loss方法如下，即首先会创建好对应的损失函数，随后将batch中的img进行前向传播，得到预测值，再将该预测值与batch(内含标注信息）一起计算损失值。

def loss(self, batch, preds=None):
        if getattr(self, "criterion", None) is None:#判断是否有criterion属性，没有的话创建
            self.criterion = self.init_criterion()

        preds = self.forward(batch["img"]) if preds is None else preds
        return self.criterion(preds, batch)

由于这是BaseModel，而不同的模型的损失函数是不一样的，因此每个子类都重写了对应的损失函数构造方法：
BaseModel的init_criterion方法，其提示要计算损失需要实现对应的任务头（任务类型）

def init_criterion(self):
        """Initialize the loss criterion for the BaseModel."""
        raise NotImplementedError("compute_loss() needs to be implemented by task heads")

重写在SegmentationModel中的init_criterion方法

def init_criterion(self):
        """Initialize the loss criterion for the SegmentationModel."""
        return v8SegmentationLoss(self)

我们看下v8SegmentationLoss方法，其也是继承了v8DetectionLoss。

class v8SegmentationLoss(v8DetectionLoss):
    """Criterion class for computing training losses."""
    def __init__(self, model):  # model must be de-paralleled
        """Initializes the v8SegmentationLoss class, taking a de-paralleled model as argument."""
        super().__init__(model)
        self.overlap = model.args.overlap_mask

在这里插入图片描述

随后便完成了损失函数的初始化，接下来便是前向传播过程：

前向传播

前向传播依旧是调用的BaseModel的forward方法，只不过，此时传入的数据是batch['img']，不再是dict类型，所有其执行的是预测操作：

 def forward(self, x, *args, **kwargs):
        if isinstance(x, dict):  # for cases of training and validating while training.
            return self.loss(x, *args, **kwargs)
        return self.predict(x, *args, **kwargs)

而predict方法定义如下：

    def predict(self, x, profile=False, visualize=False, augment=False, embed=None):
        if augment:#不执行
            return self._predict_augment(x)
        return self._predict_once(x, profile, visualize, embed)

接下来这段代码则是重中之重，其通过读取模型结构，完成了前向传播过程：

def _predict_once(self, x, profile=False, visualize=False, embed=None):
        y, dt, embeddings = [], [], []  # outputs
        for m in self.model:
            if m.f != -1:  # if not from previous layer
                x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f]  # from earlier layers
            if profile:
                self._profile_one_layer(m, x, dt)
            x = m(x)  # run
            y.append(x if m.i in self.save else None)  # save output
            if visualize:
                feature_visualization(x, m.type, m.i, save_dir=visualize)
            if embed and m.i in embed:
                embeddings.append(nn.functional.adaptive_avg_pool2d(x, (1, 1)).squeeze(-1).squeeze(-1))  # flatten
                if m.i == max(embed):
                    return torch.unbind(torch.cat(embeddings, 1), dim=0)
        return x

我们可以看到model的结构如下：

Sequential(
  (0): Conv(
    (conv): Conv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(16, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (1): Conv(
    (conv): Conv2d(16, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (2): C2f(
    (cv1): Conv(
      (conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(48, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(16, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(16, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (3): Conv(
    (conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (4): C2f(
    (cv1): Conv(
      (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0-1): 2 x Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (5): Conv(
    (conv): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (6): C2f(
    (cv1): Conv(
      (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0-1): 2 x Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (7): Conv(
    (conv): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (8): C2f(
    (cv1): Conv(
      (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(384, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (9): SPPF(
    (cv1): Conv(
      (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): MaxPool2d(kernel_size=5, stride=1, padding=2, dilation=1, ceil_mode=False)
  )
  (10): Upsample(scale_factor=2.0, mode='nearest')
  (11): Concat()
  (12): C2f(
    (cv1): Conv(
      (conv): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (13): Upsample(scale_factor=2.0, mode='nearest')
  (14): Concat()
  (15): C2f(
    (cv1): Conv(
      (conv): Conv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(96, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (16): Conv(
    (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (17): Concat()
  (18): C2f(
    (cv1): Conv(
      (conv): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (19): Conv(
    (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
    (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
    (act): SiLU(inplace=True)
  )
  (20): Concat()
  (21): C2f(
    (cv1): Conv(
      (conv): Conv2d(384, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (cv2): Conv(
      (conv): Conv2d(384, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
      (act): SiLU(inplace=True)
    )
    (m): ModuleList(
      (0): Bottleneck(
        (cv1): Conv(
          (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (cv2): Conv(
          (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
      )
    )
  )
  (22): Segment(
    (cv2): ModuleList(
      (0): Sequential(
        (0): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
      )
      (1): Sequential(
        (0): Conv(
          (conv): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
      )
      (2): Sequential(
        (0): Conv(
          (conv): Conv2d(256, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (cv3): ModuleList(
      (0): Sequential(
        (0): Conv(
          (conv): Conv2d(64, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(80, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(80, 80, kernel_size=(1, 1), stride=(1, 1))
      )
      (1): Sequential(
        (0): Conv(
          (conv): Conv2d(128, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(80, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(80, 80, kernel_size=(1, 1), stride=(1, 1))
      )
      (2): Sequential(
        (0): Conv(
          (conv): Conv2d(256, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(80, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(80, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(80, 80, kernel_size=(1, 1), stride=(1, 1))
      )
    )
    (dfl): DFL(
      (conv): Conv2d(16, 1, kernel_size=(1, 1), stride=(1, 1), bias=False)
    )
    (proto): Proto(
      (cv1): Conv(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
        (act): SiLU(inplace=True)
      )
      (upsample): ConvTranspose2d(64, 64, kernel_size=(2, 2), stride=(2, 2))
      (cv2): Conv(
        (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
        (act): SiLU(inplace=True)
      )
      (cv3): Conv(
        (conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
        (act): SiLU(inplace=True)
      )
    )
    (cv4): ModuleList(
      (0): Sequential(
        (0): Conv(
          (conv): Conv2d(64, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1))
      )
      (1): Sequential(
        (0): Conv(
          (conv): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1))
      )
      (2): Sequential(
        (0): Conv(
          (conv): Conv2d(256, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (1): Conv(
          (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
          (act): SiLU(inplace=True)
        )
        (2): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1))
      )
    )
  )
)

在这里，通过将model中的每个模块都遍历处理，从而获得每个模块的输出结果。

for m in self.model:
            if m.f != -1:  # if not from previous layer
                x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f]  # from earlier layers
            if profile:
                self._profile_one_layer(m, x, dt)
            x = m(x)  # run
            y.append(x if m.i in self.save else None)  # save output
            if visualize:
                feature_visualization(x, m.type, m.i, save_dir=visualize)
            if embed and m.i in embed:
                embeddings.append(nn.functional.adaptive_avg_pool2d(x, (1, 1)).squeeze(-1).squeeze(-1))  # flatten
                if m.i == max(embed):
                    return torch.unbind(torch.cat(embeddings, 1), dim=0)

如第一次遍历读取的 m（模块）结构如下，x 的值为torch.Size([4, 3, 640, 640])

Conv(
  (conv): Conv2d(3, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (bn): BatchNorm2d(16, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
  (act): SiLU(inplace=True)
)

输入m 结构后得到的结果为torch.Size([4, 16, 320, 320])

x = m(x)  # run
y.append(x if m.i in self.save else None)  # save output

同时还会判断哪些层输出的数据是需要我们保存的，因为有些模块的输入值并不是来源于上一层，可能需要其他层的结果
这里需要保存的输出层结果self.save为[4, 6, 9, 12, 15, 18, 21]
随后就这样依次执行，最终就遍历了整个模型，也就得到了最终的输出结果。

这里的144=80+64，其中80是类别，64为DEL模块的输出结果，用于生成预测框，这块不理解的可以参考我上一篇博客：

YOLOv8实现实例分割（一）

在这里插入图片描述
得到的预测结果与batch进行损失计算：

return self.criterion(preds, batch)

总结

在这篇博文中，我们完成了训练过程中的数据封装、损失函数定义，模型构建以及前向传播的学习，接下来便是将预测值与真值进行损失计算了，尽请期待。

彭祥.

关注

26
点赞
踩
14

收藏

觉得还不错? 一键收藏
打赏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫

专栏目录