目录
准备:先从GitHub - matterport/Mask_RCNN: Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow下载Mask_RCNN。
注意:一定要注意环境问题！!如果你是tensorflow-gpu2.x的很多函数都需要改，因为tf2中已经废除或者直接删除了一些函数，你需要用tf2中的新方法替代掉原来Mask_RCNN的tf1老方法，Mask_rcnn是tf1.x版本的，当然最好用tf1.x版运行，tf2.x的话得改不少......这里我改的过程就不说了(挺费劲的).....下面给的训练代码和预测代码为了尽量的通用性给的都是1.x版本的。
1.安装依赖
首先我们需要安装所需依赖，我们的依赖是全部写在了requirement文件里，可直接通过下面的命令Pip。
pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txtrequirements文件内容是这样的，这包含了我们所需要的依赖包。
（注意这里的tensorflow以及keras可以改成你那个版本，也可以直接去掉)
2.从项目目录下运行安装程序
python setup.py install3.下载预处理好的COCO权重模型，
可以去github也可以去csdn下载，总共二百多M，github比较慢
github: https://github.com/matterport/Mask_RCNN/releases
csdn:https://download.csdn.net/download/wanghuiqiang1/149653404.安装数据标记工具labelme
1.打开Anaconda Prompt,连接清华镜像:
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --set show_channel_urls yes
2.输入
conda create --name=labelme python=3.7
激活lableme命令
activate labelme
安装pyqt5
pip install pyqt5 -i https://pypi.mirrors.ustc.edu.cn/simple
安装lableme
pip install labelme -i https://pypi.mirrors.ustc.edu.cn/simple
（建议将labelme的启动写成一个bat脚本
这样免得每次启动labelme都要conda activate相应环境，激活，比较麻烦，举个例子，bat脚本内容可以如下：
@echo.
@echo ***进入conda命令行，并直接切换到labelme环境***
@echo.
@echo.
@echo.
CALL C:\ProgramData\Anaconda3\Scripts\activate.bat C:\ProgramData\Anaconda3\envs\labelme
@echo.
@echo.
@echo.
@echo ***启动labelme***
@echo.
@echo.
labelme.exe
）
这样就可以每次双击bat就可以启动Lableme了。5.数据集准备
python爬取百度图片(我这里爬的是送你一朵小红花剧照)
# -*- coding: utf-8 -*-
import re
import requests
from urllib import error
from bs4 import BeautifulSoup
import os
num = 0
numPicture = 0
file = ''
List = []
def Find(url, A):
global List
print('正在检测图片总数，请稍等.....')
t = 0
i = 1
s = 0
while t < 1000:
Url = url + str(t)
try:
# 这里搞了下
Result = A.get(Url, timeout=7, allow_redirects=False)
except BaseException:
t = t + 60
continue
else:
result = Result.text
pic_url = re.findall('"objURL":"(.*?)",', result, re.S) # 先利用正则表达式找到图片url
s += len(pic_url)
if len(pic_url) == 0:
break
else:
List.append(pic_url)
t = t + 60
return s
def recommend(url):
Re = []
try:
html = requests.get(url, allow_redirects=False)
except error.HTTPError as e:
return
else:
html.encoding = 'utf-8'
bsObj = BeautifulSoup(html.text, 'html.parser')
div = bsObj.find('div', id='topRS')
if div is not None:
listA = div.findAll('a')
for i in listA:
if i is not None:
Re.append(i.get_text())
return Re
def dowmloadPicture(html, keyword):
global num
# t =0
pic_url = re.findall('"objURL":"(.*?)",', html, re.S) # 先利用正则表达式找到图片url
print('找到关键词:' + keyword + '的图片，即将开始下载图片...')
for each in pic_url:
print('正在下载第' + str(num + 1) + '张图片，图片地址:' + str(each))
try:
if each is not None:
pic = requests.get(each, timeout=7)
else:
continue
except BaseException:
print('错误，当前图片无法下载')
continue
else:
string = file + r'\\' + keyword + '_' + str(num) + '.jpg'
fp = open(string, 'wb')
fp.write(pic.content)
fp.close()
num += 1
if num >= numPicture:
return
if __name__ == '__main__': # 主函数入口
##############################
# 这里加了点
headers = {
'Accept-Language': 'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2',
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Upgrade-Insecure-Requests': '1'
}
A = requests.Session()
A.headers = headers
###############################
word = input("请输入搜索关键词(可以是人名，地名等): ")
# add = 'http://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word=%E5%BC%A0%E5%A4%A9%E7%88%B1&pn=120'
url = 'https://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word=' + word + '&pn='
# 这里搞了下
tot = Find(url, A)
Recommend = recommend(url) # 记录相关推荐
print('经过检测%s类图片共有%d张' % (word, tot))
numPicture = int(input('请输入想要下载的图片数量 '))
file = input('请建立一个存储图片的文件夹，输入文件夹名称即可')
y = os.path.exists(file)
if y == 1:
print('该文件已存在，请重新输入')
file = input('请建立一个存储图片的文件夹，)输入文件夹名称即可')
os.mkdir(file)
else:
os.mkdir(file)
t = 0
tmp = url
while t < numPicture:
try:
url = tmp + str(t)
# 这里搞了下
result = A.get(url, timeout=10, allow_redirects=False)
except error.HTTPError as e:
print('网络错误，请调整网络后重试')
t = t + 60
else:
dowmloadPicture(result.text, word)
t = t + 60
print('当前搜索结束，感谢使用')
print('猜你喜欢')
for re in Recommend:
print(re, end=' ')6.做数据标记
使用数据标记工具，直接打开之前做好的脚本或者conda activate相应环境，激活，labelme也可以。在pycharm中可以直接labelme。
这个就是苦力活了.......
7. 转换标记数据
单个转换:labelme_json_to_dataset xxxx.json(替换成你生成的json文件名)
批量转换的代码:
import os
path = r'C:\Users\apple\Desktop\labelme\json' # path是你存放json的路径
json_file = os.listdir(path)
for file in json_file:
os.system("labelme_json_to_dataset %s" % (path + '/' + file))
会生成这堆东西，一定要注意一个都别少，特别是那个info.yml。
8.整理数据文件
建立四个这种空文件夹
注意:
放原来用lambel转换后的json文件，就是这些
labelme_json文件夹放用json文件批量转换后生成的东西，就是这些
pic文件夹放原图
最后要注意的就是我们需要把原来生成的那五个文件中的label.png依次改名，改成这样
，然后移动到cv2_mask，当然这里你自己一个个手动改名然后复制是比较麻烦的，可以写个python脚本实现，这里提供一个python脚本。
import os
import shutil
filename = "F:/deeplearning/mask_image/json/" # 存放json转化得到的文件夹名称，需要保证没有隐藏的文件夹
fileList = os.listdir(filename)
"""
抽取json转化得到的5个文件中的label.png，并重新命名
"""
for i in range(len(fileList)):
path = filename + fileList[i];
# 如果不是文件夹，跳过
if os.path.isfile(path):
continue
no = fileList[i][:-5] # 从文件夹上取出文件名
mask_source = path + "/label.png"
mask_target = "F:/deeplearning/mask_image/cv2_mask/{}.png".format(no) # 命名为“文件名.png”
shutil.copy(mask_source, mask_target) # 利用shutil直接copy过去
最后要确保的是这四个文件夹中的命名要注意的就是命名格式要统一，除了labelme_json里面的文件夹命名是自定义命名+编号_json ，其他的文件名都是自定义命名+编号.文件类型。
9.创建训练文件
然后我们需要在项目的根目录下创建用于训练的python文件（我定好的文件名是mask_train.py），代码是
# -*- coding: utf-8 -*-
import os
import sys
import random
import math
import re
import time
import numpy as np
import cv2
import matplotlib
import matplotlib.pyplot as plt
import tensorflow as tf
from mrcnn.config import Config
# import utils
from mrcnn import model as modellib, utils
from mrcnn import visualize
import yaml
from mrcnn.model import log
from PIL import Image
# os.environ["CUDA_VISIBLE_DEVICES"] = "0"
# Root directory of the project
ROOT_DIR = os.getcwd()
# ROOT_DIR = os.path.abspath("../")
# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
iter_num = 0
# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
utils.download_trained_weights(COCO_MODEL_PATH)
class ShapesConfig(Config):
"""Configuration for training on the toy shapes dataset.
Derives from the base Config class and overrides values specific
to the toy shapes dataset.
"""
# Give the configuration a recognizable name
NAME = "shapes"
# Train on 1 GPU and 8 images per GPU. We can put multiple images on each
# GPU because the images are small. Batch size is 8 (GPUs * images/GPU).
GPU_COUNT = 1
IMAGES_PER_GPU = 1
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # background + 1 shapes
# Use small images for faster training. Set the limits of the small side
# the large side, and that determines the image shape.
IMAGE_MIN_DIM = 320
IMAGE_MAX_DIM = 384
# Use smaller anchors because our image and objects are small
RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6) # anchor side in pixels
# Reduce training ROIs per image because the images are small and have
# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
TRAIN_ROIS_PER_IMAGE = 100
# Use a small epoch since the data is simple
STEPS_PER_EPOCH = 100
# use small validation steps since the epoch is small
VALIDATION_STEPS = 50
config = ShapesConfig()
config.display()
class DrugDataset(utils.Dataset):
# 得到该图中有多少个实例（物体）
def get_obj_index(self, image):
n = np.max(image)
return n
# 解析labelme中得到的yaml文件，从而得到mask每一层对应的实例标签
def from_yaml_get_class(self, image_id):
info = self.image_info[image_id]
with open(info['yaml_path']) as f:
temp = yaml.load(f.read())
labels = temp['label_names']
del labels[0]
return labels
# 重新写draw_mask
def draw_mask(self, num_obj, mask, image, image_id):
# print("draw_mask-->",image_id)
# print("self.image_info",self.image_info)
info = self.image_info[image_id]
# print("info-->",info)
# print("info[width]----->",info['width'],"-info[height]--->",info['height'])
for index in range(num_obj):
for i in range(info['width']):
for j in range(info['height']):
# print("image_id-->",image_id,"-i--->",i,"-j--->",j)
# print("info[width]----->",info['width'],"-info[height]--->",info['height'])
at_pixel = image.getpixel((i, j))
if at_pixel == index + 1:
mask[j, i, index] = 1
return mask
# 重新写load_shapes，里面包含自己的自己的类别
# 并在self.image_info信息中添加了path、mask_path 、yaml_path
# yaml_pathdataset_root_path = "/tongue_dateset/"
# img_floder = dataset_root_path + "rgb"
# mask_floder = dataset_root_path + "mask"
# dataset_root_path = "/tongue_dateset/"
def load_shapes(self, count, img_floder, mask_floder, imglist, dataset_root_path):
"""Generate the requested number of synthetic images.
count: number of images to generate.
height, width: the size of the generated images.
"""
# Add classes
self.add_class("shapes", 1, "person")
for i in range(count):
# 获取图片宽和高
print(i)
filestr = imglist[i].split(".")[0]
# print(imglist[i],"-->",cv_img.shape[1],"--->",cv_img.shape[0])
# print("id-->", i, " imglist[", i, "]-->", imglist[i],"filestr-->",filestr)
# filestr = filestr.split("_")[1]
mask_path = mask_floder + "/" + filestr + ".png"
yaml_path = dataset_root_path + "labelme_json/" + filestr + "_json/info.yaml"
print(dataset_root_path + "labelme_json/" + filestr + "_json/img.png")
cv_img = cv2.imread(dataset_root_path + "labelme_json/" + filestr + "_json/img.png")
self.add_image("shapes", image_id=i, path=img_floder + "/" + imglist[i],
width=cv_img.shape[1], height=cv_img.shape[0], mask_path=mask_path, yaml_path=yaml_path)
# 重写load_mask
def load_mask(self, image_id):
"""Generate instance masks for shapes of the given image ID.
"""
global iter_num
print("image_id", image_id)
info = self.image_info[image_id]
count = 1 # number of object
img = Image.open(info['mask_path'])
num_obj = self.get_obj_index(img)
mask = np.zeros([info['height'], info['width'], num_obj], dtype=np.uint8)
mask = self.draw_mask(num_obj, mask, img, image_id)
occlusion = np.logical_not(mask[:, :, -1]).astype(np.uint8)
for i in range(count - 2, -1, -1):
mask[:, :, i] = mask[:, :, i] * occlusion
occlusion = np.logical_and(occlusion, np.logical_not(mask[:, :, i]))
labels = []
labels = self.from_yaml_get_class(image_id)
labels_form = []
for i in range(len(labels)):
if labels[i].find("person") != -1:
# print "car"
labels_form.append("person")
elif labels[i].find("leg") != -1:
# print "leg"
labels_form.append("leg")
elif labels[i].find("well") != -1:
# print "well"
labels_form.append("well")
class_ids = np.array([self.class_names.index(s) for s in labels_form])
return mask, class_ids.astype(np.int32)
def get_ax(rows=1, cols=1, size=8):
"""Return a Matplotlib Axes array to be used in
all visualizations in the notebook. Provide a
central point to control graph sizes.
Change the default size attribute to control the size
of rendered images
"""
_, ax = plt.subplots(rows, cols, figsize=(size * cols, size * rows))
return ax
# 基础设置
dataset_root_path = "samples/trinmy/myinfo/"
img_floder = dataset_root_path + "pic"
mask_floder = dataset_root_path + "cv2_mask"
# yaml_floder = dataset_root_path
imglist = os.listdir(img_floder)
count = len(imglist)
# train与val数据集准备
dataset_train = DrugDataset()
dataset_train.load_shapes(count, img_floder, mask_floder, imglist, dataset_root_path)
dataset_train.prepare()
# print("dataset_train-->",dataset_train._image_ids)
dataset_val = DrugDataset()
dataset_val.load_shapes(count, img_floder, mask_floder, imglist, dataset_root_path)
dataset_val.prepare()
# print("dataset_val-->",dataset_val._image_ids)
# Load and display random samples
# image_ids = np.random.choice(dataset_train.image_ids, 4)
# for image_id in image_ids:
# image = dataset_train.load_image(image_id)
# mask, class_ids = dataset_train.load_mask(image_id)
# visualize.display_top_masks(image, mask, class_ids, dataset_train.class_names)
# Create model in training mode
model = modellib.MaskRCNN(mode="training", config=config,
model_dir=MODEL_DIR)
# Which weights to start with?
init_with = "coco" # imagenet, coco, or last
if init_with == "imagenet":
model.load_weights(model.get_imagenet_weights(), by_name=True)
elif init_with == "coco":
# Load weights trained on MS COCO, but skip layers that
# are different due to the different number of classes
# See README for instructions to download the COCO weights
# print(COCO_MODEL_PATH)
model.load_weights(COCO_MODEL_PATH, by_name=True,
exclude=["mrcnn_class_logits", "mrcnn_bbox_fc",
"mrcnn_bbox", "mrcnn_mask"])
elif init_with == "last":
# Load the last model you trained and continue training
model.load_weights(model.find_last()[1], by_name=True)
# Train the head branches
# Passing layers="heads" freezes all layers except the head
# layers. You can also pass a regular expression to select
# which layers to train by name pattern.
model.train(dataset_train, dataset_val,
learning_rate=config.LEARNING_RATE,
epochs=10,
layers='heads')
# Fine tune all layers
# Passing layers="all" trains all layers. You can also
# pass a regular expression to select which layers to
# train by name pattern.
model.train(dataset_train, dataset_val,
learning_rate=config.LEARNING_RATE / 10,
epochs=10,
layers="all")1.注意的是要在代码的33行改成我们之前下载好的预训练模型所在路径(一般放在项目根目录下)。比如这是我的
2.要注意的是第53行中修改我们的类别数，这个类别数是1(背景有的类别)+X(你的类别)
3.在第186行代码中我们需要改成我们之前制作数据源的路径(那个路径下有之前准备好的四个文件夹)
4.在代码122行中改一下相应的类别，比如我的是三个类别
在代码160-169中也改成相应的类别，比如我的是三个类别，照着我这个相应改就可
其他的参数有需要的话可以在45-71行修改
10.运行训练文件
python mask_train.py
成功运行的话我们应该是能看到这种的界面
11.预测模型
因为之前的前面的默认设置是每训练100步保存一次模型，我们可以到项目根目录的logs下寻找我们训练好的模型文件。
找最下面的文件夹，我们刚训练好的模型文件就在那。
然后这些就是我们生成好的模型文件了(因为我训练了400步，所以有四个模型文件，当然最下面的那个最准了）
然后我们就可以在项目根目录下创建mask_test.py。代码是
# -*- coding: utf-8 -*-
import os
import sys
import random
import math
import numpy as np
import skimage.io
import matplotlib
import matplotlib.pyplot as plt
import cv2
import time
from mrcnn.config import Config
from datetime import datetime
# Root directory of the project
ROOT_DIR = os.getcwd()
# Import Mask RCNN
sys.path.append(ROOT_DIR) # To find local version of the library
from mrcnn import utils
import mrcnn.model as modellib
from mrcnn import visualize
# Import COCO config
# sys.path.append(os.path.join(ROOT_DIR, "samples/coco/")) # To find local version
# from samples.coco import coco
# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(MODEL_DIR ,"mask_rcnn_shapes_0010.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
utils.download_trained_weights(COCO_MODEL_PATH)
print("cuiwei***********************")
# Directory of images to run detection on
IMAGE_DIR = os.path.join(ROOT_DIR, "images")
class ShapesConfig(Config):
"""Configuration for training on the toy shapes dataset.
Derives from the base Config class and overrides values specific
to the toy shapes dataset.
"""
# Give the configuration a recognizable name
NAME = "shapes"
# Train on 1 GPU and 8 images per GPU. We can put multiple images on each
# GPU because the images are small. Batch size is 8 (GPUs * images/GPU).
GPU_COUNT = 1
IMAGES_PER_GPU = 1
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # background + 3 shapes
# Use small images for faster training. Set the limits of the small side
# the large side, and that determines the image shape.
IMAGE_MIN_DIM = 320
IMAGE_MAX_DIM = 384
# Use smaller anchors because our image and objects are small
RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6) # anchor side in pixels
# Reduce training ROIs per image because the images are small and have
# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
TRAIN_ROIS_PER_IMAGE =100
# Use a small epoch since the data is simple
STEPS_PER_EPOCH = 100
# use small validation steps since the epoch is small
VALIDATION_STEPS = 50
#import train_tongue
#class InferenceConfig(coco.CocoConfig):
class InferenceConfig(ShapesConfig):
# Set batch size to 1 since we'll be running inference on
# one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
GPU_COUNT = 1
IMAGES_PER_GPU = 1
config = InferenceConfig()
model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
# Create model object in inference mode.
model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
# Load weights trained on MS-COCO
model.load_weights(COCO_MODEL_PATH, by_name=True)
# COCO Class names
# Index of the class in the list is its ID. For example, to get ID of
# the teddy bear class, use: class_names.index('teddy bear')
class_names = ['BG', 'person']
# Load a random image from the images folder
file_names = next(os.walk(IMAGE_DIR))[2]
image = skimage.io.imread("./images/5951960966_d4e1cda5d0_z.jpg")
a=datetime.now()
# Run detection
results = model.detect([image], verbose=1)
b=datetime.now()
# Visualize results
print("shijian",(b-a).seconds)
r = results[0]
visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'],
class_names, r['scores'])
在运行之前我们需要把代码的31行，修改模型路径，把路径替换成训练好的模型路径。
比如我的是这个
第二个需要修改的就是99行中要改成你需要测试图片的路径
第三个的话就是我们需要在107行中将class_names修改成我们的类名
（注意BG不要动，其他改成你增加的类名)
这样就OK了。
12.结果显示
模型其实还是不够准确，是因为我们训练的时间没足够长（其实也跟我用lambelme数据标注的时候没好好标注的原因有关.....）......不过多训练会就行了。
1929
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
