2022年用yolov3在darknet上训练与测试(on ubuntu18)

enviroment:
ubuntu 18.04.6
python 3.6
yolov3
darknet
cuda11.6
nvidia driver510
important sentenses:
export PATH=$PATH:/home/aaa/.local/bin
./darknet detector train ./../my.data  cfg/yolov3-tiny.cfg

//gpu driver version
cat /proc/driver/nvidia/version 
or
nvidia-smi
mine is 510

export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
source ~/.bashrc
nvcc -V

//==============here are steps=================
1.install ubuntu18.04
swap 8g
/home 160g
/ 30g
maybe /boot is needed if needed give it 500m
maybe others needed

if and error like ...CD shows before install
need to remove a file

if block while installing like 'preparing linux hwe 5.4'
try disconnect net and wait minutes and reconnect net

2.install darknet
git clone https://github.com/AlexeyAB/darknet
or? git clone https://github.com/pjreddie/darknet.git
cd darknet
make (if make is not installed need to sudo apt-get install make)

3.we can try it now
wget https://pjreddie.com/media/files/yolov3-tiny.weights
./darknet detect cfg/yolov3-tiny.cfg yolov3-tiny.weights data/dog.jpg
there will be a predictions.jpg created

4.install labelImg
sudo apt-get install pyqt5-dev-tools

sudo apt-get install python3-lxml

sudo apt-get install libxml2-dev libxslt-dev

sudo pip3 install lxml

git clone https://github.com/tzutalin/labelImg.git

export PATH=$PATH:/home/aaa/.local/bin

//this works! resolving this 'python setup.py egg_info failed' issue
python3 -m pip install -r  ./requirements/requirements-linux-python3.txt -i https://pypi.mirrors.ustc.edu.cn/simple

cd labelImg

make qt5py3

python3 labelImg.py

5.use labelImg
mkdir lll
put all pics in lll
mkdir lll_txt
click Open Dir, choose lll
click Change Save Dir, choose lll_txt
click to change to yolo
click "Create\nRectBox" to biaozhu
there are hotkeys
save on every pic biaozhu finished

6.before train
copy all txt except classes.txt from lll_txt to lll
find /home/aaa/Desktop/foryolo/lll -name \*.png > trainlist.txt
touch myvalid.txt
cut and paste 20% of lines from trainlist.txt to myvalid.txt
touch voc.names , this file write things like mycs people dog ... (every tag a line)
put and only put mycs in voc.names
this file should cooperate with the txt(corresponding img) in lll_txt folder

touch my.data
classes= 1
train  = /home/aaa/Desktop/foryolo/trainlist.txt
valid = /home/aaa/Desktop/foryolo/myvalid.txt
names = /home/aaa/Desktop/foryolo/voc.names
backup = /home/aaa/Desktop/

(note:classes=? should be really tags counts)

change darknet/cfg/yolov3-tiny.cfg
for every classes=? change it to my really tags
change the filters=? above classes to (classes + 5)*3
change from

# Testing
batch=1
subdivisions=1
# Training
#batch=64
#subdivisions=2

to

# Testing
#batch=1
#subdivisions=1
# Training
batch=64
subdivisions=16

save the file

7.install gpu driver
in 软件和更新->附加驱动
choose a gpu version(mine is 510) and apply
after install better reboot
can see version by nvidia-smi command

8.install cuda software
go to nvidia site https://developer.nvidia.com/cuda-toolkit-archive
choose cuda version according to what nvidia-smi shows (notice: version can be different from what it shows)
will get a url in nvidia's site
wget the url
sudo sh cuda_10.1.168_418.67_linux.run
wait a minute

if shows Existing package manager installation of the driver found
just go on dont worry

accept
dont choose driver cause it has bean installed

after install, need to set path
edit ~/.bashrc add two sentences like below to end of the file
export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
source ~/.bashrc

now use command nvcc -V to test if cuda has bean installed

8.5 install cudnn
not that necessary,can dont install it now

9.remake darknet
since we can use gpu now, we can modify Makefile "GPU=1" and make (notice: make clean first)
a new darknet executable file will be created

10.begin train
./darknet detector train ./../my.data  cfg/yolov3-tiny.cfg

we will get some .weights file(which is about 30m size since it's tiny v3),eventually we get yolov3-tiny_final.weights

11.after train,test
modify cfg/yolov3-tiny.cfg to below
# Testing
batch=1
subdivisions=1
# Training
#batch=64
#subdivisions=16

./darknet detector   test   ./../my.data   cfg/yolov3-tiny.cfg   ./../../yolov3-tiny_final.weights   ./../../28.png
(notice:must use our's data file)

something else:
i'm using 20 photos to train, 5 photos to varify,
and another 5 photos (num from 26 to 30) for after training test
ubuntu18 and yolov3 on darknet
learning_rate=0.002
when training, about 8g mem and whole cpu(p5420),and 50%-80%gpu(1650 laptop with cuda without cudnn) is used
max_batches is set to 2000,it takes about 2 hours to train，eventually loss is about 0.1~0.2
eventually at 28.png it gets 77% mycs result!

it worked!

references:
ubuntu+darknet+yolov3训练自己数据
https://blog.csdn.net/m0_53807457/article/details/121023419

Ubuntu18.04 RTX2070 显卡驱动、Cuda、cudnn和Pytorch深度学习环境配置——亲测可用
https://blog.csdn.net/zhanghm1995/article/details/105340538

darknet详解（yolo图像检测）
https://blog.csdn.net/m0_52571715/article/details/109713440