This job is complex, so I want to take some notes here.
So sorry.My Ubuntu do not have simple chinese.
目录
1.0 download and install labelImg and AS
1.2 download tensorflow-master
4.1 label the image by LabelImg
4.2 split train and test(train_test_split.py)
5.train, test, generate pb file
6.1 install bazel and dependency
6.2 configure the configure file
9.adb install apk into android
0.Enviroment
ubuntu18.04
#data and run the tflite model project
labelImg
Android studio
tensorflow/examples-master
tensorflow/models-v1.12.0(for training and testing, pb)
protoc3.3.0
tensorflow/tensorflow-master(for tflite)
bazel-3.0.0
python3.5(3.6 is ok)
tensorflow-v1.12.0(tensorflow-v1.14.0 is ok)
Cython
contextlib2
pillow
lxml
jupyter
matplotlib
pandas
pycocotools
1.download
if we download slowly,we can use this website.http://tool.mkblog.cn/github
1.0 download and install labelImg and AS
We need labelImg to label the image and AS for tflite model project.
About LabelImg, https://github.com/tzutalin/labelImg, https://blog.csdn.net/learning_tortosie/article/details/80947301.
About AS, https://blog.csdn.net/xulingjie_online/article/details/78501636.
1.1 download master-v.1.12.0
https://github.com/tensorflow/models
1.2 download tensorflow-master
https://github.com/tensorflow/tensorflow
If you want to download by shell,you can do like this:
git clone https://github.com/tensorflow/tensorflow.git
1.3 download protoc-3.3.0
Why do we download this high version protoc?I forgot what file and directory we can find.
https://github.com/protocolbuffers/protobuf/releases
1.4 download bazel-3.0.0
We can find configure.py in which the version of our software is.
Compile tensorflow and toco.
wget https://github.com/bazelbuild/bazel/releases/download/3.0.0/bazel-3.0.0-installer-linux-x86_64.sh
copy bazel to tensorflow/tensorflow-master
1.5 unzip the file
mkdir tensorflow
After this,unzip and copy the download file to directory.get this:
tensorflow
|_tensorflow-matser
| |_bazel
|
|_models-1.12.0
research
|_protoc-3.3
2.Install Dependency package
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple tensorflow==1.12.0 Cython contextlib2 pillow lxml jupyter matplotlib
3.Install models and protoc
download tensorflow/models-v1.12.0,unzip models into tensorflow folder;download protoc,unzip protoc-3.3.0-linux-x86_64.zip,change protoc-3.3.0-linux-x86_64 to protoc,copy to /tensorflow/models-1.12.0/research/object_detection/,copy proto/bin/proto to research
cd /tensorflow/models-1.12.0/research
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
apt install protobuf-compiler
protoc object_detection/protos/*.proto --python_out=.
python3 object_detection/builders/model_builder_test.py
output display:
----------------------------------------------------------------------
Ran 22 tests in 0.361s
OK
4.Prepare dataset
4.1 label the image by LabelImg
About LabelImg, https://blog.csdn.net/learning_tortosie/article/details/80947301.
Referring to the VOC data file format, we save the image and XML files in the corresponding directory.
VOCdevkit
|_VOC2007
|_Annotations
| |_xml
|
|_JPEGImages
|_image
4.2 split train and test(train_test_split.py)
This blog is very well written, so I refer to here.(https://blog.csdn.net/lukaslong/article/details/86649453)
I only need train and test dataset,so I change a little.
import os
import random
import time
import shutil
xmlfilepath=r'./csvdata/VOCdevkit/VOC2007/Annotations/' #Annotations
saveBasePath=r'./csvdata/VOCdevkit/VOC2007/Annotations/' #Annotations
trainval_percent=0.9
train_percent=1
total_xml = os.listdir(xmlfilepath)
num=len(total_xml)
list=range(num)
tv=int(num*trainval_percent)
tr=int(tv*train_percent)
trainval= random.sample(list,tv)
train=random.sample(trainval,tr)
print("train and val size",tv)
print("train size",tr)
# print(total_xml[1])
start = time.time()
# print(trainval)
# print(train)
test_num=0
val_num=0
train_num=0
# for directory in ['train','test',"val"]:
# xml_path = os.path.join(os.getcwd(), 'Annotations/{}'.format(directory))
# if(not os.path.exists(xml_path)):
# os.mkdir(xml_path)
# # shutil.copyfile(filePath, newfile)
# print(xml_path)
for i in list:
name=total_xml[i]
# print(i)
if i in trainval: #train and val set
# ftrainval.write(name)
if i in train:
# ftrain.write(name)
# print("train")
# print(name)
# print("train: "+name+" "+str(train_num))
directory="train"
train_num+=1
xml_path = os.path.join(saveBasePath, '{}'.format(directory))
if(not os.path.exists(xml_path)):
os.mkdir(xml_path)
filePath=os.path.join(xmlfilepath,name)
newfile=os.path.join(saveBasePath,os.path.join(directory,name))
shutil.copyfile(filePath, newfile)
else:
# fval.write(name)
# print("val")
# print("val: "+name+" "+str(val_num))
directory="validation"
xml_path = os.path.join(saveBasePath, '{}'.format(directory))
if(not os.path.exists(xml_path)):
os.mkdir(xml_path)
val_num+=1
filePath=os.path.join(xmlfilepath,name)
newfile=os.path.join(saveBasePath,os.path.join(directory,name))
shutil.copyfile(filePath, newfile)
# print(name)
else: #test set
# ftest.write(name)
# print("test")
# print("test: "+name+" "+str(test_num))
directory="test"
xml_path = os.path.join(saveBasePath, '{}'.format(directory))
if(not os.path.exists(xml_path)):
os.mkdir(xml_path)
test_num+=1
filePath=os.path.join(xmlfilepath,name)
newfile=os.path.join(saveBasePath,os.path.join(directory,name))
shutil.copyfile(filePath, newfile)
# print(name)
# End time
end = time.time()
seconds=end-start
print("train total : "+str(train_num))
print("validation total : "+str(val_num))
print("test total : "+str(test_num))
total_num=train_num+val_num+test_num
print("total number : "+str(total_num))
print( "Time taken : {0} seconds".format(seconds))
I'll split my all dataset to train and test in Annotations directory.
VOCdevkit
|_VOC2007
|_Annotations
| |_train
| | |_xml
| |
| |_test
| | |_xml
| |
| |_xml
|
|_JPEGImages
|_image
4.3 get label csv file
This blog is very well written, so I refer to here.(https://blog.csdn.net/lukaslong/article/details/86649453)
import os
import glob
import pandas as pd
import xml.etree.ElementTree as ET
PROJECT = './csvdata/VOCdevkit/VOC2007/Annotations/'
def xml_to_csv(path):
xml_list = []
for xml_file in glob.glob(path + '/*.xml'):
tree = ET.parse(xml_file)
root = tree.getroot()
for member in root.findall('object'):
value = (root.find('filename').text,
int(root.find('size').find('width').text),
int(root.find('size').find('height').text),
member.find('name').text,
int(member.find('bndbox').find('xmin').text),
int(member.find('bndbox').find('ymin').text),
int(member.find('bndbox').find('xmax').text),
int(member.find('bndbox').find('ymax').text))
xml_list.append(value)
column_name = ['filename', 'width', 'height', 'class', 'xmin', 'ymin', 'xmax', 'ymax']
xml_df = pd.DataFrame(xml_list, columns=column_name)
return xml_df
def main():
for folder in ['train', 'test']:
xml_path = os.path.join(os.getcwd(), (PROJECT + folder))
xml_df = xml_to_csv(xml_path)
csv_path = PROJECT + folder + '_labels.csv'
if not os.path.isfile(csv_path):
fp = open(csv_path, 'w')
fp.close()
xml_df.to_csv(csv_path, index=None)
print('Successfully converted xml to csv.')
main()
I'll get this:
VOCdevkit
|_VOC2007
|_Annotations
| |_train
| | |_xml
| |
| |_test
| | |_xml
| |
| |_xml
| |_train_labels.csv
| |_test_labels.csv
|
|_JPEGImages
|_image
I copy train_labels.csv and test_labels.csv to csvdata directory.
csvdata
train_test_split.py
xml_to_csv.py
train_labels.csv
test_labels.csv
VOCdevkit
|_VOC2007
|_Annotations
| |_train
| | |_xml
| |
| |_test
| | |_xml
| |
| |_xml
| |_train_labels.csv
| |_test_labels.csv
|
|_JPEGImages
|_image
4.4 get tfrecord file
This step we need do it after that models was installed.So we need go to the directory of models/research and export the path.
cd /tensorflow/models-1.12.0/research
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
/csvdata directory
build label_map.pbtxt in csvdata directory,add content.
item {
name: "my_label"
id: 1
display_name: "my_label"
}
in csvdata directory, we'll get train.record,test.record with the following command.
python3 generate_tfrecord.py --csv_input=./VOCdevkit/VOC2007/Annotations/train_labels.csv --images_input=./VOCdevkit/VOC2007/JPEGImages --output_path=./train.record --label_map_path=./label_map.pbtxt
python3 generate_tfrecord.py --csv_input=./VOCdevkit/VOC2007/Annotations/test_labels.csv --images_input=./VOCdevkit/VOC2007/JPEGImages --output_path=./test.record --label_map_path=./label_map.pbtxt
output display
Successfully created the TFRecords: ./train.record
Successfully created the TFRecords: ./test.record
So far, train and test data are ready.
5.train, test, generate pb file
5.1 file prepare
(1)/tensorflow/models/research/object_detection/legacy/train.py eval.py and export_tflite_ssd_graph.py, copy train.py eval.py and export_tflite_ssd_graph.pyto our project directory.
(2)/tensorflow/models/research/object_detection/samples/config/ssd_mobilenet_v2_coco.config, copy it to our project directory,and modify ssd_mobilenet_v2_coco.config like this:
##file name change
ssd_mobilenet_v2_coco.config
-->
ssd_mobilenet_v2_our.config
##first line
model {
ssd {
num_classes: our num_classes(eg:1)
box_coder {
faster_rcnn_box_coder {
y_scale: 10.0
x_scale: 10.0
height_scale: 5.0
width_scale: 5.0
}
}
##middle
image_resizer {
fixed_shape_resizer {
height: our height
width: our width
}
}
##last line
train_input_reader: {
tf_record_input_reader {
input_path: "./csvdata/train.record"
}
label_map_path: "./csvdata/label_map.pbtxt"
}
eval_config: {
num_examples: 153
# Note: The below line limits the evaluation process to 10 evaluations.
# Remove the below line to evaluate indefinitely.
max_evals: 10
}
eval_input_reader: {
tf_record_input_reader {
input_path: "./csvdata/test.record"
}
label_map_path: "./csvdata/label_map.pbtxt"
shuffle: false
num_readers: 1
}
5.2 train
(1)don't need pretrain model:
sudo su
echo "your_password"
cd /tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
cd your_train.py_path
python3 train.py --logtostderr --train_dir=./logs --pipeline_config_path=./ssd_mobilenet_v2_our.config
(2)ssdlite pretrain :
0.download pretrain model and unzip into ssdlitepretrain_logs floder. (http://download.tensorflow.org/models/object_detection/ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz)
1.rename ssd_mobilenet_v2_coco.config as ssd_mobilenet_v2_ssdlitepretrain_our.config
2.we need modify this in ./ssd_mobilenet_v2_ssdlitepretrain_our.config:
fine_tune_checkpoint: "ssdlitepretrain_logs/ssdlite_mobilenet_v2_coco_2018_05_09/model.ckpt"
fine_tune_checkpoint_type: "detection"
train:
cd /tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
cd your_project_path
python3 train.py --logtostderr --train_dir=./ssdlitepretrain_logs --pipeline_config_path=./ssd_mobilenet_v2_ssdlitepretrain_our.config
5.2 test
test,every time,we need go to tensorflow/models/research directory and run:
cd /tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
build test_logs floder and copy 3 ckpt and checkpoint file to this directory.
explain
eval_dir: event file will save this path
pipeline_config_path: config file
checkpoint_dir: test model file(include 3 ckpt and checkpoint)
checkpoint
model_checkpoint_path: "model.ckpt-17154"
all_model_checkpoint_paths: "model.ckpt-17154"
python3 eval.py --logtostderr --eval_dir=logs/test_logs/ --pipeline_config_path=ssd_mobilenet_v2_our.config --checkpoint_dir=logs/test_logs/
output display:
2020-04-26 09:26:39.497905: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
INFO:tensorflow:Restoring parameters from logs/test_logs/model.ckpt-17154
INFO:tensorflow:Creating detection visualizations.
INFO:tensorflow:Detection visualizations written to summary with tag image-0.
WARNING:root:image b'1587473888272.jpg' does not have groundtruth difficult flag specified
INFO:tensorflow:Creating detection visualizations.
INFO:tensorflow:Creating detection visualizations.
INFO:tensorflow:Detection visualizations written to summary with tag image-1.
INFO:tensorflow:Detection visualizations written to summary with tag image-1.
......................
INFO:tensorflow:Creating detection visualizations.
INFO:tensorflow:Detection visualizations written to summary with tag image-9.
INFO:tensorflow:Detection visualizations written to summary with tag image-9.
INFO:tensorflow:Running eval ops batch 100/153
INFO:tensorflow:Running eval ops batch 100/153
INFO:tensorflow:Running eval batches done.
INFO:tensorflow:Running eval batches done.
INFO:tensorflow:# success: 153
INFO:tensorflow:# success: 153
INFO:tensorflow:# skipped: 0
INFO:tensorflow:# skipped: 0
INFO:tensorflow:Writing metrics to tf summary.
INFO:tensorflow:Writing metrics to tf summary.
INFO:tensorflow:Losses/Loss/classification_loss: 3.134053
INFO:tensorflow:Losses/Loss/classification_loss: 3.134053
INFO:tensorflow:Losses/Loss/localization_loss: 0.519499
INFO:tensorflow:Losses/Loss/localization_loss: 0.519499
INFO:tensorflow:PascalBoxes_PerformanceByCategory/AP@0.5IOU/face: 0.998818
INFO:tensorflow:PascalBoxes_PerformanceByCategory/AP@0.5IOU/face: 0.998818
INFO:tensorflow:PascalBoxes_Precision/mAP@0.5IOU: 0.998818
INFO:tensorflow:PascalBoxes_Precision/mAP@0.5IOU: 0.998818
INFO:tensorflow:Metrics written to tf summary.
INFO:tensorflow:Metrics written to tf summary.
If python version is 3.6, I 'll modify one place
(/tensorflow/models-1.12.0/research/object_detection/utils/object_detection_evaluation.py):
problem:
File "/tensorflow/models-1.12.0/research/object_detection/utils/object_detection_evaluation.py", line 307, in evaluate
category_name = unicode(category_name, 'utf-8')
NameError: name 'unicode' is not defined
solve:
try:
category_name = unicode(category_name, 'utf-8')
except TypeError:
pass
to
try:
category_name = str(category_name, 'utf-8')
except TypeError:
pass
5.3 generate pb file
cd /tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
cd our_project_path
python3 export_tflite_ssd_graph.py --pipeline_config_path=./ssd_mobilenet_v2_our.config --trained_checkpoint_prefix=./logs/test_logs/model.ckpt-17154 --output_directory=./logs/test_logs/tflite --add_postprocessing_op=true
output display:
2020-04-24 15:20:06.423710: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-24 15:20:14.968169: I tensorflow/tools/graph_transforms/transform_graph.cc:317] Applying strip_unused_nodes
output:
./logs/test_logs/tflite/tflite_graph.pb
./logs/test_logs/tflite/tflite_graph.pbtxt
6.complie tensorflow
6.1 install bazel and dependency
sudo su
echo secret
cd tensorflow/tensorflow-master
sudo apt install openjdk-8-jdk
chmod +x bazel-3.0.0-installer-linux-x86_64.sh
./bazel-3.0.0-installer-linux-x86_64.sh --user
export PATH="$PATH:$HOME/bin"
6.2 configure the configure file
configure all no include cuda.
6.3 compile tensorflow
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
This step is so long and is related to the network quality.we only need to wait or do other steps.
output display
2020-04-30 13:13:16.994147: I tensorflow/lite/toco/toco_tooling.cc:456] Estimated count of arithmetic ops: 36 ops, equivalently 18 MACs
2020-04-30 13:13:16.994157: I tensorflow/lite/toco/toco_tooling.cc:471] Number of parameters: 20
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 26380.860s, Critical Path: 289.34s
INFO: 16810 processes: 16810 local.
INFO: Build completed successfully, 18318 total actions
6.4 compile freeze_graph
bazel build tensorflow/python/tools:freeze_graph
output display
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
INFO: From ProtoCompile tensorflow/core/profiler/profiler_analysis.pb.h:
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
Target //tensorflow/python/tools:freeze_graph up-to-date:
bazel-bin/tensorflow/python/tools/freeze_graph
INFO: Elapsed time: 23842.591s, Critical Path: 279.66s
INFO: 16363 processes: 16363 local.
INFO: Build completed successfully, 16607 total actions
6.5 compile toco
bazel build tensorflow/lite/toco:toco
output display
INFO: From Compiling external/snappy/snappy.cc:
cc1plus: warning: command line option '-Wno-implicit-function-declaration' is valid for C/ObjC but not for C++
Target //tensorflow/lite/toco:toco up-to-date:
bazel-bin/tensorflow/lite/toco/toco
INFO: Elapsed time: 357.195s, Critical Path: 27.39s
INFO: 513 processes: 513 local.
INFO: Build completed successfully, 518 total actions
7.tranform pb to tflite
(Don't care the version to complie toco(tensorflow/lite version).so we don't need to be afraid about the version difference between train the model tensorflow and transform tflite flie tensorflow)
in the complie path(tensorflow/tensorflow-master/),input follow command:
float32
bazel run tensorflow/lite/toco:toco -- --input_file=./logs/test_logs/tflite/tflite_graph.pb --output_file=./logs/test_logs/tflite/ssd_mobilenetv2.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' --inference_type=FLOAT --allow_custom_ops
output:
ssd_mobilenetv2.tflite
output display:
INFO: Build completed successfully, 1 total action
2020-04-25 07:56:16.274973: I tensorflow/lite/toco/import_tensorflow.cc:660] Converting unsupported operation: TFLite_Detection_PostProcess
2020-04-25 07:56:16.342742: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 878 operators, 1282 arrays (0 quantized)
2020-04-25 07:56:16.406979: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 878 operators, 1282 arrays (0 quantized)
2020-04-25 07:56:16.506001: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 99 operators, 261 arrays (0 quantized)
2020-04-25 07:56:16.509781: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Group bidirectional sequence lstm/rnn: 99 operators, 261 arrays (0 quantized)
2020-04-25 07:56:16.513776: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before dequantization graph transformations: 99 operators, 261 arrays (0 quantized)
2020-04-25 07:56:16.515993: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Identify nearest upsample.: 99 operators, 261 arrays (0 quantized)
2020-04-25 07:56:16.524839: I tensorflow/lite/toco/allocate_transient_arrays.cc:345] Total transient array allocated size: 2611200 bytes, theoretical optimal value: 2304000 bytes.
2020-04-25 07:56:16.531669: I tensorflow/lite/toco/toco_tooling.cc:456] Estimated count of arithmetic ops: 275129688 ops, equivalently 137564844 MACs
2020-04-25 07:56:16.531754: I tensorflow/lite/toco/toco_tooling.cc:471] Number of parameters: 4593955
2020-04-25 07:56:16.534587: W tensorflow/lite/toco/tflite/operator.cc:1686] Ignoring unsupported type in list attribute with key '_output_types'
If you want to quantized_uint8, follow command may help.
https://www.free-music.cf/-----https://github.com/tensorflow/models/issues/8148
For more parameters, we can refer to https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/toco/toco_flags.proto.
8.run in AS
8.1 download tensorflow/example
There is Android object detection example.
https://github.com/tensorflow/examples.git
if we not chage the image size and want to use my own tflite model, we'll change several place.
(1)Comment out the last line of the screenshot of App/ build.gradle file.
(2) change App/Asset
detect.tflite to your own detect.tflite.
labelmap.txt include your label.
???
your_label
Now we can build it.That should be enough for simple tasks.
9.adb install apk into android
first open android studio;compile run and generate apk file ,install by this command:
adb connect yout_android_machine_IP
adb install -r /examples/lite/examples/object_detection/android/app/build/outputs/apk/debug/app-debug.apk
output display:
root@***:/examples/lite/examples/object_detection/android# adb install -t /examples/lite/examples/object_detection/android/app/build/outputs/apk/debug/app-debug.apk
Performing Push Install
/examples/lite/examples/object_detection/andr...pk: 1 file pushed, 0 skipped. 3.0 MB/s (43508148 bytes in 13.605s)
pkg: /data/local/tmp/app-debug.apk
Success
If you want to integrate this project into other projects, you can refer to as integration.
This may help.
Reference
1.使用tensorflow自带model训练SSD并且在手机上运行
2.【TF lite】从tensorflow模型训练到lite模型移植