在Android端部署yolov5（显示类别个数）

积极向上的mr.d

已于 2023-06-05 11:59:19 修改

阅读量7.7k

点赞数 20

分类专栏： yolo应用安卓部署 YOLO 文章标签： python yolo部署部署安卓

于 2023-03-27 21:17:47 首次发布

本文链接：https://blog.csdn.net/qq_45077760/article/details/129802464

版权

YOLO 同时被 3 个专栏收录

38 篇文章 61 订阅

订阅专栏

yolo应用

14 篇文章 5 订阅

订阅专栏

安卓部署

3 篇文章 3 订阅

订阅专栏

闲来无事，记录一下自己在安卓端部署yolov5的步骤，历时一天

不想部署想直接拿来玩玩的，可以直接下载手机软件：

下载链接： https://download.csdn.net/download/qq_45077760/87629448

整个工程代码下载：https://download.csdn.net/download/qq_45077760/87864014

1.pt文件转onnx

我用的是自己训练的权重文件，版本是6.1版本，直接执行以下代码对我的权重文件best.pt进行转化。过程可能会报错，如果报错就安装相应的库文件，一般是"no moudle named onnx",直接pip install onnx==1.7.0；因为如果直接装onnx，可能还是会报错，所以直接装1.7.0的onnx

python export.py --weights best.pt

转换之前记住一个细节问题！！！记得把train这里加上True。一定要注意，要不下边步骤会报错

执行完以上步骤，你会得到对应的best.onnx文件，这里我把名字best.onnx直接改成yolov5s.onnx，方便讲解

2.onnx文件转bin和param文件

2.1 直接使用工具转化

这里网上有很多方法，但是都需要配置环境，这里要感谢一个github博主，他直接做了一个转换界面，太厉害了，网址附上：https://convertmodel.com/

点击进入界面，把你的onnx文件放入，记得步骤1转换时候加上 default=True，要不这里转换会报错，成功转换界面如下，如果出现红色就代表转换失败

转换后会生成两个文件，点击下载生成的两个文件：yolov5s-slm-opt.param和yolov5s-slm-opt.bin

继续把yolov5s-slm-opt.param和yolov5s-slm-opt.bin名字改成yolov5s.param和yolov5s.bin

这个步骤没有问题的话直接忽略步骤 2.2 ，跳转步骤3

2.2 使用Ubuntu部署

2.2.1 安装onnx-simplifier

pip install onnx-simplifier
pip install onnx-runtime
python -m onnxsim ./yolov5.onnx ./yolo5-sim.onnx  ##得到简化后的模型，这里使用你自己的路径

2.2.2 ncnn代码库编译

如果实在想自己编译，可以按照以下步骤进行，不过还是建议使用 2.1 直接用工具转化的方法

git clone https://github.com/Tencent/ncnn.git
cd ncnn
git submodule update --init
mkdir build
cd build
cmake ..
make -j8
make install

2.2.3 onnx转ncnn

将简化后的模型yolo5s-sim.onnx拖进ncnn/build/tools/onnx目录，cd 切换到该目录下进行操作

./onnx2ncnn yolo5s-sim.onnx yolov5s.param yolov5s.bin

到这里你就得到 yolov5s.param 和 yolov5s.bin两个文件

2.2.4 注意事项

执行2.2.3步骤时候可能会出现以下问题，这是因为split和crop网络层高在ncnn中没有定义，我们需要做的是将这个网络层消除：

打开yolov5s.param文件，对模型进行修改，将以下yolov5s.param文件开头代码

7767517
236 197
Input            images                   0 1 images
split            splitncnn_input          l 4 images images_splitncnn_0 images_splitncnn_l images_splitncnn_2 images_splitncnn_3
crop             slice_4                  l l images_splitncnn_3 171 -23309=1,6 -23310=1,2147483647 -2331l=1,l
crop             slice_9                  l 1 171 176 -23309=1,o -23310=1,2147483647 -23311=1,2
crop             slice_14                 l l images_splitncnn_2 181 -23309=1,1 -23310=1,2147483647 -2331l=1,1
crop             slice_i9                 l 1 181 186 -23309=1,0 -23310=1,2147483647 -2331l=1,2
crop             slice_24                 l l images_splitncnn_1 191 -23309=1,0 -23310=1,2147483647-2331l=1,1
crop             slice 29                 l 1 191 196 -23309=1,1 -23310=1,2147483647 -2331l=1,2
crop             slice_34                 l l images_splitncnn_0 201 -23309=1,1 -23310=1,2147483647 -23311=1,l
crop             slice_39                 l 1 201 206 -23309=1,1 -23310=1,2147483647 -2331l=1,2
concat           concat_40                4 1 176 186 196 206 207 0=0
Convolution      Conv_0                   1 1 images 122 0=32 1=6 11=6 2=1 12=1 3=2 13=2 4=2 14=2 15=2 16=2 5=1 6=3456
Swish            Mul_2                    1 1 122 124

删除 Split、Concat 和8个 Crop 节点，并且加入新的节点 YoloV5Focus ，替换成

7767517
227 265
Input            images                   0 1 images
YoloV5Focus      focus                    1 1 images 122
Convolution      Conv_0                   1 1 images 122 0=32 1=6 11=6 2=1 12=1 3=2 13=2 4=2 14=2 15=2 16=2 5=1 6=3456
Swish            Mul_2                    1 1 122 124

其中 227是由于之前的236网络层我们删除了10行，并用YoloV5Focus网络层代替，剩227个，而YoloV5Focus网络层中的images代表该层的输入，122代表输出名

详细步骤可见：https://zhuanlan.zhihu.com/p/275989233

（注：推荐两个博主的步骤，一个Ubuntu系统，一个win系统：

1.Ubuntu系统：https://blog.csdn.net/qq_44696500/article/details/124195375

2.win系统：https://blog.csdn.net/qq_28664681/article/details/112650644 ）

3.下载ncnn-android-yolov5和ncnn-20230223-android-vulkan

（1）下载ncnn-android-yolov5文件，下载链接：https://github.com/nihui/ncnn-android-yolov5

解压后得到名为ncnn-android-yolov5-master的文件夹

（2）下载ncnn-20230223-android-vulkan文件，下载链接：https://github.com/Tencent/ncnn/releases

解压后得到把文件夹里的以下四个文件拖进C:\Users\hp\Desktop\ncnn-android-yolov5-master\app\src\main\jni\这个路径

（3）把步骤2生成的两个文件yolov5st.param和yolov5s.bin拉进C:\Users\hp\Desktop\ncnn-android-yolov5-master\app\src\main\assets这个文件夹里

4.安装Android studio

下载链接：https://developer.android.google.cn/studio/

具体安装步骤网上都有，这里就不再写了

下载后用Android studio 打开ncnn-android-yolov5-master这个文件，然后点击左上角File-settings

再点击Android SDK，选择你手机的安卓版本，博主版本是9.0

继续在这个界面点击SDK Tools，博主画红色框的记得选一下，SDK、cmake和NDK一定要选择

配置一下NDK，点击左上角File--Project Structure，这一步可能你的NDK选择会报错，因为正常情况下时会自动选择的，这是因为NDK版本装得太高了，可以执行上边步骤，重新安装低版本的NDK，我这里安装的是21.0.6113669，如果不行就多试几个版本就可以

5.修改源码

（1）修改CMakeLists.txt

打开ncnn-android-yolov5-master\app\src\main\jni\CMakeLists.txt，把里边的代码

set(ncnn_DIR ${CMAKE_SOURCE_DIR}/ncnn-20201218-android-vulkan/${ANDROID_ABI}/lib/cmake/ncnn)

替换成下边代码

set(ncnn_DIR ${CMAKE_SOURCE_DIR}/${ANDROID_ABI}/lib/cmake/ncnn)

（2）修改yolov5s.param

在yolov5s.param里搜索reshape，把reshape后边的数值全修改成0=-1，如果不修改的话，部署后会出现很多密密麻麻的小检测框

（3）修改yolov5ncnn_jni.cpp

在yolov5s.param里搜索Permute，如图所示，记住最后两个Permute后边画框的数值，我这里是353和367

打开yolov5ncnn_jni.cpp文件，把最后两个Permute的数值353和367分别写进下边图中

修改yolov5ncnn_jni.cpp文件里的类别，改成你的全中所用的类别名称

6.编译工程

（1）这里重复使用可能会混乱，如果文件夹里有build这个文件记得先删除，没有的话不用理会

然后点击Build--Clean Project

（2）点击build gradle这个文件运行等待生成，我这里有两个build gradle文件，点击红色这个，记得不要点错了

（3）生成project

（4）连接手机，点击运行按钮将软件下载到手机

7.部署完成

如果安装后把软件分享给别人，别人安装apk时遇到提示“无效的安装包”或者“安装包解析出错”等状况，说明你没有签名方案，解决方法参考 https://blog.csdn.net/qq_45077760/article/details/130551940

8. 显示类别

如果想要显示类别和类别数，修改ncnn-android-yolov5-master\app\src\main\java\com\tencent\yolov5ncnn\MainActivity.java文件，具体将public void showObjects里的内容

 private void showObjects(YoloV5Ncnn.Obj[] objects)
    {
        if (objects == null)
        {
            imageView.setImageBitmap(bitmap);
            return;
        }

        // draw objects on bitmap
        Bitmap rgba = bitmap.copy(Bitmap.Config.ARGB_8888, true);

        final int[] colors = new int[] {
            Color.rgb( 54,  67, 244),
            Color.rgb( 99,  30, 233),
            Color.rgb(176,  39, 156),
            Color.rgb(183,  58, 103),
            Color.rgb(181,  81,  63),
            Color.rgb(243, 150,  33),
            Color.rgb(244, 169,   3),
            Color.rgb(212, 188,   0),
            Color.rgb(136, 150,   0),
            Color.rgb( 80, 175,  76),
            Color.rgb( 74, 195, 139),
            Color.rgb( 57, 220, 205),
            Color.rgb( 59, 235, 255),
            Color.rgb(  7, 193, 255),
            Color.rgb(  0, 152, 255),
            Color.rgb( 34,  87, 255),
            Color.rgb( 72,  85, 121),
            Color.rgb(158, 158, 158),
            Color.rgb(139, 125,  96)
        };

        Canvas canvas = new Canvas(rgba);

        Paint paint = new Paint();
        paint.setStyle(Paint.Style.STROKE);
        paint.setStrokeWidth(4);

        Paint textbgpaint = new Paint();
        textbgpaint.setColor(Color.WHITE);
        textbgpaint.setStyle(Paint.Style.FILL);

        Paint textpaint = new Paint();
        textpaint.setColor(Color.BLACK);
        textpaint.setTextSize(26);
        textpaint.setTextAlign(Paint.Align.LEFT);

        for (int i = 0; i < objects.length; i++)
        {
            paint.setColor(colors[i % 19]);

            canvas.drawRect(objects[i].x, objects[i].y, objects[i].x + objects[i].w, objects[i].y + objects[i].h, paint);

            // draw filled text inside image
            {
                String text = objects[i].label + " = " + String.format("%.1f", objects[i].prob * 100) + "%";

                float text_width = textpaint.measureText(text);
                float text_height = - textpaint.ascent() + textpaint.descent();

                float x = objects[i].x;
                float y = objects[i].y - text_height;
                if (y < 0)
                    y = 0;
                if (x + text_width > rgba.getWidth())
                    x = rgba.getWidth() - text_width;

                canvas.drawRect(x, y, x + text_width, y + text_height, textbgpaint);

                canvas.drawText(text, x, y - textpaint.ascent(), textpaint);
            }
        }

        imageView.setImageBitmap(rgba);
    }

替换成以下：

 private void showObjects(YoloV5Ncnn.Obj[] objects)
    {
        if (objects == null)
        {
            imageView.setImageBitmap(bitmap);
            return;
        }

        // draw objects on bitmap
        Bitmap rgba = bitmap.copy(Bitmap.Config.ARGB_8888, true);

        final int[] colors = new int[] {
            Color.rgb( 54,  67, 244),
            Color.rgb( 99,  30, 233),
            Color.rgb(176,  39, 156),
            Color.rgb(183,  58, 103),
            Color.rgb(181,  81,  63),
            Color.rgb(243, 150,  33),
            Color.rgb(244, 169,   3),
            Color.rgb(212, 188,   0),
            Color.rgb(136, 150,   0),
            Color.rgb( 80, 175,  76),
            Color.rgb( 74, 195, 139),
            Color.rgb( 57, 220, 205),
            Color.rgb( 59, 235, 255),
            Color.rgb(  7, 193, 255),
            Color.rgb(  0, 152, 255),
            Color.rgb( 34,  87, 255),
            Color.rgb( 72,  85, 121),
            Color.rgb(158, 158, 158),
            Color.rgb(139, 125,  96)
        };

        Canvas canvas = new Canvas(rgba);

        Paint paint = new Paint();
        paint.setStyle(Paint.Style.STROKE);
        paint.setStrokeWidth(4);

        Paint textbgpaint = new Paint();
        textbgpaint.setColor(Color.WHITE);
        textbgpaint.setStyle(Paint.Style.FILL);

        Paint textpaint = new Paint();
        textpaint.setColor(Color.BLACK);
        textpaint.setTextSize(26);
        textpaint.setTextAlign(Paint.Align.LEFT);
        //增加一个笔刷textpaint2
        Paint textpaint2 = new Paint();
        textpaint2.setColor(Color.RED);
        textpaint2.setTextSize(26);
        textpaint2.setTextAlign(Paint.Align.LEFT);


        ArrayList<String> list = new ArrayList<String>();


        for (int i = 0; i < objects.length; i++)
        {
            paint.setColor(colors[i % 19]);

            canvas.drawRect(objects[i].x, objects[i].y, objects[i].x + objects[i].w, objects[i].y + objects[i].h, paint);

            // draw filled text inside image
            {
                String text = objects[i].label + " = " + String.format("%.1f", objects[i].prob * 100) + "%" ;
                list.add(objects[i].label);
                Map<String, Integer> map = new HashMap<>();
                for (String l : list) {
                    map.merge(l, 1, Integer::sum);
                }

                float text_width = textpaint.measureText(text);
                float text_height = - textpaint.ascent() + textpaint.descent();

                float x = objects[i].x;
                float y = objects[i].y - text_height;
                if (y < 0)
                    y = 0;
                if (x + text_width > rgba.getWidth())
                    x = rgba.getWidth() - text_width;

                canvas.drawRect(x, y, x + text_width, y + text_height, textbgpaint);

                canvas.drawText(text, x, y - textpaint.ascent(), textpaint);
            }
        }
        String Number = "物品总数：" + objects.length + "...";//字符串变量Number保存物品总数
        Map<String, Integer> map = new HashMap<>();//Map型变量保存
        for (String l : list) {
            map.merge(l, 1, Integer::sum);
        }

        Date date = new Date();
        canvas.drawText(String.format("%tc%n",date), 30, 30, textpaint2);//第一行字，显示当前时间戳
        canvas.drawText(Number, 30, 60, textpaint2);//显示总物品数量
        canvas.drawText(map.toString(), 30, 90, textpaint2);//显示map，包含每个类和数量
        imageView.setImageBitmap(rgba);

    }

替换后MainActivity.java完整代码如下：

// Tencent is pleased to support the open source community by making ncnn available.
//
// Copyright (C) 2020 THL A29 Limited, a Tencent company. All rights reserved.
//
// Licensed under the BSD 3-Clause License (the "License"); you may not use this file except
// in compliance with the License. You may obtain a copy of the License at
//
// https://opensource.org/licenses/BSD-3-Clause
//
// Unless required by applicable law or agreed to in writing, software distributed
// under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.

package com.tencent.yolov5ncnn;

import android.app.Activity;
import android.content.Intent;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.media.ExifInterface;
import android.graphics.Matrix;
import android.net.Uri;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.Button;
import android.widget.ImageView;

import java.io.FileNotFoundException;
import java.io.InputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

public class MainActivity extends Activity
{
    private static final int SELECT_IMAGE = 1;

    private ImageView imageView;
    private Bitmap bitmap = null;
    private Bitmap yourSelectedImage = null;

    private YoloV5Ncnn yolov5ncnn = new YoloV5Ncnn();

    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        boolean ret_init = yolov5ncnn.Init(getAssets());
        if (!ret_init)
        {
            Log.e("MainActivity", "yolov5ncnn Init failed");
        }

        imageView = (ImageView) findViewById(R.id.imageView);

        Button buttonImage = (Button) findViewById(R.id.buttonImage);
        buttonImage.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View arg0) {
                Intent i = new Intent(Intent.ACTION_PICK);
                i.setType("image/*");
                startActivityForResult(i, SELECT_IMAGE);
            }
        });

        Button buttonDetect = (Button) findViewById(R.id.buttonDetect);
        buttonDetect.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View arg0) {
                if (yourSelectedImage == null)
                    return;

                YoloV5Ncnn.Obj[] objects = yolov5ncnn.Detect(yourSelectedImage, false);

                showObjects(objects);
            }
        });

        Button buttonDetectGPU = (Button) findViewById(R.id.buttonDetectGPU);
        buttonDetectGPU.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View arg0) {
                if (yourSelectedImage == null)
                    return;

                YoloV5Ncnn.Obj[] objects = yolov5ncnn.Detect(yourSelectedImage, true);

                showObjects(objects);
            }
        });
    }

    private void showObjects(YoloV5Ncnn.Obj[] objects)
    {
        if (objects == null)
        {
            imageView.setImageBitmap(bitmap);
            return;
        }

        // draw objects on bitmap
        Bitmap rgba = bitmap.copy(Bitmap.Config.ARGB_8888, true);

        final int[] colors = new int[] {
            Color.rgb( 54,  67, 244),
            Color.rgb( 99,  30, 233),
            Color.rgb(176,  39, 156),
            Color.rgb(183,  58, 103),
            Color.rgb(181,  81,  63),
            Color.rgb(243, 150,  33),
            Color.rgb(244, 169,   3),
            Color.rgb(212, 188,   0),
            Color.rgb(136, 150,   0),
            Color.rgb( 80, 175,  76),
            Color.rgb( 74, 195, 139),
            Color.rgb( 57, 220, 205),
            Color.rgb( 59, 235, 255),
            Color.rgb(  7, 193, 255),
            Color.rgb(  0, 152, 255),
            Color.rgb( 34,  87, 255),
            Color.rgb( 72,  85, 121),
            Color.rgb(158, 158, 158),
            Color.rgb(139, 125,  96)
        };

        Canvas canvas = new Canvas(rgba);

        Paint paint = new Paint();
        paint.setStyle(Paint.Style.STROKE);
        paint.setStrokeWidth(4);

        Paint textbgpaint = new Paint();
        textbgpaint.setColor(Color.WHITE);
        textbgpaint.setStyle(Paint.Style.FILL);

        Paint textpaint = new Paint();
        textpaint.setColor(Color.BLACK);
        textpaint.setTextSize(26);
        textpaint.setTextAlign(Paint.Align.LEFT);
        //增加一个笔刷textpaint2
        Paint textpaint2 = new Paint();
        textpaint2.setColor(Color.RED);
        textpaint2.setTextSize(26);
        textpaint2.setTextAlign(Paint.Align.LEFT);


        ArrayList<String> list = new ArrayList<String>();


        for (int i = 0; i < objects.length; i++)
        {
            paint.setColor(colors[i % 19]);

            canvas.drawRect(objects[i].x, objects[i].y, objects[i].x + objects[i].w, objects[i].y + objects[i].h, paint);

            // draw filled text inside image
            {
                String text = objects[i].label + " = " + String.format("%.1f", objects[i].prob * 100) + "%" ;
                list.add(objects[i].label);
                Map<String, Integer> map = new HashMap<>();
                for (String l : list) {
                    map.merge(l, 1, Integer::sum);
                }

                float text_width = textpaint.measureText(text);
                float text_height = - textpaint.ascent() + textpaint.descent();

                float x = objects[i].x;
                float y = objects[i].y - text_height;
                if (y < 0)
                    y = 0;
                if (x + text_width > rgba.getWidth())
                    x = rgba.getWidth() - text_width;

                canvas.drawRect(x, y, x + text_width, y + text_height, textbgpaint);

                canvas.drawText(text, x, y - textpaint.ascent(), textpaint);
            }
        }
        String Number = "物品总数：" + objects.length + "...";//字符串变量Number保存物品总数
        Map<String, Integer> map = new HashMap<>();//Map型变量保存
        for (String l : list) {
            map.merge(l, 1, Integer::sum);
        }

        Date date = new Date();
        canvas.drawText(String.format("%tc%n",date), 30, 30, textpaint2);//第一行字，显示当前时间戳
        canvas.drawText(Number, 30, 60, textpaint2);//显示总物品数量
        canvas.drawText(map.toString(), 30, 90, textpaint2);//显示map，包含每个类和数量
        imageView.setImageBitmap(rgba);

    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data)
    {
        super.onActivityResult(requestCode, resultCode, data);

        if (resultCode == RESULT_OK && null != data) {
            Uri selectedImage = data.getData();

            try
            {
                if (requestCode == SELECT_IMAGE) {
                    bitmap = decodeUri(selectedImage);

                    yourSelectedImage = bitmap.copy(Bitmap.Config.ARGB_8888, true);

                    imageView.setImageBitmap(bitmap);
                }
            }
            catch (FileNotFoundException e)
            {
                Log.e("MainActivity", "FileNotFoundException");
                return;
            }
        }
    }

    private Bitmap decodeUri(Uri selectedImage) throws FileNotFoundException
    {
        // Decode image size
        BitmapFactory.Options o = new BitmapFactory.Options();
        o.inJustDecodeBounds = true;
        BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o);

        // The new size we want to scale to
        final int REQUIRED_SIZE = 640;

        // Find the correct scale value. It should be the power of 2.
        int width_tmp = o.outWidth, height_tmp = o.outHeight;
        int scale = 1;
        while (true) {
            if (width_tmp / 2 < REQUIRED_SIZE
               || height_tmp / 2 < REQUIRED_SIZE) {
                break;
            }
            width_tmp /= 2;
            height_tmp /= 2;
            scale *= 2;
        }

        // Decode with inSampleSize
        BitmapFactory.Options o2 = new BitmapFactory.Options();
        o2.inSampleSize = scale;
        Bitmap bitmap = BitmapFactory.decodeStream(getContentResolver().openInputStream(selectedImage), null, o2);

        // Rotate according to EXIF
        int rotate = 0;
        try
        {
            ExifInterface exif = new ExifInterface(getContentResolver().openInputStream(selectedImage));
            int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);
            switch (orientation) {
                case ExifInterface.ORIENTATION_ROTATE_270:
                    rotate = 270;
                    break;
                case ExifInterface.ORIENTATION_ROTATE_180:
                    rotate = 180;
                    break;
                case ExifInterface.ORIENTATION_ROTATE_90:
                    rotate = 90;
                    break;
            }
        }
        catch (IOException e)
        {
            Log.e("MainActivity", "ExifInterface IOException");
        }

        Matrix matrix = new Matrix();
        matrix.postRotate(rotate);
        return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
    }

}

结果如下