OpenCV && Tesseract-OCR in Android Studio

由于本周工作的需要，我利用Java重构了之前自己C++实现的图像识别算法。因为自己之前只在慕课网上面看过一些Java基础入门教程，如下所示：

• Java入门第一季
• Java入门第二季
• Java入门第三季

所以，这五天利用Java重构图像识别算法，并进行Android开发的过程是痛苦的。我把自己实现的过程记录下来，以便遇到相关项目的小伙伴可以节省时间:)

一、版本说明

Android

安卓开发工具采用Android Studio 2.2.2，下载地址.

OpenCV

OpenCV采用steveliles基于OpenCV3.1.0编译的opencv-android1.

Tesseract-OCR

Tesseract-OCR 我利用了rmtheis封装好的tess-two2.

二、创建安卓库

设置Android SDK

参考Android Studio上进行OpenCV 3.1开发 – JohnHany的博客，设置Android SDK。

创建Android App

1. 打开Android Studio，点击Start a new Android Studio Project；
2. 在Application name 和 Company Domain输入应用名称和公司域名，并在Project location指定项目存放位置，然后点击next。例如我的测试为：
   
   • Application name: MyLibApp
   • Company Domain: www.jt.com
   • Project location: D:\Projects\androidstudio\MyLibApp
3. 在Phone and Tablet的Minimum SDK选择API 21: Android 5.0 (Lollipop)，点击Next；
4. 选择Empty Activity，点击Next；
5. Customize the Activity 页面保持默认设置，点击Finish。

创建Android Library

• 依次点击File->New->New Module…；
• 选择Android Library，点击Next；
• 设置Application/Library name(例如： Tesscv)，点击Finish.

配置Android Library

• 修改Android视图下，Project的build.gradle，再点击右上角附近的Sync Now；

allprojects {
    repositories {
        jcenter()
        maven {
            url  "http://dl.bintray.com/steveliles/maven"
        }
    }
}

• 修改Android视图下，tesscv的build.gradle，再点击右上角附近的Sync Now；

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:23.4.0'
    testCompile 'junit:junit:4.12'
    compile 'com.rmtheis:tess-two:6.1.1'
    compile 'org.opencv:OpenCV-Android:3.1.0'
}

Sync的过程可能会比较慢，需要下载tess-two和OpenCV-Android的aar文件，依据网速不同，需要等待10分钟左右。

---

修改tesscv相关文件

在Android视图下，右击tesscv->java->com.jt.www.tesscv文件夹，选择New->Java Class，在弹出的Create New Class对话框的Name后面输入tesscv，点击OK。

• tesscv.java

package com.jt.www.tesscv;

import android.graphics.Bitmap;
import android.os.Environment;
import android.util.Log;

import com.googlecode.tesseract.android.TessBaseAPI;

import org.opencv.android.Utils;
import org.opencv.core.CvException;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

/**
 * Created by Administrator on 2016/12/2.
 */

public class tesscv {
    private final static String TAG = "TessCV";
    private Bitmap m_phone;                      // The path of phone image
    private TessBaseAPI m_tessApi;               // Tesseract API reference
    private String m_datapath;                   // The path to folder containing language data file
    private final static String m_lang = "eng";  // The default language of tesseract
    private InputStream m_instream;

    public tesscv(Bitmap phone, InputStream instream) {
        m_phone = phone;
        m_instream = instream;

        /// initial tesseract-ocr
        m_datapath = Environment.getExternalStorageDirectory().toString() + "/MyLibApp/tesscv/tesseract";
        // make sure training data has been copied
        checkFile(new File(m_datapath + "/tessdata"));

        m_tessApi = new TessBaseAPI();
        m_tessApi.init(m_datapath, m_lang);
        // 设置psm模式
        //m_tessApi.setPageSegMode(TessBaseAPI.PageSegMode.PSM_SINGLE_BLOCK);
        // 设置白名单
        //m_tessApi.setVariable(TessBaseAPI.VAR_CHAR_WHITELIST, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz");
        //m_tessApi.setVariable(TessBaseAPI.VAR_CHAR_WHITELIST, "0123456789");
    }


    private void saveTmpImage(String name, Mat image) {
        Mat img = image.clone();
        if (img.channels() ==3 ) {
            Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2RGBA);
        }

        Bitmap bmp = null;
        try {
            bmp = Bitmap.createBitmap(img.cols(), img.rows(), Bitmap.Config.ARGB_8888);
            Utils.matToBitmap(img, bmp);
        } catch (CvException e) {
            Log.d("mat2bitmap", e.getMessage());
        }
        File mediaStorageDir = new File(Environment.getExternalStorageDirectory(), "MyLibApp/tesscv");
        if (!mediaStorageDir.exists()) {
            if (!mediaStorageDir.mkdirs()) {
                Log.d("saveTmpImage", "failed to create directory");
                return;
            }
        }
        //String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
        //File dest = new File(mediaStorageDir.getPath() + File.separator + name + timeStamp + ".png");
        File dest = new File(mediaStorageDir.getPath() + File.separator + name + ".png");
        FileOutputStream out = null;
        try {
            out = new FileOutputStream(dest);
            bmp.compress(Bitmap.CompressFormat.PNG, 100, out);
            // bmp is your Bitmap instance
            // PNG is a lossless format, the compression factor (100) is ignored
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (out != null) {
                try {
                    out.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }


    public String getOcrOfBitmap() {
        if (m_phone == null) {
            return "";
        }

        Mat imgBgra = new Mat(m_phone.getHeight(), m_phone.getWidth(), CvType.CV_8UC4);
        Utils.bitmapToMat(m_phone, imgBgra);
        Mat imgBgr = new Mat();
        Imgproc.cvtColor(imgBgra, imgBgr, Imgproc.COLOR_RGBA2BGR);
        Mat img = imgBgr;
        saveTmpImage("srcInputBitmap", img);
        if (img.empty()) {
            return "";
        }
        if (img.channels()==3) {
            Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2GRAY);
        }
        return getResOfTesseractReg(img);
    }


    private String getResOfTesseractReg(Mat img) {
        String res;
        if (img.empty()) {
            return "";
        }
        byte[] bytes = new byte[(int)(img.total()*img.channels())];
        img.get(0, 0, bytes);
        m_tessApi.setImage(bytes, img.cols(), img.rows(), 1, img.cols());
        res = m_tessApi.getUTF8Text();
        return res;
    }


    private void checkFile(File dir) {
        //directory does not exist, but we can successfully create it
        if (!dir.exists() && dir.mkdirs()){
            copyFiles();
        }
        //The directory exists, but there is no data file in it
        if(dir.exists()) {
            String datafilepath = dir.toString() + "/eng.traineddata";
            File datafile = new File(datafilepath);
            if (!datafile.exists()) {
                copyFiles();
            }
        }
    }


    private void copyFiles() {
        try {
            if (m_instream == null) {
                //TODO
                String resInPath = "/tessdata/eng.traineddata";
                //Log.d(TAG, "copyFiles: resInPath " + resInPath);
                m_instream = new FileInputStream(resInPath);
            }

            //location we want the file to be a
            String resOutPath = m_datapath + "/tessdata/eng.traineddata";

            //open byte streams for writing
            OutputStream outstream = new FileOutputStream(resOutPath);

            //copy the file to the location specified by filepath
            byte[] buffer = new byte[1024];
            int read;
            while ((read = m_instream.read(buffer)) != -1) {
                outstream.write(buffer, 0, read);
            }
            outstream.flush();
            outstream.close();
            m_instream.close();

        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

---

修改app相关文件

• 在Android视图下，右击app文件夹，选择New->Folder->Assets Folder， 点击Finish；

• 右击新建的assets文件夹，选择New->Directory，在弹出的对话框中，输入新建文件夹名称为tessdata！ 一定要是tessdata！！！

• 右击新建的tessdata文件夹，选项Show in Explorer，点击进入tessdata文件夹内。
  

  tessdata文件夹用于存放训练好的语言数据集合，可以从 这里下载，本文采用 eng.traineddata。将下载好的 eng.traineddata文件放到 tessdata文件夹内！

添加app依赖库tesscv

• 点击File->Project structure…；
• 在弹出框左边Modules下选择app；
• 右边选择Dependencies；
• 再在最右边点击+，选择Module Dependecy；
• 在弹出框中，点击:tesscv，点击OK；
• 最后，在Project Structure对话框中，就可以看到新添加的:tesscv文件，然后点击OK。

AndroidManifest.xml/activity_main.xml/MainActivity.java

依次修改Android视图下，app文件内的AndroidManifest.xml、activity_main.xml和MainActivity.java三个文件。

• app->manifests->AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.jt.www.mylibapp">

    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

• app->res->layout->activity_main.xml

<?xml version="1.0" encoding="utf-8"?>

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:orientation="vertical"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity"
    android:weightSum="1">

    <Button
        android:id="@+id/photo_album"
        android:text="PhotoAlbum"
        android:layout_height="50dp"
        android:layout_width="match_parent" />


    <ImageView
        android:id="@+id/imageID"
        android:layout_width="match_parent"
        android:layout_height="360dip" />

    <TextView
        android:id="@+id/OCRTextView"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:text="OCR Text will appear here..."
        android:textSize="18dip"
        android:layout_centerVertical="true"
        android:layout_centerHorizontal="true"
        android:textColor="#a3a3a3" />
    <!--android:background="#dedede"-->

</LinearLayout>

• app->java->com.jt.www.mylibapp->MainActivity.java

package com.jt.www.mylibapp;

import android.content.Intent;
import android.content.res.AssetManager;
import android.graphics.Bitmap;

import android.net.Uri;
import android.os.Bundle;
import android.provider.MediaStore;
import android.support.v7.app.AppCompatActivity;

import android.view.View;
import android.widget.Button;
import android.widget.ImageView;
import android.widget.TextView;

import com.jt.www.tesscv.tesscv;

import java.io.IOException;
import java.io.InputStream;

import org.opencv.android.OpenCVLoader;


public class MainActivity extends AppCompatActivity {
    public static final String IMAGE_UNSPECIFIED = "image/*";
    public static final int PHOTOALBUM = 1;   // 相册
    Button photo_album = null;                // 相册
    ImageView imageView = null;               // 截取图像
    TextView textView = null;                 // OCR 识别结果

    Bitmap m_phone;                           // Bitmap图像
    String m_ocrOfBitmap;                     // Bitmap图像OCR识别结果
    InputStream m_instream;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        imageView = (ImageView) findViewById(R.id.imageID);
        photo_album = (Button) findViewById(R.id.photo_album);
        textView = (TextView) findViewById(R.id.OCRTextView);
        photo_album.setOnClickListener(new View.OnClickListener(){
            @Override
            public void onClick(View v) {
                Intent intent = new Intent(Intent.ACTION_PICK, null);
                intent.setDataAndType(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, IMAGE_UNSPECIFIED);
                startActivityForResult(intent, PHOTOALBUM);
            }
        });

        //get access to AssetManager
        AssetManager assetManager = getAssets();
        //open byte streams for reading/writing
        try {
            m_instream = assetManager.open("tessdata/eng.traineddata");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        if (resultCode == 0 || data == null) {
            return;
        }
        // 相册
        if (requestCode == PHOTOALBUM) {
            Uri image = data.getData();
            try {
                m_phone = MediaStore.Images.Media.getBitmap(getContentResolver(), image);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        // 处理结果
        imageView.setImageBitmap(m_phone);
        if (OpenCVLoader.initDebug()) {
            // do some opencv stuff
            tesscv jmi = new tesscv(m_phone, m_instream);
            m_ocrOfBitmap = jmi.getOcrOfBitmap();
        }
        textView.setText(m_ocrOfBitmap);
        super.onActivityResult(requestCode, resultCode, data);
    }
}

真机测试截图

小米5标准版，测试截图如下所示：

三、在新项目中引用tesscv库

找到tesscv生成的release版本的tesscv-release.aar文件

按照上图打开tesscv-release.aar文件所在目录，将tesscv-release.aar重命名为tesscv-1.0.0.aar

将tesscv-1.0.0.aar复制到新建Android Studio项目，Project视图下的app->libs文件夹内；

修改Android视图下，Project的build.gradle，再点击右上角附近的Sync Now；

allprojects {
    repositories {
        jcenter()
        maven {
            url  "http://dl.bintray.com/steveliles/maven"
        }
    }
}

修改Android视图下，app的build.gradle，再次点击右上角附近的Sync Now；

• 在dependencies前添加：

allprojects {
    repositories {
        jcenter()
        flatDir {
            dirs 'libs'
        }
    }
}

• 修改dependencies部分为：

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:23.4.0'
    testCompile 'junit:junit:4.12'

    compile(name:'tesscv-1.0.0', ext:'aar')
    compile 'com.rmtheis:tess-two:6.1.1'
    compile 'org.opencv:OpenCV-Android:3.1.0'
}

添加训练的语言库

参考上述工程中添加tesseract-ocr语言库eng.traineddata方法。

调用实例代码

// 导入包
import com.jt.www.tesscv.tesscv;
import org.opencv.android.OpenCVLoader;

// 调用OpenCV代码
if (OpenCVLoader.initDebug()) {
    // do some opencv stuff
    tesscv jmi = new tesscv(m_phone, m_instream);
    String ocrResult = jmi.getOcrOfBitmap();
}

参考

1. Simple OCR Android App Using Tesseract Tutorial

2. How to manually include external aar package using new Gradle Android Build System - Stack Overflow

3. 拍照+相册选取+剪裁图片，不过百行代码搞定 - JavAndroid - 博客频道 - CSDN.NET

---

1. OpenCV-Android ↩
2. Tess-two ↩