基于Kivy的伤口图像处理App的开发

Acknowledgement 致谢

FYP 项目基于 Fuh Ying Hsi 教授 FYP 项目：一款基于 AI 的 用于伤口愈合治疗的区域检测应用程序的开发。
感谢傅教授和卢教授在项目完成过程中给予的大力帮助。感谢NUS给予的机会。

伤口分析App使用记录

写这篇文章的目的是介绍FYP应用程序的功能。

1. Introduction 功能介绍

App主要包括四个功能，分别是：

• CV - 使用 传统计算机视觉方法分析伤口图像，
• DL - 使用 深度学习方法从原始图像中分割伤口区域，
• UI - 使用 Kivy Framework 设计UI 界面。
• More - 以及其他一些工具，例如 PDF 生成和通过电子邮件发送、天气和 AI 聊天机器人作为客户服务。

下面的内容将介绍主页面的分别的功能。

2. Login Verification: Updated at 2020/02/19

Version 0.1.0

The current login page is just an empty shell with no specific functions,while the aim is using Google authorization to achive the function of signing up and signing in.

The User are supposed to type the username and corresponding password to log in the app,the eyes on the right can be clicked to hide the password and the UI interface is illustrated as below:

2.1 Login Page

Updated at 2020/02/18

• Integration & Optimization of Functions such as Style Replacement - Add the original application style setting function to the setting page.

• Optimized Realization of Login Function - The original login interface has only been designed for the interface. This update adds the back-end database to realize the login function. At the same time, consider using third-party logins such as Github and Facebook to simplify the database structure and load. The existing interface is shown in the figure below. The user icon in the upper left corner can be used to create a new user and store it. At the same time, this page can jump to the password recovery interface to provide password recovery services for registered users.

• Add Verification or Error Prompt - Added prompts of incorrect types, including incorrect password, input user does not exist, etc. Set the login button to Progress Button to optimize the interface design and user experience.

Updated at 2020/02/25

• Optimization of Textfield -Adjust the position of the icon on the right of the textfield, design a new FloatLayout to adjust the icon on the right to the textfield, beautify the design, and increase the user’s operability.

• Optimization of Page Layout -Move the password recovery page to the Toolbar to reduce the layout items of the login page. Its icon icon is shown in the figure below. Click to jump to the password recovery page.
  

• Optimization of E-mail Typing -Add a list to visit the created users. Activate the form column at the bottom by clicking the button to the right of the first textfield. Further, the user can click the corresponding user name to automatically fill it in the first textfield, which simplifies the cumbersome login required Input content.

• Added Remember User Option -In order to reduce the tedious operation of entering a password every time a user logs in to the application, this update uses Checkbox to change the value of Remember (BOOLEAN) in the database, so that logged-in users can use this option to avoid repeated password input. On this basis, by double-clicking the text box in the password field, the corresponding user login password can be automatically filled in.The updated page is shown below:
  

Updated at 2020/03/02

• Adding Login Via Github -Users can log in through their personal GitHub account, which reduces the time it takes for users to register a new account. If you successfully log in through your Github account, record the Github value (BOOLEAN) as True in the database. On this basis, the password reset option of the personal page is optimized: if you log in through Github, the password change function will be invalid, and the user needs to go to the official Github website to update or reset the password.
  

2.2 User Register Page

Updated at 2020/02/17

The addition of the user registration function, this function can create new user personal information based on the user name and the email address that the user fills in. After the creation is completed, you can return to the login page to log in.And the UI interface is illustrated as the figure below:

2.3 Password Retrieve Page

Updated at 2020/02/18

This function is designed to take into account the possibility of users forgetting their passwords, enter the email used during registration to retrieve and reset the password, and then log in. The design of its interface is shown in the figure below.

3. Adding Patient Information

The App is designed with the ability to store patients information via sqlite which is the Python’s built-in database.

The user can click the Adding button on the navigation drawer to jump to the specified page,which is demonstrated as the figure below.

The required information are:Patient’s Name,Age,Gender,Creation Date.The creation date can be choosed by using date-picker,where you should double click the icon button.
　　Meanwhile,this page can be skipped directly and jump to Upload Page or Home.
　　At present stage,the data is collected in database,I am considering use Firebase to replace it in further work,as fire-base can loose the burden on the local storage.

4. Home Page

You can get to the home page if you have finished creating patient information.And the tabhint is used to guide the user for activating navigation drawer.Meanwhile,the chips on the bottom of the home shows the related knowledge and skills.

The interface of the home page is showed as follow:

To jump to the corresponding tab, you can use the navigation drawer. In the figure below, the unique page implementation is shown.

Updated at 2020/02/14

The chips on the Home Page can be clicked and then you can view the relevant knowledge used in this App through the official website or Wikipedia,such as Python,Kivy,Json,etc.

The icon button on the right corner of the home page have the ability of activating the dialog with the QRcode,which you can scan to visit this personal blog.

5. Upload Wound Image & Analyze

This is one of the main tasks of the application,which contains the wound image upload.The core idea is to process and analyze images based on OpenCv through traditional computer vision methods, and then obtain the contour and specific characteristics of the wound.

More specifically, it mainly includes three functions: taking pictures by camera, uploading local pictures /taking pictures & wound pictures to the database, analyzing and processing wound pictures.

5.1 Camera

The camera function can be achieved by the camera on mobile phone,which can store the picture taken as the format "%Y%m%d%H%M%S.png",at present the picture is stored in local address,I am considering using cloud-database as well.

5.2 Upload Wound Image

Professional doctors will need to upload wound pictures to the database,thus this function aims to pop up a window to choose file from album in mobilephone.

Updated at 2020/03/05

The former format for the user uploaded image is "uploadimage/input.png",which is stored at the local address.After optimization,the COS is taken into practical account.The new format for the uploaded image is showed in the code below:

    create = App.get_running_app().screen_manager.get_screen('Create')
    PatientName = create.children[0].ids.PatientName.text.lstrip()
    upload_image_file_path = "uploadimage/{}_input.jpg".format(PatientName)

Meanwhile,the code used for uploading image to the COS is also showed as below,the more detailed information can be found at Tencent Official Site of COS:

    response=client.upload_file(
        Bucket='wound-1301658428',
        LocalFilePath=upload_image_file_path,  # Path to local file
        Key=upload_image_file_path,  # File name after uploading to the bucket
    )

5.3 Analyze Wound Image

This module is implemented as the main function of the application, which can realize the identification of the wound contour and the analysis of the corresponding wound characteristics.
　　After uploading the picture, the back-end will recognize it, and display the uploaded original picture and the processed picture on the page. On this basis, The corresponding report can be generated and sent to the mailbox.An example usage is demonstrated as follow:

Note:If the user click the E-mail Button,the application will send an e-mail with an attachment containing Wound Analysis Report to designated mailbox(for example:fhlielts8@gmail.com).Please refer to the PDF Generator for detailed completion steps and source code.

A demo of the attachment is demonstrated as the figure below:

Updated at 2020/02/15

Optimize the file management pop-up window, beautify its appearance and increase its use functions, the nature of its asynchronous processing and wound image analysis has not changed.

Updated at 2020/03/01

Optimized the source of the patient’s name, age, gender, etc. for wound analysis details. Originally, it was set to determine the patient’s name to obtain the detailed information obtained from the analysis. Therefore, only one line of information for the same patient can be displayed. This information has been analyzed in the previous analysis. Stored in the patient information of the corresponding patient. Now it has been optimized by the following code. The realization idea is mainly based on the patient’s name as the identification, so as to retrieve the relevant information in the corresponding database and display it in the wound details page and the generated pdf file.

    def Name(self):
		# sql = "SELECT PatientName FROM Patient WHERE PatientName='FanHaolin'"
		# The above is the former code,which is childish
        create = App.get_running_app().screen_manager.get_screen('Create')
        PatientName = create.children[0].ids.PatientName.text.lstrip()
        sql = "SELECT PatientName FROM Patient WHERE PatientName='{}'".format(PatientName)
        res = select_data(sql)
        self.ids.name.text = str(res[0][0])

6. Wound Image Segmentation–Deep Learning

7. Patient List

This page is an important part of the realization of electronic medical records

The setting of this page is mainly to be able to access the patient’s medical records, which can be jumped by clicking the search button in the upper right corner of the Home Page, or by the History option in the Navigation Drawer. After jumping to this page, you can search for the patient’s name through the text box above to obtain detailed information about the patient. The page is displayed as follows:

Updated at 2020/02/26

• Search Funtion Optimization -This update adds a button click event corresponding to the patient’s name on the patient record page. Specifically, clicking the patient’s name can access its detailed medical history information. The analyzed medical history can be obtained through the Swipe Menu at the bottom of the sliding page. The next step is to add a new medical record modification function, which can be modified according to the doctor’s actual diagnosis. The current medical history interface is shown in the figure:

• Some Details Modification - When click on the corresponding patient name within the patient list,the search textfield will change into the patient’s name,and you can use the icon-button on the right side of the textfield to delete the current content of the textfield.

8. Weather

The original intention of the weather module is to be used as a small life tool to provide doctors with timely weather information to deal with possible weather conditions.

The temperature of the displayed area can be selected by yourself ,the default is (Singapore), and the automatic positioning function is to be developed[].

Note:The Weather funtion is supported by OpenWeatherMap API

Meanwhile,the button on the toolbar have the ability to back to home or refresh the widget.

9. Personal Setting

The personal setting page now contains name-chage-box,where the user can change the user name and avatar.Consider adding theme replacement function later.The the UI interface is illustrated as below:

9.1 Avatar Change

Updated at 2020/02/21

The user can change the user avatar with clicking on the current avatar,which will activate the avatar list,the user can randomly choose one by clicking the image in the image list.

9.2 Dark Mode

Updated at 2020/02/21

The dark mode is set to meet the needs of users in different environments, and the mode can be switched through the switch.The schematic diagram of switching dark mode is shown below:

9.3 Password Visibility

Updated at 2020/02/21

This setting option can be used to set the visibility of the password, currently this setting is mainly for the login verification interface.

9.4 Password Reset

Updated at 2020/02/21

This function provides the option of password reset for users who have successfully logged in. When the old password is inconsistent with the original password, the new password is stored in the corresponding user information in the database. This function also considers using FireBase as storage in subsequent iterations database.For specific operations, please follow the pop-up prompts and confirm.

9.5 Logout Current Account

The logout function will log out of the current login account and jump to the login page. This function provides the application with the function of independent use of multiple accounts and stores user information separately.

10. Help Page

The help page contains the chatbot,which has the ability to answer questions for the users.The chatbot can answer the questions like:

• How are you
• How to upload
• How to use AI
• How to add
• What are you doing
  　　The figure below shows some example usages of help page:
  

11. Exit App

To ensure the convenience of the user, the application has set up an exit button, and before the user confirms the exit, a pop-up prompt will be displayed to continue. For example, the following gif shows the specific operation process