#Python –Detecting #Hololens in realtime in webcam feed using #ImageAI and #OpenCV (thanks to @Olafe

Hi!

Let’s start with a very quick intro:

During the past months, I’ve been playing around with several Image Analysis tools. And ImageAI (see references) is one that deserves a full series of posts. Please take a look at the product and the source code in GitHub, and also please thank the one behind this: Moses Olafenwa (@OlafenwaMoses).

And now, my 2 cents. I’ve started to test ImageAI to create my own image detection models. Most of the times, this is a hard path to do, however ImageAI show me an interesting option.

… with the latest release of ImageAI v2.1.0, support for training your custom YOLOv3 models to detect literally any kind and number of objects is now fully supported, …

Wow! That’s mean that I can pick up my own set of images dataset and train on top of a YOLOv3 and use it as a trained model. Again, this is amazing.

So, I started to read the article [Train Object Detection AI with 6 lines of code, see references] where Olafenwa explains how to do this using a data set with almost 500 rows with images for Hololens and Oculus Rift.

The code is very simple and easy to read. There are also examples on how to analyze a single file, or a video, or even a camera feed. The output for the analysis can be also in a new file, in a processed video or even a full log file with the detected information.

 

I started to read the code samples and I realized that I’m missing a scenario:

Display the realtime feed from a webcam, analyze each webcam frame and if a device is found, add a frame to the realtime feed to display this.

I use OpenCV to access to my camera, and it took me some time to figure out how to convert my OpenCV2 camera frame to the format needed by ImageAI. At the end, thanks to the GitHub code I manage to create this (very slow but working) demo

 

As usual in this scenario, now it’s time to improve the performance and start testing with some tweaks to get a decent up and running App.

And of course, the code

# load HL detection model from imageAI
# open camera with openCV, analyze frame by frame
# draw a red frame around the detected object

from imageai.Detection.Custom import CustomObjectDetection
import os
import cv2

detector = CustomObjectDetection()
detector.setModelTypeAsYOLOv3()
detector.setModelPath("hololens-ex-60--loss-2.76.h5") 
detector.setJsonPath("detection_config.json")
detector.loadModel()

# init camera
execution_path = os.getcwd()
camera = cv2.VideoCapture(0)

while True:

    # Grab a single frame of video
    ret, frame = camera.read()

    detected_image, detections = detector.detectObjectsFromImage(input_image=frame, input_type="array", output_type="array")

    for detection in detections:
        print(detection["name"], " : ", detection["percentage_probability"])
        (x1, y1, x2, y2) = detection["box_points"]
        print("x1: ", x1, " - y1: ", y1, " - x2: ", x2, " - y2: ", y2)
    
        # frame for the detected object
        cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 0, 255), 2)

        # Draw a label with the detected object type below the frame
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, detection["name"], (x1 + 6, y1 - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

• Image AI http://imageai.org/
• GitHub, ImageAI https://github.com/OlafenwaMoses/ImageAI
• Train Object Detection AI with 6 lines of code https://medium.com/deepquestai/train-object-detection-ai-with-6-lines-of-code-6d087063f6ff