[opencv+qt]利用光流法实现通过移动摄像头使显示画面移动

番茄炒蛋o.O

已于 2024-02-18 21:52:26 修改

阅读量662

点赞数 11

文章标签： opencv qt python

于 2024-02-18 21:49:25 首次发布

本文链接：https://blog.csdn.net/renshiye1/article/details/136142692

版权

一，主要介绍

本文主要通过python+opencv对摄像头获取画面进行处理，并将移动信息发送给qt端同步处理以控制显示界面的移动

硬件：树莓派4B 4GB，摄像头

二，实现流程

1，获取坐标

通过角点检测选择出该画面中最明显的角点，将此点的作为坐标零点，移动画面时，只要该角点仍处于画面中则不会丢失，通过光流法来计算此点偏移时的坐标。（若画面移动过快会重新检测角点，因为重新检测的角点大概率会与前一点的坐标有较大偏差，所以可通过代码实现，角点更换造成的坐标大幅度改变不会移动显示界面，此段在qt端实现）

#设置Lucas-Kanade光流法的参数
lk_params = dict(winSize=(15, 15), maxLevel = 100, criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

color = np.random.randint(0, 255, (100, 3))

ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)

#在初始帧上使用Shi-Tomasi角点检测方法提取特征点。这里设置提取最好的1个角点用于提供坐标变化
p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, maxCorners=1, qualityLevel=0.3, minDistance=7)
while True:
            ret, frame = cap.read()
            frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

            mask = np.zeros_like(frame)

            if p0 is not None:
                p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

                if p1 is not None:
                    good_new = p1[st == 1]
                    good_old = p0[st == 1]

                    for i, (new, old) in enumerate(zip(good_new, good_old)):
                        a, b = new.ravel()
                        c, d = old.ravel()
                        a, b, c, d = map(int, (a, b, c, d))
                else:
                    p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, maxCorners=1, qualityLevel=0.3, minDistance=7)
                    
                    #计算新旧帧之间的光流，从而获取新的特征点位置。如果新的特征点存在，则更新特征点的坐标；如果不存在，则重新检测特征点。
                    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
                    good_new = p1[st == 1]
                    good_old = p0[st == 1]
                    
                    for i, (new, old) in enumerate(zip(good_new, good_old)):
                        a, b = new.ravel()
                        c, d = old.ravel()
                        a, b, c, d = map(int, (a, b, c, d))
            old_gray = frame_gray.copy()
            coordinates = f"X:{a}, Y:{b}"
            #发送坐标
            client_socket.send(coordinates.encode())
            p0 = good_new.reshape(-1, 1, 2)

2，通信：发送坐标与接收

借助socket通过本地回环地址127.0.0.1的端口来实现进程间通信，（端口选择随意，不过建议偏大一点）。

python（server服务端）

import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("127.0.0.1", 9999))
server_socket.listen(1)
print("wait for connect...\n")
client_socket, _ = server_socket.accept()
print("connected\n")

qt（client客户端）

在qtcreator创建的工程文件中使用socket，需要在.pro中添加network库

QT       += network

之后可以正常使用socket的函数

QTcpSocket socket;
    socket.connectToHost("127.0.0.1", 9999);
    if(socket.waitForConnected())
    {
        qDebug()<<"connect successfully!"<<endl;
        while(socket.state() == QAbstractSocket::ConnectedState)
        {
            if(socket.waitForReadyRead())
            {
                QByteArray data = socket.readAll();
                QString coordinates = QString(data);
                Position_trans(coordinates);
                //qDebug()<<"get the position:X:"<<X<<"  Y:"<<Y<<endl;
                emit send_position(X, Y);
            }
        }
    }

由于python端发送的坐标格式为：“X:{a}, Y:{b}“，所以需要提取字符串a和b并转换为整型或浮点型

（可按照自己的发送格式自行编写转换函数）

void Get_Pos_Socket::Position_trans(QString str)
{
    bool ok;
    QString x,y,temp,pos;
    int len = str.length();
    int i;
    temp = str.right(len - 2);
    for(i = 0; i < len-2; i++)
    {
        if(temp[0] == ',')
        {
            break;
        }
        temp = str.right(len-2-(i+1));
    }
    pos = str.mid(2,i);
    Get_Pos_Socket::X = pos.toDouble(&ok);
    pos = str.right(len-i-3-3);
    Get_Pos_Socket::Y = pos.toDouble(&ok);
}

由于坐标的接收应该是一个持续性的过程，所以接收坐标我认为应该另开一个线程单独运行。

三，根据坐标移动画面

1，建立测试画面

由于坐标是二维的，所以利用qt的组件制作一个可二维移动的画面即可，我是用qscrollarea加其他widget制作而成，简单如下

在获取坐标时就提及过，角点的大幅度位移是角点的更换导致的，不应作为使得画面移动的依据，因此，我们可以使用简单的偏移判断来控制，设定一个偏移阈值，当坐标偏移量大于阈值时，说明属于角点的更换，不必移动画面，小于阈值时则正常移动。简单显示效果如下

2，实现代码

产生坐标完整代码如下

#need key's number : 2 (release camera and get camera again)
import cv2
import numpy as np
import socket
import time

cap = cv2.VideoCapture(0)

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 320)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 240)
cap.set(cv2.CAP_PROP_FPS, 10)
#use when the ui is done, so does the blows
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("127.0.0.1", 9999))
server_socket.listen(1)
print("wait for connect...\n")
client_socket, _ = server_socket.accept()
print("connected\n")

time.sleep(3)

lk_params = dict(winSize=(15, 15), maxLevel = 100, criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

color = np.random.randint(0, 255, (100, 3))

ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)

p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, maxCorners=1, qualityLevel=0.3, minDistance=7)

while True:
    ret, frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    mask = np.zeros_like(frame)

    if p0 is not None:
        p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

        if p1 is not None:
            good_new = p1[st == 1]
            good_old = p0[st == 1]

            for i, (new, old) in enumerate(zip(good_new, good_old)):
                a, b = new.ravel()
                c, d = old.ravel()
                a, b, c, d = map(int, (a, b, c, d))
        else:
            p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, maxCorners=1, qualityLevel=0.3, minDistance=7)
            p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)
            good_new = p1[st == 1]
            good_old = p0[st == 1]
                    
            for i, (new, old) in enumerate(zip(good_new, good_old)):
                a, b = new.ravel()
                c, d = old.ravel()
                a, b, c, d = map(int, (a, b, c, d))
        #print(a,b)
    coordinates = f"X:{a}, Y:{b}"
    client_socket.send(coordinates.encode())
    old_gray = frame_gray.copy()
    p0 = good_new.reshape(-1, 1, 2)

    if (cv2.waitKey(1) & 0xFF == 27):
        break
#time.sleep(1)
#cap.release()
#cv2.destroyAllWindows()