使用 PowerShell 自动化图像识别与鼠标操作

前言

在日常工作中，我们有时需要进行一些自动化操作，比如在屏幕上找到特定图像并执行鼠标点击操作。本文将介绍如何使用 PowerShell 实现这一功能。

功能概述

本文提供的脚本包括以下功能：

读取配置文件，获取需要操作的图像路径、鼠标按键类型以及延迟时间。
截取屏幕并寻找特定图像的位置。
在找到图像后模拟鼠标点击。
根据配置文件中的延迟时间，执行下一次操作。

代码实现

1. 引入必要的程序集

Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing

2. 定义读取文件行的函数

function Read-FileLines {
    param (
        [string]$filePath
    )
    return Get-Content -Path $filePath
}

3. 定义加载图片的函数

function Load-Image {
    param (
        [string]$imagePath
    )
    return [System.Drawing.Image]::FromFile($imagePath)
}

4. 定义查找小图像在大图像中的位置的函数

function Find-ImagePosition {
    param (
        [System.Drawing.Bitmap]$bigImage,
        [System.Drawing.Bitmap]$smallImage
    )
    # 查找逻辑...
    return $null
}

5. 定义截取全屏的函数

function Capture-Screen {
    $screenWidth = [System.Windows.Forms.SystemInformation]::VirtualScreen.Width
    $screenHeight = [System.Windows.Forms.SystemInformation]::VirtualScreen.Height
    $bitmap = New-Object System.Drawing.Bitmap($screenWidth, $screenHeight)
    $graphics = [System.Drawing.Graphics]::FromImage($bitmap)
    $graphics.CopyFromScreen(0, 0, 0, 0, [System.Drawing.Size]::new($screenWidth, $screenHeight))
    $graphics.Dispose()
    return $bitmap
}

6. 定义模拟鼠标点击的函数

Add-Type @"
using System;
using System.Runtime.InteropServices;

public class User32 {
    [DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
    public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);

    public const int MOUSEEVENTF_MOVE = 0x0001;
    public const int MOUSEEVENTF_LEFTDOWN = 0x0002;
    public const int MOUSEEVENTF_LEFTUP = 0x0004;
    public const int MOUSEEVENTF_RIGHTDOWN = 0x0008;
    public const int MOUSEEVENTF_RIGHTUP = 0x0010;
    public const int MOUSEEVENTF_ABSOLUTE = 0x8000;
}
"@
function Simulate-Click {
    param (
        [int]$x,
        [int]$y,
        [int]$button
    )
    [System.Windows.Forms.Cursor]::Position = [System.Drawing.Point]::new($x, $y)
    switch ($button) {
        1 { 
            [User32]::mouse_event([User32]::MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0)
            [User32]::mouse_event([User32]::MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)
        }
        2 {
            [User32]::mouse_event([User32]::MOUSEEVENTF_RIGHTDOWN, 0, 0, 0, 0)
            [User32]::mouse_event([User32]::MOUSEEVENTF_RIGHTUP, 0, 0, 0, 0)
        }
    }
}

7. 定义主函数

function Run {
    param (
        [string]$configPath
    )
    $configLines = Read-FileLines -filePath $configPath
    while(1){
        foreach ($line in $configLines) {
            $parts = $line -split ' '
            $imagePath = $parts[0]
            $button = [int]$parts[1]
            $delay = [int]$parts[2]
            $smallImage = Load-Image -imagePath $imagePath
            while(1){
                $bigImage = Capture-Screen
                $position = Find-ImagePosition -bigImage $bigImage -smallImage $smallImage
                if ($position -ne $null) {
                    Write-Output "Click Position $position"
                    Simulate-Click -x $position.X -y $position.Y -button $button
                    break
                } else {
                    Write-Output "Not Find $imagePath"
                    Start-Sleep -Milliseconds 500
                }
            }
            Start-Sleep -Seconds $delay
            Write-Output "Time-Sleep $delay"
        }
    }
}
Run -configPath "./conf.dd"

配置文件示例

image1.png 1 5
image2.png 2 3

配置文件中，每一行代表一组操作，依次为图像路径、鼠标按键（1为左键，2为右键）和延迟时间（秒）。

运行脚本

# 如果遇到执行权限问题，请运行以下命令解除限制
Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process

# 执行脚本
Run -configPath "./conf.dd"

结语

通过以上脚本，我们可以实现自动化图像识别与鼠标点击操作，大大提高了工作效率。希望本文对您有所帮助。如果您有任何问题或建议，欢迎留言讨论。

全部代码

Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing

# 读取指定位置的文件，返回按行分割的string列表
function Read-FileLines {
    param (
        [string]$filePath
    )
    return Get-Content -Path $filePath
}

# 读取指定位置的图片，返回图片的指针
function Load-Image {
    param (
        [string]$imagePath
    )
    return [System.Drawing.Image]::FromFile($imagePath)
}

# 查找小图片在大图片中的位置，返回小图片在大图片中的位置（中心点）
function Find-ImagePosition {
    param (
        [System.Drawing.Bitmap]$bigImage,
        [System.Drawing.Bitmap]$smallImage
    )

    $bigData = $bigImage.LockBits([System.Drawing.Rectangle]::FromLTRB(0, 0, $bigImage.Width, $bigImage.Height), [System.Drawing.Imaging.ImageLockMode]::ReadOnly, $bigImage.PixelFormat)
    $smallData = $smallImage.LockBits([System.Drawing.Rectangle]::FromLTRB(0, 0, $smallImage.Width, $smallImage.Height), [System.Drawing.Imaging.ImageLockMode]::ReadOnly, $smallImage.PixelFormat)

    try {
        for ($x = 0; $x -le $bigData.Width - $smallData.Width; $x++) {
            for ($y = 0; $y -le $bigData.Height - $smallData.Height; $y++) {
                $found = $true
                for ($i = 0; $i -lt $smallData.Width; $i++) {
                    for ($j = 0; $j -lt $smallData.Height; $j++) {
                        $bigColor = [System.Drawing.Color]::FromArgb([System.Runtime.InteropServices.Marshal]::ReadInt32($bigData.Scan0, (($y + $j) * $bigData.Stride) + ($x + $i) * 4))
                        $smallColor = [System.Drawing.Color]::FromArgb([System.Runtime.InteropServices.Marshal]::ReadInt32($smallData.Scan0, ($j * $smallData.Stride) + $i * 4))
                        if ($bigColor.ToArgb() -ne $smallColor.ToArgb()) {
                            $found = $false
                            break
                        }
                    }
                    if (-not $found) { break }
                }
                if ($found) {
                    return [System.Drawing.Point]::new($x + [math]::Round($smallData.Width / 2), $y + [math]::Round($smallData.Height / 2))
                }
            }
        }
    } finally {
        $bigImage.UnlockBits($bigData)
        $smallImage.UnlockBits($smallData)
    }

    return $null
}

# 截取全屏，返回指针
function Capture-Screen {
    $screenWidth = [System.Windows.Forms.SystemInformation]::VirtualScreen.Width
    $screenHeight = [System.Windows.Forms.SystemInformation]::VirtualScreen.Height
    $bitmap = New-Object System.Drawing.Bitmap($screenWidth, $screenHeight)
    $graphics = [System.Drawing.Graphics]::FromImage($bitmap)
    $graphics.CopyFromScreen(0, 0, 0, 0, [System.Drawing.Size]::new($screenWidth, $screenHeight))
    $graphics.Dispose()
    return $bitmap
}

# 模拟鼠标点击移动，传入x，y
Add-Type @"
using System;
using System.Runtime.InteropServices;

public class User32 {
    [DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
    public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);

    public const int MOUSEEVENTF_MOVE = 0x0001;
    public const int MOUSEEVENTF_LEFTDOWN = 0x0002;
    public const int MOUSEEVENTF_LEFTUP = 0x0004;
    public const int MOUSEEVENTF_RIGHTDOWN = 0x0008;
    public const int MOUSEEVENTF_RIGHTUP = 0x0010;
    public const int MOUSEEVENTF_ABSOLUTE = 0x8000;
}
"@

function Simulate-Click {
    param (
        [int]$x,
        [int]$y,
        [int]$button
    )

    [System.Windows.Forms.Cursor]::Position = [System.Drawing.Point]::new($x, $y)
    switch ($button) {
        1 { # 左键
            [User32]::mouse_event([User32]::MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0)
            [User32]::mouse_event([User32]::MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)
        }
        2 { # 右键
            [User32]::mouse_event([User32]::MOUSEEVENTF_RIGHTDOWN, 0, 0, 0, 0)
            [User32]::mouse_event([User32]::MOUSEEVENTF_RIGHTUP, 0, 0, 0, 0)
        }
    }
}

# 主函数
function Run {
    param (
        [string]$configPath
    )

    $configLines = Read-FileLines -filePath $configPath
    while(1){
        foreach ($line in $configLines) {
            $parts = $line -split ' '
            $imagePath = $parts[0]
            $button = [int]$parts[1]
            $delay = [int]$parts[2]
            $smallImage = Load-Image -imagePath $imagePath

            while(1){
                
                $bigImage = Capture-Screen
                $position = Find-ImagePosition -bigImage $bigImage -smallImage $smallImage
                if ($position -ne $null) {
                    Write-Output "Click Position $position"
                    Simulate-Click -x $position.X -y $position.Y -button $button
                    break
                } else {
                    Write-Output "Not Find $imagePath"
                    Start-Sleep -Milliseconds 500
                }
            }
            Start-Sleep -Seconds $delay
            Write-Output "Time-Sleep $delay"
        }
    
    
    }
    
}


 Run -configPath ”./conf.dd“
# Set-ExecutionPolicy -ExecutionPolicy Bypass -Scope Process
# 示例使用

提示

打包exe

Install-Module -Name ps2exe -Scope CurrentUser
Invoke-ps2exe -InputFile .\run.ps1 -OutputFile .\run.exe

下载地址

传送门