AppleScript 做macOS UI自动化的,点击无效的排查记录
目标:点击下图中 CheckBox 元素
第一步:获取这个元素的位置,并打印出value,确认是开关是关的(value=0)再继续
set targetbox to checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1 of window "Login Items & Extensions"
set theValue to value of targetbox
第二步:点击目标元素
1. 用click 事件, 不生效,也无报错
set targetbox to checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1 of window "Login Items & Extensions"
if targetbox exists then
click targetbox
2. 用perform action “AXPress”, 不生效,也无报错
perform action "AXPress" of targetbox
3. 用osascript 执行AppleScript,报错10006, 猜测是安全限制问题
do shell script "osascript -e 'tell application \"System Events\" to tell process \"System Extensions (Login Items (System Settings))\" to set value of checkbox \"checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1\" of window \"Login Items & Extensions\" to 1'"
4. 直接改value 从0 → 1 , 不生效,也无报错
set value of targetbox to 1
5. 点击元素的坐标,无报错,不生效
try
tell checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1 of window "Login Items & Extensions"
set {xPosition, yPosition} to position
set {xSize, ySize} to size
end tell
-- modify offsets if hot spot is not centered:
click at {xPosition + (xSize div 2), yPosition + (ySize div 2)}
return {xPosition + (xSize div 2), yPosition + (ySize div 2)}
end try
6. 尝试点击其他按钮
点击同层级按钮的坐标,也不行
点击 上上上上层按钮【Done】,可以点
也就是说 Done 这层是可以操作的,Done兄弟的下下下下一层就不能操作了
以下是我的猜测:
从UI Browser中看到这个element 不能显示树状结构,且Application 也变成了 “LoginItems” 而非一开始的“Systems Settings”,所以是到了某个加密的元素了。
AppleScript 使用 macOS 提供的 API 来控制和自动化 macOS 应用程序。所以API不会破解macOS的加密元素。
而pyautogui 是跨平台的 GUI 自动化,尤其是基于屏幕坐标的操作,所以在AppleScript获取到元素位置后,再尝试用pyautogui.click()事件。
解决方法:
方法一:写click.py,其中包含pyautogui.click()方法,然后直接在AppleScript中run python click.py
方法二:在python文件中运行 AppleScript(以下方法中还包含根据元素名称点击元素的下一个button)
def enable_macOS15_extension(self,software_name):
log.info("enabling %s", software_name)
applescript = f"""
if major_version is greater than or equal to 15 then
delay 10
-- 获取目标窗口
set targetWindow to window "{self.get_L10N_text("LoginItems_Extensions")}"
-- 获取目标元素的所有兄弟元素(同层次元素)
set siblingElements to UI elements of group 3 of scroll area 1 of group 1 of group 2 of splitter group 1 of group 1 of targetWindow
-- 初始化变量
set targetElementFound to false
set nextElement to missing value
-- 遍历所有同层次元素,找到目标元素并获取其下一个元素
repeat with i from 1 to count of siblingElements
set currentElement to item i of siblingElements
if value of currentElement contains "{software_name}" then
set targetElementFound to true
else if targetElementFound then
set nextElement to currentElement
exit repeat
end if
end repeat
-- 点击目标元素的下一个元素的信息
if nextElement is not missing value then
click nextElement
delay 3
else
return "not found Antivirus Extension button"
end if
--点击弹窗中的CheckBox
set targetbox to checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1 of targetWindow
if targetbox exists then
if value of targetbox is 0 then
try
tell checkbox 1 of group 1 of scroll area 1 of scroll area 1 of group 1 of group 1 of sheet 1 of targetWindow
set {{xPosition, yPosition}} to position
set {{xSize, ySize}} to size
end tell
-- modify offsets if hot spot is not centered:
return {{xPosition + (xSize div 2), yPosition + (ySize div 2)}}
end try
else
return "CheckBox value is 1"
end if
else
return "Not Found CheckBox"
end if
else
return "Not macOS15"
end if
"""
wrappedscript = self.wrapper(applescript)
output = run_applescript(wrappedscript)
checkbox_1 = output.split(",")
x_position = int(checkbox_1[0])
y_position = int(checkbox_1[1])
log.info("CheckBox button at %s, %s",x_position, y_position)
pyautogui.click(x=x_position,y=y_position)
time.sleep(10)