Flaui使用说明

小禾家的

已于 2023-04-28 17:49:50 修改

阅读量3.5k

点赞数 2

文章标签： ui c#

于 2023-04-28 17:40:38 首次发布

本文链接：https://blog.csdn.net/u011234288/article/details/130429511

版权

简介

FlaUI 是一个 .NET 库，可帮助对 Windows 应用程序（Win32、WinForms、WPF、Store Apps 等）进行自动化 UI 测试。
它基于 Microsoft 的本机 UI 自动化库，因此是它们的包装器。
FlaUI 包装了 UI Automation 库中的几乎所有内容，但也提供了本机对象，以防有人有 FlaUI 尚未涵盖的特殊需求。
源代码网址:http://www.github.com/Roemer/FlaUI
FlaUI源代码文档:https://github.com/FlaUI/FlaUI/wiki

FlaUI官方介绍讲分两个版本

UIA2：原生 UI 自动化 API 的托管库
- UIA2 是只管理的，这对 C# 来说很好，但它不支持更新的功能（如触摸），而且它也不能很好地与 WPF 一起工作，甚至更糟糕的是与 Windows 应用商店应用程序一起工作。
UIA3：原生 UI 自动化 API 的 Com 库
- UIA3 是最新的，非常适合 WPF/Windows 商店应用程序，但不幸的是，它可能有一些 WinForms 应用程序的错误（请参阅常见问题解答），这些错误在 UIA2 中不存在。

尝试下来：winform界面尽量用UIA2，其他用UIA3，并且同一APPDomain只能存在一个对象，也切换时需要重启。

一、获取窗体

获取窗体有时候最困难，你可能遇到且不限于：类名重复、类名变化、窗体名重复、窗体父子关系但是父窗体设置成MDI，甚至有的窗体实际上是个Panle等等诸如此类。

获取窗体分为四种模式：

为什么要分为四种模式呢？写RPA脚本的时候要根据窗体的层级关系选择合适的代码，查找的执行效率和查询的范围有关，预设的范围越小，查询的效率越快。这几种模式下边会一一介绍。

        [Description("当前进程主窗体")]
        ByMain,
        [Description("当前进程主窗体的子窗体")]
        ByMainChild,
        [Description("当前进程所有弹出窗体")]
        ByAllTopLevel,
        [Description("当前桌面所有弹出窗体")]
        ByAllDesktop,

获取的条件：

获取的条件也要根据实际情况进行选择，有些窗体类名一致、窗体名不同，有些类名窗体名都不同。值的注意的是XPath组件内部自己生成的类似xml结构的固定格式，执行效率很高，建议获取元素的时候使用。

       [Description("类名")]
        ClassName,
        [Description("窗体名")]
        Title,
        [Description("AutomationId")]
        AutomationId,
        [Description("XPath(窗体不可用)")]
        XPath,
        [Description("类名和窗体名")]
        ClassNameAndTitle,
        [Description("类名或窗体名")]
        ClassNameOrTitle,

1.ByMain

要捕获的窗体只有一个、或者有父子关系时的父窗体（模态框）。

注意：非模态框不能用此模式

 System.Diagnostics.Process[] processes = System.Diagnostics.Process.GetProcessesByName(processName);//获取进程
 var id = processes.First().Id;
 var app = FlaUI.Core.Application.Attach(id);
//new TimeSpan(1) 不能缺省 否则会假死
 var mainWindow = appl?.GetMainWindow(AutomationBase, new TimeSpan(1));

2.ByMainChild

要捕获的窗体是父子关系时的子窗体（模态框）

 System.Diagnostics.Process[] processes = System.Diagnostics.Process.GetProcessesByName(processName);//获取进程
 var id = processes.First().Id;
 var app = FlaUI.Core.Application.Attach(id);
//new TimeSpan(1) 不能缺省 否则会假死
 var mainWindow = appl?.GetMainWindow(AutomationBase, new TimeSpan(1));
 var property = model.WindowProperty;
 var controlType = model.ControlType;
 var frameworkType = model.FrameworkType;
 var className = model.ClassName;
 var name = model.Title;
 var automationId = model.AutomationId;
  mainWindows = FindWindowByAllChildren(mainWindow, property, controlType, frameworkType, className, name, automationId);
 private static AutomationElement[] FindWindowByAllChildren(AutomationElement windows,
            PropertyType property, ControlType controlType, FrameworkType frameworkType,
            string className, string name, string automationId)
        {
            AutomationElement[] mainWindows = null;
            switch (property)
            {
                case PropertyType.ClassName:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.Title:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByName(name).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.AutomationId:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByAutomationId(automationId).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameAndTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).And(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameOrTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).Or(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                default:
                    break;
            }
            return mainWindows;
        }

3.ByAllTopLevel

当一个进程会弹出多个窗体并且要捕获的窗体是非模态框（和主窗体同级）

 System.Diagnostics.Process[] processes = System.Diagnostics.Process.GetProcessesByName(processName);//获取进程
 var id = processes.First().Id;
 var app = FlaUI.Core.Application.Attach(id);
//new TimeSpan(1) 不能缺省 否则会假死
 var windows = appl?.GetAllTopLevelWindows(AutomationBase).ToList();
//下边又分两种模式一种是
//主窗体A 窗体B和主窗体A同级 需要的是窗体B此时走isheet =0
//主窗体A 窗体B和主窗体A同级 窗体C是窗体B的子窗体（模态）isheet>0
//此处应该循环
 var property = model.WindowProperty;
                    var controlType = model.ControlType;
                    var frameworkType = model.FrameworkType;
                    var className = model.ClassName;
                    var name = model.Title;
                    var automationId = model.AutomationId;
                    if (isheet == 0)
                    {
                        mainWindows = FindWindowByAllTopLevelWindows(windows, property, className, name, automationId)?.ToList();
                    }
                    else
                    {
                        mainWindows = FindWindowByAllChildren(mainWindow, property, controlType, frameworkType, className, name, automationId).ToList();
                    }
                    if (mainWindows?.Count() != 1)
                    {
                        var txt = $"{className},{name},{automationId}";

                        msg = $"第{isheet}层\r\n{GetDescription(windowtype)}|通过条件：{GetDescription(property)}值：{txt}\r\n查询到窗体个数{mainWindows.Count()}不唯一," +
                            $"请尝试切换其他模式重试，" +
                            $"重试全部失败时，需定制插件";
                        break;
                    }
                    mainWindow = mainWindows?.FirstOrDefault();



  private static AutomationElement[] FindWindowByAllTopLevelWindows(List<Window> windows,
            PropertyType property,
            string className, string name, string automationId)
        {
            List<Window> mainWindows = null;
            //主窗体和子窗体同级时
            //简单方式（常用）
            switch (property)
            {
                case PropertyType.ClassName:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAll(cf => cf.ClassName.Equals(className));
                        break;
                    }
                case PropertyType.Title:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAll(cf => cf.Name.Equals(name));
                        break;
                    }
                case PropertyType.AutomationId:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAll(cf => cf.AutomationId.Equals(automationId));
                        break;
                    }
                case PropertyType.ClassNameAndTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAll(cf => cf.ClassName.Equals(className) && cf.Name.Equals(name));
                        break;
                    }
                case PropertyType.ClassNameOrTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAll(cf => cf.ClassName.Equals(className) || cf.Name.Equals(name));
                        break;
                    }
                default:
                    break;
            }
            return mainWindows?.ToArray();
        }
  
 private static AutomationElement[] FindWindowByAllChildren(AutomationElement windows,
            PropertyType property, ControlType controlType, FrameworkType frameworkType,
            string className, string name, string automationId)
        {
            AutomationElement[] mainWindows = null;
            switch (property)
            {
                case PropertyType.ClassName:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.Title:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByName(name).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.AutomationId:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByAutomationId(automationId).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameAndTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).And(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameOrTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).Or(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                default:
                    break;
            }
            return mainWindows;
        }

4、ByAllDesktop

效率最慢，当进程窗体很多时巨慢我本机试的1-3s

                 var property = model.WindowProperty;
                    var controlType = model.ControlType;
                    var frameworkType = model.FrameworkType;
                    var className = model.ClassName;
                    var name = model.Title;
                    var automationId = model.AutomationId;
                    mainWindows = FindWindowByAllChildren(windows, property, controlType, frameworkType, className, name, automationId);
                    windows = mainWindows?.FirstOrDefault()?.AsWindow();
        
        private static AutomationElement[] FindWindowByAllChildren(AutomationElement windows,
            PropertyType property, ControlType controlType, FrameworkType frameworkType,
            string className, string name, string automationId)
        {
            AutomationElement[] mainWindows = null;
            switch (property)
            {
                case PropertyType.ClassName:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.Title:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByName(name).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.AutomationId:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByAutomationId(automationId).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameAndTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).And(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                case PropertyType.ClassNameOrTitle:
                    {
                        //桌面的直接子窗口
                        mainWindows = windows?.FindAllChildren(cf => cf.ByClassName(className).Or(cf.ByName(name)).
                       And(cf.ByControlType(controlType).And(cf.ByFrameworkType(frameworkType))));
                        break;
                    }
                default:
                    break;
            }
            return mainWindows;
        }

二、获取元素

1.通过Xpath获取推荐使用

类似于文件路径，这是个相对路径，可以稍微理解一下，下付转换代码

 //通过XPath获取
 var childrens = window?.FindAllByXPath(xPath);
//转化Xpath
   public static string TranslationXpath(string xpath)
        {
            if (!xpath.ToLower().Contains("window"))
            {
                return xpath;
            }
            var eles = xpath.Split(new string[] { "/" }, StringSplitOptions.RemoveEmptyEntries);
            var split = "/";
            var res = "";
            for (int i = 1; i < eles.Length; i++)
            {
                res += split + eles[i];
            }
            return res;
        }
//通过类名窗体名查找
   switch (property)
                {
                    case PropertyType.ClassName:
                        {
                            if (controlType == ControlType.Unknown)
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]));
                            }
                            else
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]).And(cf.ByControlType(controlType)));
                            }
                            break;
                        }
                    case PropertyType.Title:
                        {
                            if (controlType == ControlType.Unknown)
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByName(value[1]));
                            }
                            else
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByName(value[1]).And(cf.ByControlType(controlType)));
                            }
                            break;
                        }
                    case PropertyType.AutomationId:
                        {
                            if (controlType == ControlType.Unknown)
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByAutomationId(value[2]));
                            }
                            else
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByAutomationId(value[2]).And(cf.ByControlType(controlType)));
                            }
                            break;
                        }
                    case PropertyType.ClassNameAndTitle:
                        {
                            if (controlType == ControlType.Unknown)
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]).And(cf.ByName(value[1])));
                            }
                            else
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]).And(cf.ByName(value[1])).And(cf.ByControlType(controlType)));
                            }
                            break;
                        }
                    case PropertyType.ClassNameOrTitle:
                        {
                            if (controlType == ControlType.Unknown)
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]).Or(cf.ByName(value[1])));
                            }
                            else
                            {
                                elements = window?.FindAllDescendants(cf => cf.ByClassName(value[0]).Or(cf.ByName(value[1])).And(cf.ByControlType(controlType)));
                            }
                            break;
                        }
                    default:
                        break;
                }

三、元素操作

我现在只用到了textbox、combox、checkbox、RadioButton之类的简单的，复杂的可以自行参照文档，注意有时候winform和wpf或者其他win32的界面同样的控件可能发送的写法不同，需要定制，我这里定义了个插件来定制化不能使用的界面，有需要的可以自行去除

  public static void SendText(string key, AutomationElement element, string txt)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.TextBox(element, txt);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var textbox = element?.AsTextBox();
                    if (textbox != null)
                    {
                        textbox.Text = txt;
                        //发送失败时尝试用粘贴板粘贴
                        if (string.IsNullOrEmpty(textbox.Text))
                            textbox.Enter(txt);
                    }
                }
            }
            catch (Exception ex) { throw ex; }
        }

        public static void ComboBoxSelectText(string key, AutomationElement element, string txt)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.ComboBox(element, txt);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var combox = element?.AsComboBox();
                    if (combox != null)
                    {
                        combox.Select(txt);
                        combox.Collapse();
                    }
                }
            }
            catch (Exception ex) { throw ex; }

        }
        public static void ComboBoxSelectIndex(string key, AutomationElement element, int index)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.ComboBox(element, index);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var combox = element?.AsComboBox();
                    if (combox != null)
                    {
                        try
                        {
                            combox.Select(index);
                            combox.Collapse();
                        }
                        catch (Exception ex)
                        {
                            throw ex;
                        }
                    }
                }
            }
            catch (Exception ex) { throw ex; }
        }

        public static void CheckBoxChecked(string key, AutomationElement element, bool isCheck)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.CheckBox(element, isCheck);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var check = element?.AsCheckBox();
                    if (check != null)
                        check.IsChecked = isCheck;
                }
            }
            catch (Exception ex) { throw ex; }

        }
        public static void RadioButtonCheck(string key, AutomationElement element, bool isCheck)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.RadioButton(element, isCheck);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var radioButton = element?.AsRadioButton();
                    if (radioButton != null)
                        radioButton.IsChecked = isCheck;
                }
            }
            catch (Exception ex) { throw ex; }

        }
        public static void RadioButtonClick(string key, AutomationElement element)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.RadioButton(element);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var radioButton = element?.AsRadioButton();
                    if (radioButton != null)
                        radioButton.Click();
                }
            }
            catch (Exception ex) { throw ex; }

        }

        public static void DateTimePicker(string key, AutomationElement element, DateTime dateTime)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.DateTimePicker(element, dateTime);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var picker = element?.AsDateTimePicker();
                    if (picker != null)
                        picker.SelectedDate = dateTime;
                }
            }
            catch (Exception ex) { throw ex; }

        }

        public static void Calendar(string key, AutomationElement element, DateTime dateTime)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.Calendar(element, dateTime);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var calendar = element?.AsCalendar();
                    if (calendar != null)
                        calendar.SelectDate(dateTime);
                }
            }
            catch (Exception ex) { throw ex; }

        }

        public static void Spinner(string key, AutomationElement element, double value)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.Spinner(element, value);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var spinner = element?.AsSpinner();
                    if (spinner != null)
                        spinner.Value = value;
                }
            }
            catch (Exception ex) { throw ex; }

        }
        public static void Slider(string key, AutomationElement element, int value)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.Slider(element, value);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var slider = element?.AsSlider();
                    if (slider != null)
                        slider.Value = AdjustNumberIfOnlyValue(slider, value);
                }
            }
            catch (Exception ex) { throw ex; }

        }
        public static void ButtonClick(string key, AutomationElement element)
        {
            try
            {
                var appl = GetApplication(key);
                var pElement = appl?.AutomationComponent?.Button(element);
                if (!(pElement.HasValue && pElement.Value))
                {
                    var button = element?.AsButton();
                    if (button != null)
                        button.Invoke();
                }
            }
            catch (Exception ex) { throw ex; }

        }

        public static void ListSelete()
        {

        }
          private static double AdjustNumberIfOnlyValue(Slider slider, double number)
        {
            if (slider.IsOnlyValue)
            {
                return number * 10;
            }
            return number;
        }

先写这么多吧，后边有空写下类似Spy++的工具的坑。。。

小禾家的

关注

2
点赞
踩
10

收藏

觉得还不错? 一键收藏
4
评论
Flaui使用说明

FlaUI 是一个 .NET 库，可帮助对 Windows 应用程序（Win32、WinForms、WPF、Store Apps 等）进行自动化 UI 测试。它基于 Microsoft 的本机 UI 自动化库，因此是它们的包装器。FlaUI 包装了 UI Automation 库中的几乎所有内容，但也提供了本机对象，以防有人有 FlaUI 尚未涵盖的特殊需求。FlaUI官方介绍讲分两个版本UIA2：原生 UI 自动化 API 的托管库。
复制链接

扫一扫