Ranorex automation test qt

Enabling automation for custom QT widgets by adding Accessibility

Posted by Tobias Walter on Thursday, December 9th, 2010 at 1:31 pm to Improve Object Recognition

To make automation available for QT applications, an accessibility interface is provided by QT, which is based on different technologies for each platform QT is available on. In our case MSAA (Microsoft Active Accessibility) is used when working on a Microsoft Windows platform.

QT implements accessibility for most of its built-in widgets, but if you want to add this feature to your own widgets, you have to implement it yourself.

To illustrate the steps that have to be implemented to make a custom widget ready for automation, we want to extend an example widget which shows an analog clock. To provide accessibility for our clock widget, we need to implement an accessible interface for it and send accessible events from the widget when the time changes.

The sample project can be found at “AnalogClock.zip“.

• QT Accessibility Architecture
• Implementing Accessibility
• Connecting Accessibility to our Custom Widget
• Let’s spy it out

QT Accessibility Architecture

The whole structure of the user interface you want to make accessible (“automatable”) is represented by a tree of accessible objects. All of these objects are derived from the class QAccesibleInterface which provides a wide range of predefined information (role, action and relation) that can be defined for these objects. (See QAccessible Class Reference)

So in our example the sub-tree architecture should look something like this:

Let’s have a look at this tree structure and the relationship between those tree items. We have the clock itself which is mapped as “clock”. And as child elements of the clock we have both the minute hand and the hour hand mapped as “slider”.

As you can see above, our clock consists of three accessible objects: the clock itself, as well as the hour and minute hands. We use an enum ClockElements to identify them. The code for this enumeration is shown later when we start to implement our accessibility interface for the clock widget.

 

Implementing Accessibility

To provide accessibility for our clock widget, we need to implement an accessible interface for it and send accessible events from the widget when the time changes. We will first take a look at the implementation of the interface.

Let’s start with the header file:

view plain copy to clipboard print

1. class AccessibleClock : public QAccessibleWidget  
2. {  
3. public:  
4.   
5.     enum ClockElements {  
6.         ClockSelf = 0,  
7.         HourHand,  
8.         MinuteHand  
9.     };  
10.   
11.     AccessibleClock(QWidget *widget, Role role = Client,  
12.         const QString & name = QString());  
13.   
14.     int childCount() const;  
15.     QRect rect(int child) const;  
16.     QString text(Text text, int child) const;  
17.     Role role(int child) const;  
18.   
19. private:  
20.     AnalogClock *clock() const  
21.     { return qobject_cast(widget()); }  
22.   
23. };  

We extend the QAccessibleWidget class which inherits from QAccessibleInterface and helps us handling relationships and keeps track of events, roles, texts and bounding rectangles. Accessible interfaces for all subclasses of QWidget should use QAccessibleWidget as their base class.

Custom interfaces must implement states and give appropriate strings for the Text enum. This is implemented by the text and state functions. Our clock consists of three accessible objects: the clock itself, as well as the hour and minute hands. As described above we use the enum ClockElements to identify them. We also have to define the number of children held by our clock. That can be done with the function childCount. To define the bounding rects of our accessibility objects the function rect is used.

So let’s have a look at the implementation of all of those functions mentioned above, starting with the function which returns the number children our clock element has:

view plain copy to clipboard print

1. int AccessibleClock::childCount() const  
2. {  
3.     return 2;  
4. }  

We return “2″, one for the hour hand and one for the minute hand.

The next function provides the bounding rectangles of our widget:

view plain copy to clipboard print

1. QRect AccessibleClock::rect(int child) const  
2. {  
3.     QRect rect;  
4.     QPoint topLeft = clock()->mapToGlobal(QPoint(0,0));  
5.     switch (child) {  
6.           case ClockSelf:  
7.               rect = clock()->rect();  
8.               break;  
9.           case HourHand:  
10.               rect = clock()->hourHandRect();  
11.               break;  
12.           case MinuteHand:  
13.               rect = clock()->minuteHandRect();  
14.               break;  
15.           default:  
16.               return QAccessibleWidget::rect(child);  
17.     }  
18.     return QRect(topLeft.x() + rect.x(), topLeft.y()  
19.         + rect.y(), rect.width(), rect.height());  
20. }  

As you can see, the child enumeration (or index) is an argument of  the rect() function, so we have to decide which rectangle to return. The rects of the clock and their hands are provided by our analog clock implementation. We only have to calculate the screen coordinates of our rectangles before returning them.

After that we want to define role and text of our analog clock and its child objects:

view plain copy to clipboard print

1. QAccessible::Role AccessibleClock::role(int child) const  
2. {  
3.     switch (child) {  
4.         case ClockSelf:  
5.             return Clock;  
6.         case HourHand:  
7.         case MinuteHand:  
8.             return Slider;  
9.         default:  
10.             ;  
11.     }  
12.     return QAccessibleWidget::role(child);  
13. }  

We have decided to use the role clock for our analog clock and the role slider for the clocks hands. You can find the specific roles at QAccessible::Role.

Now we have to define the values for description, name and value for each of our accessibility objects:

view plain copy to clipboard print

1. QString AccessibleClock::text(Text text, int child) const  
2. {  
3.     if (!widget()->isVisible())  
4.         return QString();  
5.   
6.     switch (text) {  
7.         case Description:  
8.             switch (child)  
9.             {  
10.                 case ClockSelf:  
11.                     return "an Analog Clock";  
12.                 case HourHand:  
13.                     return "a Hour Hand";  
14.                 case MinuteHand:  
15.                     return "a Minute Hand";  
16.             }  
17.         case Name:  
18.             switch (child)  
19.             {  
20.                 case ClockSelf:  
21.                     return "Analog Clock";  
22.                 case HourHand:  
23.                     return "Hour Hand";  
24.                 case MinuteHand:  
25.                     return "Minute Hand";  
26.             }  
27.         case Value:  
28.             switch (child)  
29.             {  
30.                 case ClockSelf:  
31.                     return clock()->currentTime.toString();  
32.                 case HourHand:  
33.                     return QString("%1").arg(clock()->currentTime.hour());  
34.                 case MinuteHand:  
35.                     return QString("%1").arg(clock()->currentTime.minute());  
36.             }  
37.   
38.         default:  
39.             return QString();  
40.     }  
41.     return QString();  
42. }  

After implementing an accessibility interface for our analog clock, we have to somehow inform the accessibility object about changes of state (in our case, when time passes). Therefore we have to hook in the timeout function of our analog clock which is connected to a timer which fires every minute. Here we have to call updateAccesibility to inform our accessibility objects about the new values:

view plain copy to clipboard print

1. void AnalogClock::timeout()  
2. {  
3.     currentTime = QTime::currentTime();  
4.     QAccessible::updateAccessibility(this, 0, QAccessible::ValueChanged);  
5.     update();  
6. }  

 

Connecting Accessibility to our Custom Widget

At this point we have implemented our accessibility interface and we update our state whenever the minute or hour hands move. The next step is to connect the accessibility implementation with the widget itself. This can be done by either implementing an accessibility plug-in or by implementing an interface factory. A plug-in is a class stored in a shared library that can be loaded at run-time. In our example, we choose to implement the interface factory. An explanation of how to implement the plug-in can be found at Accessibility in Qt.

To implement the factory we extend our main.cpp for the factory implementation and in the main function we are installing this factory:

view plain copy to clipboard print

1. QAccessibleInterface *clockFactory(const QString &classname, QObject *object)  
2. {  
3.     QAccessibleInterface *interface = 0;  
4.   
5.     if (classname == "AnalogClock" && object && object->isWidgetType())  
6.         interface = new AccessibleClock(static_cast<QWidget *>(object));  
7.     return interface;  
8. }  
9.   
10. int main(int argc, char *argv[])  
11. {  
12.     QApplication a(argc, argv);  
13.     QAccessible::installFactory(clockFactory);  
14.     AnalogClock analogClock;  
15.     analogClock.show();  
16.     return a.exec();  
17. }  

Our factory is a function pointer for a function that takes two parameters, a QString and a QObject. We check whether the widget an accessible interface is requested for is actually an AnalogClock. If it is, we create and return our custom interface for it.

 

Let’s spy it out

Now we have done everything needed to make our clock accessible. Let’s give it a try. Just compile your project and use Ranorex Spy on the application. You will see that our clock now will have a name, description, role and of course a value. The same applies to the two hands.

As you can see from our example, your custom QT widgets will no longer be an unautomatable black blox if you spend some time implementing basic accessibility. For more information on QT Accessibility, visit Accessibility in Qt.