Playback.PlaybackSettings Introduction

In aprevious post, Abhishek had talked about the Record and Playback Engine in VSTT 2010. The Coded UI Test feature in VSTT 2010 also uses this engine to playback UIactions.  In this post, i will describe the various settings for Playbackand how you can modify them to improve the resilience of your Coded UI Tests.Playback is configured by modifying the fields in Playback.PlaybackSettingsclass.

 

1. Continue on Error

Whenyou are recording a Coded UI test on a login page, the “Auto Completepasswords” popup may have appeared.

Whenyou playback actions on the login page, this dialog will *not* come up. So theaction that you performed will cause an error. To improve test resilience, theRecord & Playback engine automatically marks this action as “Continue onError”.  When the test is being played back and the error occurs, theengine will continue on to the next action. Specific actions (IE popups,implicit Hovers) when performed by the user are tagged by the engine as“Continue on Error”.

 

InCoded UI Test, you have the flexibility to turn on “Continue on Error” for anyaction. This is done by the following code snippet.

Playback.PlaybackSettings.ContinueOnError= true;

Rememberto turn this flag to false after the section where there are potentialfailures.  Otherwise the engine will ignore all errors and your test willreturn incorrect results. By default this flag is set to false in Coded UITest.

 

2. Match Exacthierarchy

TheRecord and Playback Engine uses a hierarchical structure to locatecontrols.  The Yes button in the dialog above is identified as

Tolocate YesButton during playback, the engine will search forAutoCompletePasswordWindow, then for YesWindow in it, and finally YesButton inYesWindow.

Sometimesapplication may change so that the hierarchy gets modified. Record and Playbackengine attempts to locate the control, in such cases, by skipping theintermediate elements in the hierarchy. Thus in the example, if it is unable tolocate YesWindow, it will search for YesButton directly underAutoCompletePasswordWindow and all its children.  This behavior can becontrolled by the “Match Exact Hierarchy” setting. This is done by thefollowing code snippet.

Playback.PlaybackSettings.MatchExactHierarchy= true;

Ifthe flag is set to true, every control is the hierarchy will be searched inorder. Any failure will result in failure to locate the control.

Bydefault, this flag is set to false. Note that this may sometimes lead to falsepositives i.e an unintended control may be identified. In such cases, you willhave to change this setting.

 

3. Search inminimized windows

TheRecord and Playback Engine searches for controls even inside minimized windows.The minimized window is restored and actions performed on the control insideit.

Youhave the ability to turn this feature off i.e. prevent the engine from lookingat minimized windows.  This is done by the following code snippet.

Playback.PlaybackSettings.SearchInMinimizedWindows= false;

 

4. Smart Match

TheRecord and Playback Engine identifies controls on the user interface by itssearch properties.  e.g:- The standard calculator window is identified asbelow.

 

SearchProperties[WinProperties.Window.ControlType]= ControlType.Window.Name;

SearchProperties[WinProperties.Window.Name]= "Calculator"; 
SearchProperties[WinProperties.Window.ClassName] = "CalcFrame";

Someof the search properties may change over time. Record and Playback engine usesa Smart Match algorithm to identify controls if they cannot be located usingthe exact properties. The smart match algorithm uses heuristics and attempts tolocate the control using variations on the search properties.

Youcan control when Smart Match should be applied. By default Smart Match isapplied for Top Level Windows and all controls. You can turn off Smart matchusing the following code snippet.

Playback.PlaybackSettings.SmartMatchOptions= SmartMatchOptions.None;

Thedefault settings (Smart match top level windows and controls) is optimal for aresilient coded UI test. But sometimes it may lead to false positives and thenyou will have to modify this setting.

 

5. Wait For ReadyLevel

TheRecord and Playback engine ensures that each UI control is ready to be actedupon before performing any action.  The smart “Wait For Ready” algorithm significantly improves the resilience of your Coded UI Test. Now you don’t need to add the annoying Sleep statements whenever a UI Controlis busy and not ready to receive input. By default, the engine checks the UIThread (foreground thread) to determine if a control is ready.

 

Youcan turn off Wait For Ready completely by the following code snippet.

Playback.PlaybackSettings.WaitForReadyLevel= WaitForReadyLevel.Disabled;

 

Alternatelyyou may want to wait for all threads including background threads. This may berequired if the application is making asynchronous calls that results in UIchanges.  You can do this by the following code snippet.

Playback.PlaybackSettings.WaitForReadyLevel= WaitForReadyLevel.AllThreads;

NOTEthat this setting will slow down the engine and should be carefully used.

 

6.  Set PropertyVerification

Aftersetting the property of any UI control, the record and playback engine performsa verification pass to ensure that the set succeeded and the UI control now hasthe value assigned to it.

 

e.g:- You type the text “MSFT” in the search edit box in Bing. After playing backthis action, the engine reads the text property of the search edit box toensure that this set has succeeded.

 

Incertain applications, the set property may trigger off some business logicwhich then changes that property itself. In such cases, the verification fails.If your application exhibits such a behavior you can turn of the set propertyverification using the following code snippet.

Playback.PlaybackSettings.SkipSetPropertyVerification= true;

 

7.  Fail Fast

TheRecord and playback engine’s search algorithm has a fail fast logic. i.e. Ifthe engine is reasonably confident that the search will not succeed, it willreturn with a failure. This ensures that you are not waiting for a non-existentcontrol for 2 mins (the default search timeout).

 

Sometimesyou may want to disable fail fast. Typically, you do this along with increasingyour search timeout. You are confidant that the UI control will becomeavailable within a specified duration  (which is longer than the searchtimeout) and you want the engine to wait for that duration.  You can dothis by using the following code snippet.

Playback.PlaybackSettings.ShouldSearchFailFast= false;

 

8. Misc

Otherplayback settings which can be modified include

a. Delay Between Actions: How long should the engine wait before performing twosuccessive actions? By default, 100 ms.

b. Search timeout – How long should the engine attempt to locate a control? Bydefault, 2 mins.

c. Wait for Ready Timeout – How long should the engine continue waiting for acontrol to be ready?  By Default, 60s

d.Think Time multiplier – For slow machines, assign a high multiple so that therecorded think time is scaled up to handle slower application responses.

e.Send Keys As ScanCode - By default, the engine sends keys to the applicationunder test as Unicode. But in certain cases, it may need to send keys asscancode. Sending Unicode to such applications will result in failure

 

NOTE:The Test runner tool in VSTT 2010 also uses the Record and Playback Engine. Youcan configure the playback settings described above in Test runner tool from

a.mtlm.exe.config – located in %VSINSTALLDIR%\Common7\IDE

b. Microsoft Test & Lab Manager, navigate toLab Center –> Test Settings , open the applicable Test Settings, select Dataand Diagnostics section and click on the Configure button next to Action Logand Action Recording. In the Advanced section of the dialog, some of thePlayback settings described above may be configured.