UI automation patterns and best practice for desktop applications

Question

Background

I'm currently automating some tests for a plugin for MS Office. We are creating Coded UI tests in VS 2010. I suppose I could use the "Coded UI test builder" tool, but it does not really suit my particular case.

Because of this I created my own UI Map class and extension methods for each UI Control/Map where I add different action functionality. For example press buttons or assert some UI values.

The scenarios of the test cases are in the test classes.

I am new in this area and also I'm new in working as a automation tester.

The question

Would people be kind enough to share their experience and advice for some good practices for test automation on desktop applications from a programming/design point of view?

Answer 1

The best practice for UI Automation Testing is to do as little as possible. UIs change frequently, which means you're constantly having to update your automation. It's generally preferable to structure the product code in a way which allows automated testing without UI Automation.

That said, you can't always get rid of UI Automation. You mention office so I'm assuming you're coding for Windows and using .Net. I do quite a bit in my current job. Here are some of the things I've learned.

1) Look at the UIAutomation libraries which were introduced in .Net 3.0. They provide an extensive and fairly simple to use library for automation. (http://msdn.microsoft.com/en-us/library/ms753107.aspx)

2) Download UISpy (http://msdn.microsoft.com/en-us/library/ms727247.aspx)

3) Make your product's UIs Automatable.

3a) If it's WPF put AutomationIDs on everything.

3b) Try to create distinctive control and window class names (UI Class names, not source code class name). If you don't know what I mean, load up UI Spy and start looking at windows. Notice how many windows across different apps have a class name of #32770. This is the class name for a Windows Dialog Box. Any window which extends the dialog and doesn't set it's own name, defaults to this. This causes all sorts of grief from an UI Automation standpoint.

4) Avoid Thread.Sleep() statements. Try to use Waiters (see UIAutomation docs) instead.

5) NEVER mix the test code with the UI Automation code. Create separate libraries to perform the UI Automation. Call these libraries from your tests. When the UI changes, this will make it much easier to update the automation.

6) Always register a listener for a UI Event before performing the action which would cause the event to fire. In practice, this means you'll be working with threads.

6a) Example: don't start waiting for a Window Opened event after you have clicked a button to open the window. The window may open before the waiter is registered and never get the event.

7) Never assume the window that just opened is the one you want. All sorts of window may open unexpectedly in Windows.

I could go on more, but this is getting a bit long.

Answer 2

Build functional tests from re-usable use cases

When the time comes to test your application end to end in-situ, you're performing functional testing. Typically you'll have a set of requirements that you're testing against and you'll be able to construct various use cases that represent them.

As an example, consider the "Login as standard user" use case. Your test framework fires up the application, waits for the login screen, enters some credentials, clicks the login button and verifies that the appropriate screen is now showing that login was successful.

After you've done the "Login as standard user" use case, you'll want to build on that to do something else, perhaps the "Edit my user details" use case. You won't want to repeat all the code from the "Login as standard user" use case, so you just make a reference to the test framework code that does that bit.

This implies that you have some kind of over-arching functional test containing a list of use cases. These use cases contain the test framework methods to cause application behaviour (click button X) and verify the behaviour (screen turned blue).

Overall, you can build up a body of re-usable use cases that target specific sequences and test for specific responses, and then aggregate them into various functional tests that correlate closely with the business requirements. Once you've got that in place, then you're in a great position to fully automate your entire build process.

If you're interested in reading further I've written about this approach elsewhere, but the article targets web applications in Java (using Maven and SeleniumRC) rather than desktop applications which you have requested.