Scriptable user-interfaces/frameworks for automated UI testing

Question

I'm planning on using scripting for automated UI testing. Main application is written in c#, and I want it to be scriptable, so I can do everything end-user can do, but programmatically.

I'll avoid opinion-based questions like "What do you think of software that provides an interface for scripting, like VBA macros in Excel?", or "Can this be future of all programming, big and small?" although some inside would be great.

Here goes the question:

• Which is most suitable to the purpose of building such an interface for your own application, dll-based approach or by parsing own scripting language?

Answer 1

In my optionion gui-scripting is most valuable if it is combined with a gui recorder that translates the actions of a gui-user into a replayable script.

The problematic side of gui-scripting is that if the gui changes the scripts have to be updated, too.

if for example the logindialog changes and you have 150 tests that depend on login then in the worst case you have update 150 tests.

if you are fortunate you have modularized scripts with a central method "Login" that is used by all you test-scripts so only one script needs update.

but never the less don't underestimate maintanace-costs of automated test.

if you want to gui-script html-pages there might also be problems with javascript/ajax.

Update:

writing manual test-scripts is boring but using a script recorder leads to scripts that are hard to maintain. Often it is easier to record a new test then to update an existing :-(.

What i would do to automate html application tests using free software

• use Selenium for recording and generate c# code from it.
• use bdd - style human readable language to describe the testscenario you recorded (in order to ... as a ... i want ... : given ... when ... then....)
• use specflow to implement the bdd tests
• refactor the recorded/generated code into methods that fit into the bdd-scenario-steps. Common Steps(like the login example) can be shared by multible bdd-tests.

Note: I havenot done this solution yet but read a lot about and worked with two comercial recording products. If something is wrong with my selenium/specflow proposal please leave a comment.

If you know other free tools please update stackoverflow-s wiki most-useful-free-net-libraries

Answer 2

Since you're working in c#, why not just make sure your application honors the existing Microsoft automation framework ?

With that in place, you may be able to leverage existing tools.

If you do expose your own scripting methodologies:

• Don't create your own language. There are a lot of great languages that span any spectrum you may care about.
• Get a good set of use cases to understand what most users really want to control.
• Consider external access (coming in and going out). Often, people want to script a product to integrate it to another product.

Answer 3

If you are going to create a gui recorder, you should generate the script actions directly from your application. This way you are able to generate the most meaningful scripts and the scripting interface can be made user-friendly.

It is not an option to have some generic script recorder that listens to all UI events and tries to generate a script from them.

Answer 4

I don't think there's any good reason to invent your own scripting language. There are several languages that are designed specifically to be embedded in other applications (eg: tcl, lua). Save yourself the trouble of inventing yet another language and instead consider adapting one of those.