1

This question is a follow up to my previous question, which was a bit general, so I shall be more specific with this one.

I want to automate acceptance testing on a web application. Briefly, this application allows the user to create contracts for subscribers with two constraints:

  • You cannot create more than one contract for a subscriber.
  • Once a contract is created, it cannot be deleted (from the UI).

Let's say TestCreate is a test case with tests for the normal creation of a contract.

The constraints have introduced complexities to the testing process, mainly dependencies between test cases and test executions.

  • Before we run TestCreate we need to make sure that the application is in a suitable state (the subscriber has no contract)
  • If we run TestCreate twice, the second run will fail since the state of the application will have changed.
  • So we need to revert back to the initial state (i.e. delete the contract), which is impossible to do from the UI.

More generally, after each test case we should guarantee that the state is reverted back. And since, in this case, it is impossible to do it from the UI, how do you handle this?

Possible solution: I thought about doing a backup of the database in the state that I desire, and after each test case, run a script which deletes the db and restores the backup. However, I find that to be too heavy to do for each single test case. In addition, what if some information are stored in files? or in multiple or unaccessible databases?

My question: In this situation, what would an experienced tester do to write automated and maintanable tests. Thank you.

More info: I'm trying to integrate tests into a BDD (Behaviour Driven Development) framework, which I find to be a neat solution for test documentation and communication, but it does not solve this particular problem (it even makes it harder)

Improve this question
4
  • I would create a new subscriber each time, create a contract for that subscriber, and then delete the subscriber at the end of the test. Now you can run two tests at once because they will work with different subscribers.
    – btilly
    Feb 19, 2011 at 16:31
  • You should put your system in a known state before executing the integrations test. This means populating your database accordingly by inserting or removing records.
    – Martin Wickman
    Feb 20, 2011 at 0:04
  • @btilly: As you can't delete the subscriber from the UI, how would you do it? Delete records directly from the database?
    – H-H
    Feb 20, 2011 at 17:31
  • You said you can't delete contracts. You didn't say you couldn't delete subscribers. Not being able to get rid of bad data from the UI sounds like a bad UI decision.
    – btilly
    Feb 20, 2011 at 17:48

2 Answers 2

Reset to default
1

How about creating a bunch of virtual machine's with snapshots of the desired states, then automating your testing in these? VMware offers an API (VIX) for automating virtual machine operations.

Improve this answer
0

I don't understand why BDD solutions fall short here. Cucumber seems to fit the bill. You set up your system with the "Given" constuct and write your matchers (steps) to set the system up according to those constraints.

Scenario: No contracts
Given a subscriber named "HH" that has no contract
When I create a contract
Then I should see "Contract Created!"

Scenario: Has a contract
Given a subscriber named "HH" that has one contract
When I create a contract
Then I should see the error "HH already has a contract!"

...

If I recall correctly, cucumber resets the system after each scenario. I think you are short-changing BDD frameworks...

Improve this answer
2
  • How does cucumber "resets" the system? is it a hook that you add? But the problem is that I can't delete a contract from the UI, and it might very hard to do it directly in the database.
    – H-H
    Feb 20, 2011 at 17:33
  • Cucumber, by default, uses database transactions and these transactions are rolled back after each scenario.
    – Derek Prior
    Feb 20, 2011 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.