My boss has three websites and each of those websites have forms on them that people can sign up for nightclubs.
He asked me to look up usability tests, however, I don't think that's what hes actually looking for. In fact, he wants the forms tested frequently (for instance once per day) to make sure they are working properly.
Is usability testing the right tool for that? If not, what is the right way to do it?
You're right, usability testing has a very different purpose. It consists of asking a user to manipulate a part of your product, and look at the way the user tries to achieve something. The goal is to ensure that user experience of your product is right and if there are things which could be improved to make the product easier, more intuitive or faster to use.
What are you looking for is regression testing. Regression testing is about checking whether things which worked before still work after the code was changed. Smoke tests, functional tests, unit tests or system tests are all part of regression testing.
Depending of what are you precisely testing, the type of tests—and the tools you will use—will differ. For instance, a form on a website can be tested in several ways:
You may test the underlying code using unit tests. Unit tests are short pieces of code which ensure a very short piece of code in your code base (usually a method, although sometimes it might be a whole class) works as expected.
For example, you may have a method which checks that the user entered an e-mail address which looks valid, and use a bunch of unit tests which call this method with e-mail addresses you know to be valid or invalid and check whether the result of the method matches the expectation.
You may also test whether the form itself can be submitted, or that the web app returns an error (such as HTTP 403) if the form contains invalid entries. This is done with system tests and will consist of doing a POST request with specific data, and compare the resulting HTTP status code. Such test, when sending valid input, may even query the database to check that the associated record was created successfully.
You may also test whether the HTML form itself works as expected (for instance, if you inadvertently change the action attribute of the form, it won't work any longer). This is the domain of system and functional testing, with tools such as Selenium which may help you to automate the tests.
Finally, you may need to ensure that the form is showing exactly as it was shown before the latest changes, that is you have a pixel perfect match of the page. For example, adding two pixels to a padding will shift the entire form, eventually breaking the layout.
Since such changes are often barely noticeable when simply looking at the page, you can use pdiff to check for visual differences.
Those four modes of testing ensure that you have a high probability of finding regressions. This being said, there are two aspects which are extremely important if you want regression testing to be effective:
Every test should be automated. Always. No exceptions. You don't have to run tests by hand, by filling the form fields, clicking on buttons and checking whether the result is the same as required by the test. Machines are much better than us at repetitive tasks; they do it in milliseconds instead of hours, and they don't do mistakes.
Tests should run as frequently as possible. Once per day, as you suggested, might be acceptable for some projects, but why can't you do better and run all those tests after every commit? Running tests more often ensures that you find regressions sooner, when you still remember what you're changing right now. If you changed a class and five minutes later, your build server notifies you about a regression, you'll probably find the bug faster than if you receive the notification the next morning, after spending six hours doing dozens of other changes.
