0

I am very new to the concept of testing and I had not yet have any experience with this.

At my company, no one does write tests as of now, neither is any test framework set up or anything. Also, we don't have testers. You bet how testing works: send new version to one customer, then wait for his call... if he hasn't called in for 24 hours to report bugs, call him and ask, and if the program seems to work, send it to the other customers. And then, wait for their calls...

Now, you can guess how much I like this. In fact, it's down to two options by now: either I can change my company, or I can change to another company. For both, I could use some background knowledge on testing, which I am collecting right now. But where to start?

A friend told me the basics on how his company does automated tests each night - a status I want to establish in our company as well. I asked him to find out which frameworks they use for unit tests and web tests, and he will find out for me, but I would also like to ask you which ones are recommended or even de-facto standard.

What we deliver are ExtJS applications which do CRUD operations on data via C# backends into MSSQL databases.

So I guess I would have to look for

  • a webtest framework, which "clicks" through the browser/webapp
  • an extjs/javascript unittest framework,
  • a C# unittest framework (I will have a closer look at VisualStudio UnitTests on Monday)
  • perhaps a framework which tests whether web calls give correct return values, and how web calls react to "garbage in"?
  • perhaps even an sql unittest framework for stored procedures?

What exactly would I really need; and what would be best/most useful to start with? (from the accepted answer here I read "webtest" it is, but then, testing web calls could be easier to start with!?)

Are recommendable low-cost/no-cost frameworks available, or are there frameworks which provide a limited no-cost version, so I can try whether it suits my/our needs before I go to my boss for funding?

Where can I start finding information, are there good books on testing?

closed as off-topic by Telastyn, Michael Durrant, user40980, jwenting, Oded Oct 12 '14 at 7:22

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions asking us to recommend a tool, library or favorite off-site resource are off-topic for Programmers as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it." – Telastyn, jwenting, Oded
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    Way too broad. Try and ask one question at a time. – Michael Durrant Oct 12 '14 at 0:03
4

Here are a few suggestions to at least get you started. Everything listed here is free.

  • a webtest framework, which "clicks" through the browser/webapp - Selenium

  • an extjs/javascript unittest framework - Jasmine or Mocha

  • a C# unittest framework - NUnit

  • perhaps a framework which tests whether web calls give correct return values, and how web calls react to "garbage in"? - NUnit, implemented as tests calling the web services and verifying expected responses

  • perhaps even an sql unittest framework for stored procedures - NUnit, implemented as tests calling the stored procs and verifying the expected results (either from the stored proc output, or with follow-up queries to confirm expected data changes)

By the way, I worked at a company that was nearly clueless about automated/programmatic testing (testing was done manually or not at all). But I threw together an automated UI regression test using Selenium, and it blew them away. Sometimes people just need a good demonstration.

1

This is all general, not specific to your needs.

Random stuff I learned from the process of adding unit tests to my C++ project (not all from personal experience):

  • integration tests are not like unit tests. There are a lot of harnesses that only support unit tests, not integration tests. The same script that calls your test harness should also run a code formatter and error out if it makes a diff.
  • A properly-written make test will be able to only rerun tests if the relevant code has been changed. Hint: each test binary outputs a log file (more useful than a stamp file).
  • a good test framework supports process-level isolation and splits the responsibility of choosing tests and recording their results from actually running them. TAP, which originated from perl, is a good example of doing this and is usable for any language.
  • Use a freaking CI service with a matrix. This is useful even without any testsuite, to catch compiler errors that only occur with one compiler version.
  • If your matrix is too big, or if your CI bot is taking to long for a single matrix entry, maybe you have too many bundled libraries. Even if it's a library specifically written for this project, if it is isolated enough to be a separate repo with its own CI builds (which indirectly generate binaries for use in your main project's CI builds), it should be. If it is not isolated enough, make it so.
  • it's impossible to unit test in the presence of global variables. Kill them with fire. If killing them is too hard, at least isolate them into a single Globals globals object per library/binary. But libraries should never expose an API that relies on globals anyway.
  • realistically, you won't add tests to all your old code. Rather, add tests both when writing new code and when bug reports come in.
  • when writing new code, first quickly write some code so you get an idea of what you really need, then delete it all and write unit tests, then build an implementation as necessary to pass the unit tests. This ends up being a lot easier.
  • your tests can and will rely on implicit assumptions about the class/function they're testing, but the tested class/function will be changed by an idiot someday (probably you).

My general advice is: do not tie yourself to one particular technology. You have special needs, write a little code that works with your code instead of twisting around to fit in a framework. TAP is particular good at letting you write your own glue instead of forcing you to use legos, and even letting you call out to other test harnesses.

  • +1 for "realistically, you won't add tests to all your old code. Rather, add tests both when writing new code and when bug reports come in." I will cover the old code with a bunch of simple "does it throw JS errors" webtests, and as long as they pass, I won't write unit tests. Some of the other suggestions I don't understand as of now, but I'll take your post as a reference once I do... – Alexander Oct 12 '14 at 9:36

Not the answer you're looking for? Browse other questions tagged or ask your own question.