How should I tackle code testing on a solo project?

Question

I am writing a piece of software and I would like to release it open source.

Should I try just getting it stable on my computer (with normal use; no "things a monkey wouldn't even do") before releasing it, or should I take a different approach?

---

What I am leaning towards doing right now is:

1. Write it myself during the pre-alpha stage to get it partially usable. Play around with the functions (like a normal person would).
2. Release it publicly and add some bells and whistles during the alpha stage
3. Say it's in beta once all features are complete, and then, along with some others (hopefully) do the "monkey testing."
4. Generate the publicity for my [fairly basic and boring, but still good enough for free and stable] project.

Then keep improving it. This seems to be a usual system (to me, the one who has never really done this whole thing ever before).

---

Is there something I could do to automatically test for basic errors before I go alpha? Or is this one of the best things I can do? How should I go about testing?

Answer 1

As I am somewhat in your situation (currently developing a PHP-based CMS) I hope that my experience can help you.

Testing is key. Repeat that until you believe. It depends on the programming language though how easy it is. As I am developing in PHP I will give some examples on that.

Developing

The most important point is: You develop the software in the first place for yourself. If you do something in your spare time, do it for yourself, do it for fun. Don't do it in the expectation of "I want that others contribute to the project". If you put up an empty codebase, nobody will contribute. For the first contributors you need a running demo, something that makes your software popular.

For that goal you need to develop the software in the first place.

• Create a feature list.
• Ask yourself: What should be in the product, what not?

This feature list is not holy. It will change during development and you won't be able to realize everything as planned. But it gives you a general direction, some goals to aim on.

Testing

Create software that is test-friendly. Use dependency injection, prevent singletons and use unit tests. At best use test driven development. That means you create the test first and than the tested class. That way you have to know the specification of your class before you write it. It saves a lot of pain later on and motivates contributors (they just have to test their own classes). Changes of implementation can simply be tested and if they fail, you can immediately resolve the errors.

Release

Announce the software only if it is working and without problems. But use Github or another code hosting site to give early contributors a chance to see the code. If they want, they can contribute. But you are the main developer of the first version.

Once the first version comes close to Beta (all components are mainly written and tested and work together), you can put up a public demo (showing off your product in case of web software). Collect feedback of the general users of the software (What is missing?, What can be done better?). That ensures that you don't miss the customer base. Your initial feature list will be concentrated on features developer focus on. That is natural and OK. But because of that you need this Beta feedback of your potential users.

Then go over the feedback, develop missing features, fix existing bugs, etc. Once everything works (automated tests like unit tests and functional tests will make that a lot easier) you can release your product.

Maintenance

Your job isn't done when it's out. When it's out your real job begins. Now you need to promote your product (both usage and code development) on your product's website. Write good developer documentation (in addition to the API documentation generated from code comments) that explains things that you won't find in API documentation (best practices, do's and don'ts, etc.).

Answer 2

Unit testing is what your project needs for several different reasons.

1. Allows you to test your application's basic functionality.
2. Allows you to ensure that new features are not causing regressions.
3. Good unit tests give consumers and maintainers an idea of what your application is capable of.
4. Good concise unit tests encourage you to write good concise code. After all, good concise code is easier to test.

Answer 3

Unit testing will make your initial development time increase. But in compensation you will avoid many of those stealthy emergent bugs as your system grows. Also, when you hit a certain level of project complexity your unit tests might start saving development time, since most coding will probably involve previous code and unit tests give you a pretty solid assurance in regards of not breaking anything that was previously working.

One more point in favor of coding unit tests is that it tends to result in better code since, in general, testable code is less coupled than hard to test code.

If it is a web application you're developing, then functional automated tests are a very good way to reverse test your application after some changes were made. I've used Selenium for this purpose before.