Assuming that you're working on a piece of software which is required to pass a third party security/penetration test before being released to the client, at which point in the project life-cycle would you perform the tests?

Passing the test means that no major flaws are detected, or, more likely, that any flaws detected have been properly corrected before release.

  • 1
    Has anyone pointed out to your management that penetrate and patch is a waste of time and uneffective? ranum.com/security/computer_security/editorials/dumb Feb 8, 2011 at 14:45
  • @whatsisname: Penetrate and patch is a really bad way to build security, yes. However, a penetration test is a reasonable part of some user acceptance testing, and if it finds problems they should be addressed. It's no excuse for not starting with security as part of the requirements, design, and implementation. Feb 8, 2011 at 17:07

5 Answers 5


I guess it depends on how expensive (in cash, time and resources) is to conduct the test, how often it can be repeated, and how much in advance you need to negotiate your test dates. And also about how limited its validity is, i.e. if you do the simplest change in the code after passing the test, does it invalidate the whole test, forcing you to repeat it fully, or partly?

If it is expensive and hard to arrange, I would schedule it to the UAT phase and try to run as much similar testing as possible in-house before. Obviously, the easier and less expensive to do it, the earlier it is worth starting the tests (once I have a reasonably stable version including all critical features, that is).


The obvious simple answer would be: at the end. Doing it half way through just leaves room for flaws to creep in during the second half of the development, etc.

More effective would be doing it every so often. That way you'll be able to find and fix the flaws closer to when you actually made them, which is much easier than remembering what you did months ago. If you can put some of the basics of penetration testing into a unit test, then first up a continuous integration server which can alert you if you make any real stinkers (though can't really replace real testing).


When in the project life-cycle would you fit in external penetration/security testing of the software?

It happens at every step in the life cycle.

First, penetration testing is only one small piece of a more complex security puzzle.

Second, penetration testing includes a run-time configuration and proper administration as well as a software implementation issue. Penetration testing requires proper patching and configuration of OS, web server, database, firewalls and application software.

You do penetration testing of the architecture before you start doing detailed designs. Is the architecture itself secure?

You do penetration testing of the design. How secure is the design? What penetration opportunities will be present?

You do penetration testing as part of coding and unit testing. Does the code meet the design? Does it properly use architectural features to prevent security problems?

You can then subject the final software product to all kinds of security scrutiny including -- but not limited to -- penetration testing.

Security is pervasive. Or it has holes. It's not a "step" in the work plan. It's an underlying principle.


Obviously it is a risk to do at the end, fixing things might just cost to much time.

Obviously it is a risk to do it to early, added features might add security vulnerabilities.

My advice, do it as late as possible but reduce the risk.

  • Get insight form the testing company what they are going to test so you can design and build for that
  • Build the potential attack surface (Website/web service anything that allows interaction with the outside world) first and when is ready do the test.
  • during building use static analysis tools like fortify to test basic security
  • create your own security integration tests.
  • keep enough time after the test to fix and retest.

Hope this helps

  • I do not see the idea behind the your first point - why would you want to design and build to pass the testing company tests? This would give a false sense of security, as the attacker is just as likely go around the wall than through the wall. Perhaps I misunderstood your point?
    – Schedler
    Feb 8, 2011 at 11:43
  • 1
    Good question. That is not the point, the point is to get to a secure application and make sure you don't forget major parts of the security that will give you a lot of trouble if you find them when you are finished. It is just leveraging the knowledge of the external party as early in the project as you can. The false sense of security you describe will always be there, only when you know 100% sure that the third party has tested everything any hacker can think of it will be completely secure.
    – KeesDijk
    Feb 8, 2011 at 12:06

I would make it part of the system integration test. Unless your system is highly modular, it probably wouldn't be worthwhile to do it as part of your unit testing. I'd also schedule it early in the integration test phase, as security issues are notorious for causing significant system redesign.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.