I am required to analyze and make an architecture of an application. While analyzing the requirements I find, in my system user personally identifiable information (PII) confidentiality is a very sensitive quality requirement and it must be taken to the NFR (non functional requirement) section. NFR requires that all the responses should have measurable equation which is used by tester in verification phase.

I am new in this area and facing hurdle for preparing data security related measure and their allowance limit. I prepared the NFR table as below and I am afraid that might not be proper way because, in verification phase it will be tough for tester to verify this requirement.

So, the question is, how usually data confidentiality and security attribute measures and allowance limits are set by architects for such software verification phase by testers?

  • 1
    So is it OK if I do 1,000,000 failed attacks followed by 10 attacks which leak your entire database? May 17, 2019 at 17:06
  • 1
    @PhilipKendall: No system is totally secure. It's like trying to provide mathematical proof that your system works; there is no such thing. May 17, 2019 at 17:10
  • @PhilipKendall the term attack is a very complex stuff here due to nature of it and required environment. Linear test cases cannot make us assured that the system is secured even though all pass. That's why the question and it is really a tough analysis for me. May 17, 2019 at 17:13
  • @RobertHarvey That's the point I was trying to make. I'll be less subtle next time :-) May 17, 2019 at 17:20

2 Answers 2


As you've already surmised, a metric like "Attack Success Rate <= 0.001%" isn't a particularly useful test metric for an acceptance text.

Have a look at this page. It says:

Non-functional requirements


Provider systems SHALL resist unauthorised, accidental or unintended usage and provide access only to legitimate users. Please refer to the Security guidance page for technical details.

Now, of course, that's not a testable requirement; it is a wish. However, if you go to the Security guidance page that is linked there, you will find (among other things) this:

Secure connection negotiation

Provider systems:

  • SHALL only accept connections from the Spine Secure Proxy (SSP)

  • SHALL authenticate the SSP prior to responding to any requests using its client certificate

  • SHALL only permit approved supported ciphers to be utilised

  • SHALL only accept encrypted connections and drop connection attempts presented over insecure protocols

  • SHALL only accept requests for its allocated address space identifier (ASID), as specified by the Ssp-To header on its matching endpoint URL

  • SHALL check that the Ssp-InteractionID value is consistent with the endpoint being requested

  • SHALL check for the presence of all SSP headers

  • SHALL check that an authorization bearer token is present and correctly formed

  • MAY authorise access to API endpoints through examining acceptable values in the JSON Web Tokens (JWT) requested_scope claim

  • SHALL risk-manage the security of the endpoints of the Transport Layer Security (TLS) communications, so as to prevent inappropriate risks (for example, audit logging of the GET parameters into an unprotected audit log)

Ah, now we're getting somewhere. For the most part, these are testable requirements. Not only are they specific and testable, they also provide a detailed outline of your security strategy.

  • My architecture documents require a tabular representation of quality scenarios with allowance. You are essentially recommending splitting this security requirement into several checklists. Is that the standard process for evaluating security concerns? Thanks for your guidance. May 17, 2019 at 17:43
  • "Security" is inherently multi-faceted. That means that requirements are multi-faceted; hence, multiple checklists with multiple dimensions. May 17, 2019 at 18:23
  • 1
    @SazzadHissainKhan: If there are "standards" involved here, they are the ones embodied in the specific requirements, such as "SHALL only accept connections from the Spine Secure Proxy (SSP)". The point here is that the specific requirements are testable. If your suits need a metric, then assign points to each specific requirement, add them all up and create a score. May 17, 2019 at 22:53
  • @RobertHarvey How is my question an "exact duplicate" of the previous question? In your comment to me, you said that the question I posed in my comment was different from the one I originally asked, so I posted a new one in a new thread. This is very inconsistent to me.
    – Sid
    May 20, 2019 at 14:39
  • @RobertHarvey I took pains to make sure in the new question that I wanted to know how to add "some" type annotations. My question was definitely not an "exact duplicate."
    – Sid
    May 20, 2019 at 14:40

NFR requires that all the responses should have measurable equation which is used by tester in verification phase.

I thinking this is probably the root cause of your issue. Some problems are just hard, and can't easily be reduced to a simple metric.

If your business truly cares about protecting its users' PII, the best thing to do is almost certainly to get it tested by a professional penetration testing service, and then set the metric to something like "all security vulnerabilities above trivial threat level have been resolved".

  • Who defines the term “trivial threat level”? Whats the measurable scale? May 8, 2020 at 13:04
  • Stop trying to reduce everything to a quantifiable number. The world isn't like that, sometimes you just have to use the best judgement of your trusted domain experts. May 8, 2020 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.