5

Im working on the implementation of an IEC 62304 compliant development process in our company and I have a question considering SOUP (Software of unknown Provenance).

Our goal is the develop a web application in the future.

For this application we will use different open-source libraries on the backend as well as on the frontend.

For the most of these libraries our SOUP documentation procedure is known.

  1. Identify the library, version number, licence etc.
  2. List the requirements we have for the library.
  3. Test these requirements in form of Unit/Integration tests.

For these libraries it is also possible to separate their code from our code with clear interfaces and document this separation.

Now on to the problem. What should we do with Frameworks like Spring/JEE on the backend or Angular/React on the frontend. The code of these libraries isn't separable from our code. We even use these frameworks to structure our code. We can generate requirements to these frameworks, but I’m not sure how to test that these requirements are fulfilled. The two options im considering are:

  1. There are several Integration and System tests which test that the overall software system, which is build using the frameworks, runs as it should. Are these tests enough even if they are not explicitly testing the framework functionality?
  2. Write lots and lots of framework tests which test if dependencies are injected or that Rest endpoints are available or that html template injection works and so on.

Personally I tend to option 1. What I don’t like about option 2 is that unit testing a framework is a completely redundant effort. These frameworks already have a huge amount of tests by themselves which already cover all of the functionality they have. Also, In some cases, I need to use parts of the framework itself to test the framework. (eg. Spring MockMVC to test Controllers).

So my questions are:

  1. Are Software Frameworks (Spring/JEE/Angular/React) considered SOUP?
  2. If yes, how to document and test these Frameworks to become IEC 62304 compliant?
Improve this question
2
  • Thank you. If "YES" is your perfectly viable answer, do you have further input o question number 2, How to document frameworks as SOUP?
    – TardigradeX
    May 14, 2018 at 15:53
  • Sorry, gotta delete my comment which was a ridiculous pun. Frameworks => bad code => soup as in "tag soup".
    – user44761
    May 14, 2018 at 18:30

1 Answer 1

Reset to default
5

Keep in mind that I'm not familiar explicitly with IEC 62304 (it's for medical device software and I've never worked with medical devices), I am familiar with similar standards, such as ISO 9001 and various aerospace standards that serve a similar purpose as IEC 62304.

Are Software Frameworks (Spring/JEE/Angular/React) considered SOUP?

Unless the software has been appropriately certified, any software is considered SOUP. I doubt that the development processes used to create these frameworks has been certified to be compliant with IEC 62304. The required processes would add significant overhead to open-source projects that are primarily maintained by organizations outside of a regulated environment.

If yes, how to document and test these Frameworks to become IEC 62304 compliant?

Standards such as IEC 62304 are about reducing risk. So, the question becomes how you can sufficiently reduce the risk of using third-party software so that it is acceptable to stakeholders.

This depends greatly on the type of product that you are making, it's safety classification, and how the components that use these frameworks interact with the product.

There is an advantage of open source projects - you can analyze them. You can read the source code (and any kind of tests). You can write additional tests. You can run a static code analysis tool against them. If it needs to be compiled, you can compile it yourself with a known toolchain and be certain that the executable is safe. Depending on the project, you may also have full access to commit history, defect tracking, and get insight into the responsiveness of the development community into bugs and CVEs. All of these can help paint a picture of the level of risk associated with using a given product.

If you can perform a risk assessment of the framework and then build your product to IEC 62304 standards, then maybe you'll be OK. However, you'll need to be convincing of auditors and stakeholders. If you are building a new product, you should engage your stakeholders early to communicate what your development plans around reducing risk of the SOUP is and get buy-in.

Improve this answer
3
  • Thank you for your insights. I do also think we have to evaluate the Frameworks in our risk assesement. You mentioned to run static code analysis, compile and test the frameworks. This would generate an enourmous overhead for us. Do you think it is enough to document several available statistics(code coverage, maintanance, defect tracking methods used) and a small analysis for a given Framework in our Architecture Document, which then can be referenced in the risk assessment?
    – TardigradeX
    May 14, 2018 at 15:44
  • 1
    @TardigradeX It depends on what your stakeholders are expecting. I'm not familiar with the projects you mention, but obtaining a release from their version control system, running static analysis on it, and compiling in your own environment shouldn't be too burdensome. However, maybe documenting the information and performing an analysis would be sufficient. Like I said - it's about balancing risk. Are you OK with not running and reviewing static analysis on the tools? Are you OK with not knowing if the build artifacts match the source code exactly? Are your stakeholders OK with this?
    – Thomas Owens ♦
    May 14, 2018 at 16:36
  • Thank you very much. I will discuss the issue further with our PM, QM and incorporate your input into the discussion.
    – TardigradeX
    May 15, 2018 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.