7

Does anyone know how to visually diagram the ways in which the flow of control in code passes between code produced by different groups and how that affects the amount of code that needs to be carefully written/reviewed/tested for system properties to hold?


What I am trying to help people visualize are arguments of the form:

  1. For property P to hold, nd developers have to write application code, Ca, without certain kinds of errors, and nm maintainers have to make sure that the code continues to not have these kinds of errors over the project lifetime.
  2. We could reduce the error rate by educating nd developers and nm maintainers.
  3. For us to be confident that the property holds, ns specialists still need to test or check |Ca| lines of code and continue to test/check the changes by nm maintainers.
  4. Alternatively, we could be confident that P holds if all code paths that could violate P went through tool code, Ct, written by our specialists.
  5. In our case,
    • test suites alone cannot give confidence that P holds
    • nd ≫ ns
    • nm > ns
    • |Ca| ≫ |Ct|
    so writing and maintaining Ct is economical, frees up our developers to worry about other things, and reduces the ongoing education commitment by our specialists.

or those conditions do not hold, so focusing on education and testing is preferable.


Example 1

As a concrete example, suppose we want to ensure that our web-service only produces valid JSON output.

Our web-service provides several query and mutation operators that can be composed in interesting ways.

We could try to educate everyone who maintains those operations about the JSON syntax, the importance of conformance, and libraries available so that when they write to an output buffer, every possible sequence of appends results in syntactically valid JSON.

Alternatively, we don't expose an output stream handle to application code, and instead expose a JSON sink so that every code path that writes a response is channeled through a JSON sink that is written and maintained by a specialist who knows JSON syntax and can use well-written libraries to produce only valid output.

Example 2

We need to make sure that a service that receives a URL from an untrusted source and tries to fetch its content does not end up revealing sensitive files from the file-system, like file:///etc/passwd.

If there is a single standard way that any developer familiar with the application language's libraries would use to fetch URLs, which has file-system access turned off by default, then simply educating developers about the standard mechanism, and testing that file probing fails for some inputs, will probably be sufficient.

  • Technical arguments won't help your situation. Been there, done that. If they could follow technical arguments they wouldn't have hired a bunch of asshats. They will dump a couple million into a big pile of WTF and either cancel the project entirely, push it to a buggy conclusion, or start over with a new contractor. Your best move is to look for a position on a project where someone has a clue. – kevin cline Jun 12 '14 at 22:07
  • @kevincline, It sucks that you've had that experience. What I'm trying to do with visualization is present the intuition behind the argument before presenting the technical details to increase the proportion that will follow the argument. – Mike Samuel Jun 13 '14 at 2:41
  • More clarifications about the difference between test suite and tool code would be helpful. I guess tool code refers to reusable component, but in some companies tool code refers to validation tools such as Valgrind. The skills it take to write these are (IMHO): validation tools (greater than) reusable components (similar to) full coverage suite (greater than) basic test suite. – rwong Jun 13 '14 at 4:40
  • @MikeSamuel: has someone asked you to do this? Or are you trying to convince someone that the product will be buggy if they don't make all those developers and maintainers follow some practice that you will define? – kevin cline Jun 13 '14 at 7:16
  • @rwong, Yeah. I would include among "tools", programs that do static or dynamic analysis. For my purposes, "tool" is what we're considering building or not building. – Mike Samuel Jun 13 '14 at 13:08
1

There may be better ways to do this, but my first take would be to use some sequence diagrams and put boundaries (like alt or opt) around calls into different code sources. Most UML tools I have used will allow you make arbitrary boundaries like alt, opt, loop. So you can have "oss" "lib1" "lib2" etc

Once you have the call stack segregated into different buckets, you can then annotate the sections with code metrics like number of lines of code, or cyclomatic complexity in ppt so you can show whoever needs showing where the code allocation is.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.