1

After some time in every project sooner or later you will need to find a feature in it a fix it or extend it so I was wondering if there is already a system or a convention for marking those features in code... what I mean is to somehow mark a place or places in code where a particular functionality is implemented or tested.

I gave it some thought and came up with the idea that I could crate a new comment token in visual studio like FX and use it to mark particular features/tests with a random three letter code:

class Person
{
    public string FirstName
    {
       set
       {
          // fx abc
          if (string.IsNullOrEmpty(value)) { throw new ArgumentNullException...}
       }
    }
}

Then I could track all the features in Excel:

  • abc - person first name must not be null or empty / does not allow empty/null value - hastest: yes

The same three letter code I would also use in a unit test to test this features:

(putting test naming conventions etc. aside)

[TestMethod]
[ExpectedException(typeof(ArgumentNullException)]
public void PersonFristNameTest()
{
   // fx abc
   new Person().FirstName = null;
}

It's obvious that we have a Person class and it has a FirstName property... but it's not so easy to keep track or document each and every requirement or condition inside every property or method especially if you need to find something later or tell if you already have it or not and where.

Have you seen something like this or do you use anything similar by yourself? I'm curious about your experience and suggestions.

I don't use this idea yet. I kind a invented it just yesterday after I was checking for the hundredth time if I already tested some ifs and where I did it and whether I already documented them.

  • 3
    are you aware of Requirement Traceability matrix? "Basically it covers all the relationships of your project where you can trace if there was change in requirement or after long time you came and work on project but you don't remember anything related to project." – gnat May 13 '16 at 7:46
  • @gnat no, I wasn't :) that's probably why I needed to invent something else. I'll give it a chance. – t3chb0t May 13 '16 at 7:50
  • 1
    What level of detail are your requirements? I've worked on some very safety critical projects where we were required to map requirements to the method level. However, even in the best of circumstances and the best developers who designed the system to encapsulate requirements into modules, this is a herculean effort. If you have a system and are trying to get to that level of detail "after the fact" then forget it. Go look up how to write a "Software Test Description" and write tests that prove your system meets the requirements by mapping requirements to your test procedure steps. – Dunk May 13 '16 at 18:18
  • 1
    "... mapping requirements to your test procedure steps." +1. This is an aspect of behavior driven development (BDD).. While BDD is a complete development framework (or mindset at least) - and there are frameworks - You can at least stylize tests to express requirements: What’s the difference between Unit Testing, TDD and BDD?. – radarbob May 26 '16 at 19:48
1

I'm not sure if you are asking for high-level features, let's say something you work on for a couple of days or low-level ones.

For the first one, if you have a bug/feature/task tracking system, e.g. JIRA, you could automatically match the code/code changes with the feature by including the ID of the task to the commit message e.g. [FOO-312] Add basic user data

However, I think that you suggest linking specific statements, their requirements and their tests; please correct me if I'm wrong.

E.g. if you had to check (a) name is not null, (b) age is greater than 18 and (c) country is in EU, under your suggested system you'd write something like:

// fx aaa
if isNull(name) { throw ...}
// fx aab
if age < 18 { throw ...}
// fx aac
if inEU(country) { throw ...}

// fx aaa
testRejectNullName ... 
// fx aab
testRejectChildern ...
// fx aac
testRejectNonEU ...

aaa: we need a name for the package
aab: customers have to be of age
aac: we only ship to EU

Now for this specific example I'd suggest to group everything under one validation method and keep track of that - in the end, unless we create abstractions, no matter how leaky, we'll end up with an intractable ocean of little features.

But let's assume that that's not possible and there are very exact specifications. It is true that having good names can go a long way to make it easier to find everything and by using comments you can explain why we have this restriction. Nevertheless, sometimes it's not an ideal solution especially if the person that's looking for that information is not technical, e.g. a product manager.

In that case, I'd suggest to use some specification framework, such as Cucumber (just providing it as an example, not even sure if you can use it in C#), to have a readable (even by non-technical people) configuration that can be matched to your actual code and unit tests. Of course, it involves more work so it's up to you to decide what level of detail you should cover with it.

Another thing to keep in mind is that matching tests with specific statements/lines of code sounds a bit dubious. If it's truly an independent condition, I'd suggest extracting it to a different method (and the name of that method could help with understanding the code) otherwise you have the risk of making the tests too specific and inflexible.

  • This is exactly how I was going to use it. However, where ever I can I already try to move the validation code into a more specialized validation class or method. Unfortunately this is not always easy like checking for a null in a loop or another simple condition that doesn't really qualify for refactoring but is actually a requirement. It should also be a way to find them more easily and connect them with the test. – t3chb0t May 13 '16 at 11:27
  • In a situation where several IF-s are involved you need to write at leas one test for each one. I cannot think of any way of connecting them so that you can easily find what you are really testing and whether you are testing it at all. Also very often additional processing takes places between each IF and it is not possible to create a general validation and group them. But I'm open minded and I'll give your idea a thought :] otherwise I wouldn't have asked for it :) – t3chb0t May 13 '16 at 11:38
6
// fx abc
  if (string.IsNullOrEmpty(value)) { throw new ArgumentNullException...}

Please do not do this. This is a technique that was invented in the previous millennium and is best left in that previous millennium. Follow this rigorously and the code quickly becomes unreadable. This defeats one of the primary purpose of code, which is to make it comprehensible to a human reader. (If this was not a primary purpose we would simply checkin the object code.)

Any modern revision management system will be able to tell one who to blame or praise for each and every line of code in the codebase. Subversion, git, Mercurial, bazaar, ClearCase, and many other revision management systems provide commands that do exactly this.

Any modern change management system will be able to trace change requests (a bug report is a change request) to interpretations of those change requests. Change requests to changes to the test base is a many-to-many relationship, as is the mapping from test cases to code.

This multi-dimensional, many-to-many relationship is not something that should be trusted to humans to maintain.While we humans are very good at some things, we are not so good at others. This is one of those things that fall in the category of "others." We humans are not good at that. Bad software has turned out the lights on tens of millions people, crashed rockets, and sent lethal doses of radiation into people.

Instead, trust this to modern 21st century tools that can do this automatically. The results are quite amazing in organizations that properly combines a modern revision management system, a modern change management system, and a modern requirements management system (if needed).

  • I'm afraid I meant something different. The short code shouldn't track the developer... as you say this is already done by all kinds of source code management systems. The idea was to make it a marker for a feature/requirement that in this way can be easily found especially if and where it is tested. I find it extremely difficult to track all conditions/requirements/ifs/switches and their exceptions, error codes etc and their tests. I admit it's not perfect because it's just a string but I don't see any other way to link an IF or an exception with a unit test and a description why it is there. – t3chb0t May 13 '16 at 10:29
  • 3
    Full disclosure: I am the author of one of someone's top ten list of software bugs, royal.pingdom.com/2009/03/19/… . The Nimbus-7 SBUV/TOMS experiment did not discover the ozone hole, and my code was at fault. Strictly speaking, what I wrote was not a software bug. It was instead a requirements bug. It ultimately was my fault because I was not an insistent, persistent, and curmudgeonly person back then. OTOH, those who are insistent, persistent, and curmudgeonly in their 20s are not going to get far in their life. – David Hammen May 13 '16 at 10:34
  • I particularly like the last heading in your article: more careful testing would save money (and lives). I thought with my idea which seems to be actually a light version of the requirement matrix it'd be a little bit easier to get there :) – t3chb0t May 13 '16 at 11:17
  • @t3chb0t - That's not my article. My code is one of the subjects of that article. – David Hammen May 13 '16 at 17:35
  • And no, modern testing would not have found that bug. If we tested code back then the way we do now, there would have been an explicit test that ensured unexpectedly low or high ozone values at low solar angles were zeroed out and marked as missing. – David Hammen May 13 '16 at 17:42
1

You were right, the three letter codes and excel tables weren't a good method for tracking requirements.

I have experimented with other systems and came up with this. I divide the tests into UsesCases and ErrorHandling. This allows me to better focus on what I am testing and I can avoid weird naming conventions so that each test has a pretty clear and meaningfull name. Here's an example:

Let's say I want to test this class:

class FileParser
{
    public static List<string> Parse(string fileName) { ... }
}

so I would create these tests:

namespace FileParserTests
{
    [TestClass]
    public class Parse_UseCases
    {
        [TestMethod]
        public void ParsePlainTextFile() { ... }

        [TestMethod]
        public void ParseXmlFile() { ... }
    }

    [TestClass]
    public class Parse_ErrorHandling
    {
        [TestMethod]
        public void NullOrEmptyFileName() { ... }

        [TestMethod]
        public void FileDoesNotExist() { ... }

        [TestMethod]
        public void UnsupportedFileType() { ... }
    }
}
  • 1
    Having "defect" tests is the best way to track these in code, as well as guarding against their re-emergence in later versions. I would even go so far as to commment the test method with why the bug fix exists, or add a Description attribute to the test method. Referencing work item or trouble ticket numbers is largely useless because those systems change, and therefore so do the ticket numbers. – Greg Burghardt May 26 '16 at 15:27
0

Not sure if this is exactly what you're getting at but in Visual Studio you can do this with //todo comments. I.e.

//todo test this function
private void test()
{
}

This'll make it appear in the 'Task List' panel (View -> Task List) under the 'Comments' dropdown. It only tracks comments starting with '//todo' which is pretty handy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.