8

Forward

I've read a lot of things before asking this question, including many relevant questions right here on SE:

However, I can't help but feel that the itch hasn't been scratched yet after reading for help.


TL;DR

How do I write unit tests for legacy code that I can't run, simulate, read about, or easily understand? What regression tests are useful to a component that presumably works as intended?


The Whole Picture

I'm a returning summer intern again as I'm transitioning into grad school. My tasking involves these requirements:

  1. For a particular product, evaluate whether our software team can upgrade their IDE and JUnit version without losing compatibility with their existing projects.
  2. Develop unit tests for some component in the existing Java code (it's largely not Java). We want to convince the software team that unit testing and TDD are invaluable tools that they should be using. (There's currently 0% code coverage.)
  3. Somehow, end the days of cowboy coding for a critical system.

After obtaining a copy of the source code, I tried to build and run it, so that I might understand what this product does and how it works. I couldn't. I asked my supervisors how I do, and I was issued a new standalone machine capable of building it, including the build scripts that actually do. That didn't work either because as they should've expected, their production code only runs on the embedded system it's designed for. However, they have a simulator for this purpose, so they obtained the simulator and put it on this machine for me. The simulator didn't work either. Instead, I finally received a printout of a GUI for a particular screen. They also don't have code comments anywhere within the 700,000+ Java LOC, making it even harder to grasp. Furthermore, there were issues evaluating whether or not their projects were compatible with newer IDEs. Particularly, their code didn't load properly into the very IDE version they use.

My inventory is looking like this:

  • NetBeans 8, 9, 10, 11
  • JUnit 4, 5
  • Their source code for a particular product (includes 700,000+ Java LOC)
  • Virtually no code comments (occasionally a signature)
  • No existing tests
  • A physical photo of a GUI window
  • A software design document (109 p.) that doesn't discuss the component in the picture

I at least have enough to theoretically write tests that can execute. So, I tried a basic unit test on this said component. However, I couldn't initialize the objects that it had as dependencies, which included models, managers, and DB connections. I don't have much JUnit experience beyond basic unit testing, so follow me to the next section.


What I've Learned From My Reading

  1. Mocking: If I write a unit test, it likely needs to have mock variables for production dependencies that I can't easily initialize in setUp.
  2. Everyone here liberally suggests the book "Working Effectively with Legacy Code" by Michael Feathers.
  3. Regression tests are probably a good place to start. I don't think I have enough weaponry to attempt integration testing, and regression tests would provide more instant gratification to our software team. However, I don't have access to their known bugs; but, I could possibly ask.

And now an attempt to articulate the uncertainty I still have as a question. Essentially, I don't understand the how part of writing these tests. Assuming I don't receive any further guidance from my supervisors (likely), it's in my ballpark to not only learn what this component does but to decide what tests are actually useful as regression tests.

As professionals who've worked with projects like this longer than I have, can you offer any guidance on how to write unit tests in this kind of situation?

  • 12
    How do I write unit tests for legacy code that I can't build, run, simulate, read about, or otherwise understand? -- You can't. You have to at least know what the expected output is for a given input. – Robert Harvey Jun 19 at 15:35
  • 1
    Have you read Michael Feathers' book yet? – Robert Harvey Jun 19 at 15:38
  • @RobertHarvey It’s slight hyperbole. Obviously I can take the time to read code to understand it, but that’s typically a last resort for something so big. And no, I haven’t yet, since I just found the book this morning. – Joshua Detwiler Jun 19 at 15:59
  • If you have legacy code that you can't even build, are you sure that you even have the right version of the code? You need to resolve this first before worrying about automated tests -- is the code that you're trying to build even the same as whatever your users are running? The first step in all of this if you did have a running and working system would be to try to understand it from the user's point of view; it can help to read user documentation and maybe look at user data if possible to try to figure out what it does – Ben Cottrell Jun 19 at 16:01
  • 2
    If sensor data is crucial to execution, mocking the sensors (or whatever the external dependencies are) seems like a good place to start. – JimmyJames Jun 19 at 17:35
10

To a first approximation, the stakeholders of a test suite are the code developers/maintainers. You're going to need some of their time. Insist on this.

Ask them about problems they're facing.
Ask them about bugs they've fixed recently (in the past couple of years, assuming this is a long slow project).

I get the impression that you're not expecting them to be amiable to your work; maybe you're right. But I don't think you can build anything useful without their help.

They're going to hold your hand through writing the first test. Maybe you'll be dragging them, but you'll be holding hands either way.

Once you have one test, hopefully it'll be clearer how to write the second. If you've managed to build any kind of rapport, it will certainly be clearer.

  • Thanks for the answer! Could you also take a look at the second half of my question too? My first half was a little duplicating of the questions I linked, and I was mainly interested in the other part of my question because of that. – Joshua Detwiler Jun 19 at 19:04
  • I don't know how helpful I can be. Are you having trouble understanding how the test framework works? Are you wondering what to test? For that, regression tests are a good choice: What test would have prevented this bug we just fixed from getting pushed, if we'd been so forsightful. – ShapeOfMatter Jun 19 at 19:10
  • It was more "What regression tests are useful to a component that presumably works as intended?" Like, if I'm to create tests that are useful, what's useful being totally blind about what works, what doesn't, and how it even works at all? I exist in a vacuum apart from the software team and the people who gave me this task exist in a vacuum apart from both of us. – Joshua Detwiler Jun 19 at 19:22
  • I'm still sure that you need to insist on getting in the same room as the the people writing the software, ideally for a couple of days, and probably more than once. I can relate to imposed absurd working situations, but at some point one either needs to risk one's job or accept that you're just a bench-warmer, I guess – ShapeOfMatter Jun 20 at 0:54
  • Regarding Regression Testing: It's not conceptually different from any other testing: What would I want the program to do (or not do) before I claimed to a coworker that it was working? If there's records of new features or recent bugs, those are good places to start. Or just look through the documentation you have and pick something that looks testable. Java has typed functions, which is good. A clear type signature can tell you a lot about what to expect from a function (and can be thought of as a kind of unit test in itself). Checking NULL/empty-string/max-int behavior may also be good. – ShapeOfMatter Jun 20 at 0:59
3

I'm going to assume at some point you can get the code to at least compile. If you can't even get that far you are on a fool's errand.


Lacking proper requirements, specifications, or screenshots is not a blocker for writing tests. As long as you can read source code, you can write tests.

If you are given permission to refactor the code base to isolate things like database connections behind their own interface, it becomes possible to write some black box unit tests — basically write tests to throw some input at a method and assert its behavior or output. Get tests covering each line of code in one method and then have one of the senior members of the team review your tests.

If you are not given permission to refactor the code base in order to write unit tests then full integration tests or UI automation tests are your only option. Even then, black box testing is your best strategy — throw some input at the UI and see how it reacts. Make your asserts. Have a senior member of the team review your tests.

At some point you will have enough automated tests that you can begin refactoring the code base with confidence to introduce unit tests. The UI tests ensure the main business use cases work, and then you can retrofit an architecture conducive to unit or component level testing.

Another benefit of UI tests is that you can build a reputation with your team that you understand the code base, which in turn makes them more open to you introducing change, because the proof is in the pudding. And you will have made pudding by way of writing passing tests.

In short:

  • Write Black Box tests as unit or automated UI tests
  • Have senior members review your tests

You would be surprised how quickly you can learn the big picture view of a 700,000 line application

1

Based on the description of the issue and your comments, I think the best you can do is to start with the Java API and try to build a single unit test around an isolated method.

Without access to the code, I can only give you limited guidance but I would look for something that in it that a) has no dependencies b) makes no state changes. For example. let's say there's a method that takes a value and checks that it falls in a given range. If you can't find something with no dependencies, try something that retrieves a value from a dependency and attempt to mock it out.

Once you find something small like this, you can start building tests. If the method tests of a positive value, then pass it a negative and make sure it catches it etc. The problem here is that you may not know for sure what the correct behavior is. You may need to ask for help or dig through documentation for that.

You are unlikely to get very far with this. The reality is that writing code so that it can be unit-tested is an art to itself. Code that was not written with this in mind can be difficult-to-impossible to unit test.

Another option which may be more easily implemented is binary compatibility testing. That is, you capture the inputs and outputs of a system and then to test, you feed those same inputs and compare the outputs. This will not tell you whether the code is right or wrong to start with but it can help detect regression errors where things changed unintentionally when making some other modification. You will need to be able to run the entire application in order to make that happen, however.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.