How do I write unit tests for legacy code (that I don't understand)?

Question

Forward

I've read a lot of things before asking this question, including many relevant questions right here on SE:

• (Software Engineering SE) Writing tests for code whose purpose I don't understand
• (Software Engineering SE) Unit testing newbie team needs to unit test
• (Software Engineering SE) Best practices for retrofitting legacy code with automated tests
• (Software Engineering SE) How to unit test large legacy systems?
• (Blog post) How to mock up your Unit Test environment

However, I can't help but feel that the itch hasn't been scratched yet after reading for help.

---

TL;DR

How do I write unit tests for legacy code that I can't run, simulate, read about, or easily understand? What regression tests are useful to a component that presumably works as intended?

---

The Whole Picture

I'm a returning summer intern again as I'm transitioning into grad school. My tasking involves these requirements:

1. For a particular product, evaluate whether our software team can upgrade their IDE and JUnit version without losing compatibility with their existing projects.
2. Develop unit tests for some component in the existing Java code (it's largely not Java). We want to convince the software team that unit testing and TDD are invaluable tools that they should be using. (There's currently 0% code coverage.)
3. Somehow, end the days of cowboy coding for a critical system.

After obtaining a copy of the source code, I tried to build and run it, so that I might understand what this product does and how it works. I couldn't. I asked my supervisors how I do, and I was issued a new standalone machine capable of building it, including the build scripts that actually do. That didn't work either because as they should've expected, their production code only runs on the embedded system it's designed for. However, they have a simulator for this purpose, so they obtained the simulator and put it on this machine for me. The simulator didn't work either. Instead, I finally received a printout of a GUI for a particular screen. They also don't have code comments anywhere within the 700,000+ Java LOC, making it even harder to grasp. Furthermore, there were issues evaluating whether or not their projects were compatible with newer IDEs. Particularly, their code didn't load properly into the very IDE version they use.

My inventory is looking like this:

• NetBeans 8, 9, 10, 11
• JUnit 4, 5
• Their source code for a particular product (includes 700,000+ Java LOC)
• Virtually no code comments (occasionally a signature)
• No existing tests
• A physical photo of a GUI window
• A software design document (109 p.) that doesn't discuss the component in the picture

I at least have enough to theoretically write tests that can execute. So, I tried a basic unit test on this said component. However, I couldn't initialize the objects that it had as dependencies, which included models, managers, and DB connections. I don't have much JUnit experience beyond basic unit testing, so follow me to the next section.

---

What I've Learned From My Reading

1. Mocking: If I write a unit test, it likely needs to have mock variables for production dependencies that I can't easily initialize in setUp.
2. Everyone here liberally suggests the book "Working Effectively with Legacy Code" by Michael Feathers.
3. Regression tests are probably a good place to start. I don't think I have enough weaponry to attempt integration testing, and regression tests would provide more instant gratification to our software team. However, I don't have access to their known bugs; but, I could possibly ask.

---

And now an attempt to articulate the uncertainty I still have as a question. Essentially, I don't understand the how part of writing these tests. Assuming I don't receive any further guidance from my supervisors (likely), it's in my ballpark to not only learn what this component does but to decide what tests are actually useful as regression tests.

As professionals who've worked with projects like this longer than I have, can you offer any guidance on how to write unit tests in this kind of situation?

Answer 1

To a first approximation, the stakeholders of a test suite are the code developers/maintainers. You're going to need some of their time. Insist on this.

Ask them about problems they're facing.
Ask them about bugs they've fixed recently (in the past couple of years, assuming this is a long slow project).

I get the impression that you're not expecting them to be amiable to your work; maybe you're right. But I don't think you can build anything useful without their help.

They're going to hold your hand through writing the first test. Maybe you'll be dragging them, but you'll be holding hands either way.

Once you have one test, hopefully it'll be clearer how to write the second. If you've managed to build any kind of rapport, it will certainly be clearer.

Answer 2

I'm going to assume at some point you can get the code to at least compile. If you can't even get that far you are on a fool's errand.

---

Lacking proper requirements, specifications, or screenshots is not a blocker for writing tests. As long as you can read source code, you can write tests.

If you are given permission to refactor the code base to isolate things like database connections behind their own interface, it becomes possible to write some black box unit tests — basically write tests to throw some input at a method and assert its behavior or output. Get tests covering each line of code in one method and then have one of the senior members of the team review your tests.

If you are not given permission to refactor the code base in order to write unit tests then full integration tests or UI automation tests are your only option. Even then, black box testing is your best strategy — throw some input at the UI and see how it reacts. Make your asserts. Have a senior member of the team review your tests.

At some point you will have enough automated tests that you can begin refactoring the code base with confidence to introduce unit tests. The UI tests ensure the main business use cases work, and then you can retrofit an architecture conducive to unit or component level testing.

Another benefit of UI tests is that you can build a reputation with your team that you understand the code base, which in turn makes them more open to you introducing change, because the proof is in the pudding. And you will have made pudding by way of writing passing tests.

In short:

• Write Black Box tests as unit or automated UI tests
• Have senior members review your tests

You would be surprised how quickly you can learn the big picture view of a 700,000 line application

Answer 3

Based on the description of the issue and your comments, I think the best you can do is to start with the Java API and try to build a single unit test around an isolated method.

Without access to the code, I can only give you limited guidance but I would look for something that in it that a) has no dependencies b) makes no state changes. For example. let's say there's a method that takes a value and checks that it falls in a given range. If you can't find something with no dependencies, try something that retrieves a value from a dependency and attempt to mock it out.

Once you find something small like this, you can start building tests. If the method tests of a positive value, then pass it a negative and make sure it catches it etc. The problem here is that you may not know for sure what the correct behavior is. You may need to ask for help or dig through documentation for that.

You are unlikely to get very far with this. The reality is that writing code so that it can be unit-tested is an art to itself. Code that was not written with this in mind can be difficult-to-impossible to unit test.

Another option which may be more easily implemented is binary compatibility testing. That is, you capture the inputs and outputs of a system and then to test, you feed those same inputs and compare the outputs. This will not tell you whether the code is right or wrong to start with but it can help detect regression errors where things changed unintentionally when making some other modification. You will need to be able to run the entire application in order to make that happen, however.