1

This question already has an answer here:

I've been tasked with studying a Python code, which runs on an orchestrator, which schedule the launch of the code in correspondence of some events. The code imports some modules, and it's basically written as an extension of a method of a certain class, defined in the orchestrator module (is extension the correct term, or maybe inheritance? unfortunately I haven't studied OOP). I've been required to:

  • understand the inner workings of the code
  • understand which web services it uses, which databases it accesses, etc.
  • write unit tests for the code
  • profile the code

The code has no documentation, and the original developer has very little time to talk to me about it. Fortunately, the orchestrator developer can help me much more, so all is not lost. I know, from a very high level, what the code should do (get a huge object, containing time series data and lots of metadata, get some more data through REST API calls, make a few computations and return the updated huge object), but definitely I don't know the details (I don't know what the single lines of code do). I'm not a professional programmer. What approach should I follow? I was thinking of:

  1. using a tool to build the call tree of the code (which function calls which other function) and get an idea of the logic flow of the code.
  2. start writing unit codes for each function in the code (basically, I will search for def: and func() statements, and write one or more test for each function)
  3. use a profiler to see how much time is spent in different parts of the code

Does it make sense? Am I missing something huge?

The code is actually used as a service (meaning that some engineer runs the code and gets some data she/he will use to provide a service to an internal customer). The code will be migrated to a different platform (another orchestrator) and become a product (i.e., it will be directly used by the internal customer). My employer wants me to provide the outputs in the bullet points above, in order to document what the code is doing and how. This way the work of the developers involved in the migration is facilitated: if they're just give the code & its dependencies as it is now, without a line of documentation (there are not even comments!) and without any hint to the structure of the code, they probably won't be having a good time.

marked as duplicate by gnat, Bart van Ingen Schenau, Robert Harvey, Neil, 17 of 26 Aug 30 '17 at 13:27

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • @gnat thanks for the suggestion. I'll look at those questions (they have about 50 answer together, so it will take time). The main difference I could see among my question and the two you point to, is that my question is not a very generic one about huge code bases (I don't have to deal with a 200k code, luckily!). I added a few details about my code (scheduled on events by an orchestrator - extension of a method of another code - uses REST APIs - etc.). Not sure if this is enough to differentiate my question. If you don't think so, I can delete it, unless you have suggestions to improve it. – DeltaIV Aug 28 '17 at 13:32
  • 1
    You've been asked to do a few things, but to what end? Does the performance need improvement? Do you need to document what this is doing? Knowing what your employer expects you to get out of this can guide answers to be better for you. – Becuzz Aug 28 '17 at 13:54
  • @Becuzz good point. I will modify the question to explain what's the final goal of my task. – DeltaIV Aug 28 '17 at 13:55
  • It helps when the code/data flow is traceable: logging and timestamp in result; maybe even more evident marking text. – Joop Eggen Aug 28 '17 at 14:22
2

Inheriting legacy code is one of the most common things in the software industry. So common that, we can find several publications regarding this topic. Worth a mention Working effectively with legacy code - Michael C. Feathers for being one of the most renowned.

However, these publications are strongly focused on refactoring code. If I understood right, that's not your goal, your goal is to document.

With this goal in mind, here some tips that might help you.

1. Get a copy of the code

Get a copy of the code in production. Fork the code in the SCM or make a branch and check it out.

2. Contextualize the code

Decontextualized code doesn't say too much about its intentions. It tells how things are done, but it doesn't say too much about the why (requirements) and when (use cases).

Any piece of code serves a purpose. The sum of many pieces serves a higher purpose. And so on. The first task is to find out the highest purpose. The overall idea (what problem is solving). Then we have to contextualize it inside the system in order to answer the why and the when.

These answers are important for the future tests. Without them, we could end up testing irrelevant uses cases or misinterpret the results.

This irremediably lead us to ask people that have very little time to talk to us about it. But, the functional knowledge is as important (or more) as the technical, keep asking3.

3. Identifying levels of abstractions (responsibilities)

If we found out the global idea, we already identified the highest abstraction. As we delve into the code, we find more and lower levels of abstractions. And by abstractions, I mean responsibilities.1

The goal here is to identify the components involved in each responsibility and how they contribute to the global solution.

4. Document the overall picture

Provide future developers with a little introduction of the code.

Add a README-like file to the project and put in it the information gathered during the contextualization. Introduce the global picture, the most relevant abstractions and why are they relevant.

Hints for running the applications and reproduce the use cases are highly appreciated.

5. Document the code

Having identified the responsibilities of the components now is time to document them in the code. Take a look at the documentation conventions of the language (Python Docstrings). Then write the code documentation according to these conventions. Be clear and concise.

6. Document traps and misleading names

No need to say that most of us are terrible when it comes to naming things. Not everybody craft self-documented or self-descriptive code. So, don't you blindly trust everything you read. Get into the functions or classes. If the components names are not aligned with their respective functionalities, document them but don't change the actual names.

7. Document dependencies

Be back to the Readme file and introduce the dependencies (3rd party libraries). Where and when are they used, versions, etc. Add links (if possible) to the official pages of the libraries.

8. Document integrations

I use to introduce the integrations with diagrams. These are much more expressive than textual documents.

Additionally, for each integration could be useful to document request/response messages, ER data models, data sources, URLs, endpoints, protocols, etc. The more info you gather the better.

Especial attention to the error handling strategies. They provide valuable information about how the business reacts to errors.

9. Test the code

This's easier to say than done. Most of the legacy code we inherit was not written to be testable. Considering the future migration and the actual task (documenting), I'm unsure about the value provided by the unit tests.

For this particular case, integration and end-to-end are more valuable than unit tests because the actual components might not be reused during the migration (normally they aren't). The migration could completely change the implementation 2.

Ask your project's manager how much time you have for completing this task. Implementing tests for legacy code uses to take longer than usual. So, address the tests to provide valuable insights of the business rather than the implementations details.

In case of doubts about the unit tests, I suggest taking a look at the following questions

10. Document inconsistencies

During the test phase, we might find inconsistencies between the results we got and the results we expected. It would be dangerous to assume that these inconsistencies are bugs. We could have found bugs or missing features not documented anywhere. Or we could have misinterpreted some requirements. In any case, we should document these cases. Especially, how to reproduce them.

11. Commit the changes frequently.

As the documentation progress, save the changes. Do frequent commits.

Related questions


Note that I have not mentioned anything about executing and debugging the code. I think I could not say anything that hasn't already been said in:


1: Remember that any responsibility can be comprised in turn by one or more secondary responsibilities.

2: If the actual code is going to be reused (adapted) as it's, then unit tests are a must.

3: One strategy that works for me is inviting these persons to a coffee.

  • Excellent answer! Just a detail: since the code uses many REST API calls, and it's scheduled by an orchestrator according to specific events, would it make sense to make a sequence diagram as part of the documentation? I'm not too familiar with sequence diagrams (I'm neither a professional developer or a software engineer), so if you think they could help, could you also point me to tools for producing them, or references explaining how to make them manually? – DeltaIV Aug 30 '17 at 14:19
  • 1
    For diagrams and UML I use PlantUML. Regarding the sequence diagram, yes it could make sense if you don't end up with a single and bloated diagram. Consider splitting the different sequences into different diagrams for the sake of the readability. Take also a look at Activity diagrams for describing procedures. – Laiv Aug 30 '17 at 14:44
1

Hands on approach:

One part is to make results verifiable with respect to the processing > done.

Weird results can be analized by intensive logging (to database maybe) and giving some mark (timestamp, autoincrement key) with the results.

Similar reports, say Excel sheets or PDFs, should receive a form serial number.

Multiple conditions, possibly even nested, should be transformed in a more declarative way: a state consisting of some variables, yielding some specific handler. In that way the state is logged at one place, and the handler is immediately clear.

Structural:

Sonar-lint/FindBugs. Dependency analysis: no cycles, no pingpong (java 9 modules would be nice as refactoring target). For code duplications (similar code blocks) ensure you refactor all at the same time. Avoid many small code improvements (endless "refactoring").

Make things quantifiable: number of sources, lines; state of refactoring, mile stones. As refactoring proceeds keep statistics and a diagram. Refactoring might be underestimated, and rather nebulous.

  • 1
    Hi, what do you mean by "no ping-pong"? Also, what's a Java 9 module? The code is Python. Thanks for the suggestion about keeping things quantifiable! That's always something to keep in mind. Hopefully, I shouldn't do any refactoring: I've been asked to "understand" and document the code, but the actual migration will be performed by others. – DeltaIV Aug 28 '17 at 19:23
  • 1
    My past experiences got away with me. Ping-pong is my moniker for two classes doing many reciprokal calls. – Joop Eggen Aug 29 '17 at 6:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.