28

I developed a big(ger) project, which is in use already and grows, gets altered, fixed, etc. every week.

Until now I am the only developer. Since the team has to grow, also we will be more developers on that project.

The code is mostly self-speaking or commented on the darker places ...

But I guess an experienced developer will find its path.

To explain the projects tech in a few points:

  • the Frontend are two SPAs (React) giving functionality to customers and administrators
  • the backend is a Node.JS API (NestJS) with the main purpose of being a flexible GraphQL and REST endpoint(s) and being capable of doing heavy computations, aggregation, organization and exports of data
  • a postgres database
  • additional (micro-)services doing cleanup, optimization, transfer, etc.
  • a CI/CD pipeline that let's us comfortably deploy changes to either a staging or a production/live server

My question is about documenting every flow (or the more complex ones), every Entity / Table, every use case, so that the user (customer) and developers know, what the whole system can do / has to do. Just to find and see first the big picture and secondly the internal details like rules, etc. without digging through the whole codebase (which btw is not an option for the customer).

The project has grown over years, so have I and so have the duties of that software. I still think the project is SOLID enough to be maintained for a longer time.

To the question.

As there is 0 documentation (except from the code), would it be a good Idea to start transfering parts or (in the best case) the whole project into

  • use case diagrams
  • a big class diagram (for the Entity Class should be OK, I think. To transfer even all controllers, services and helpers will be not too usefull, I guess)
  • activity diagrams for the more complex processes (or also even for the whole system?) Does a Double-Opt-In Process have to be drawn as a diagram?

I am asking for experiences of people working in larger projects. Is it good practice in big enterprise projects to do this? Or is it just a tool to "get into a project"? I am afraid that the work of achieving this will grow drastically while being in the middle of the process. I don't want to start and realize after weeks of work, that there will be no end and even no gain in having a "partly" with diagrams documented project.

I also would love to "feel more freedom in my mind". Until this point I feel like I have the big challenge of never forgetting anything about this detail just to be more on point if I dive back into the projects after weeks. I would like to put that knowledge (for me and others) on a sheet of paper or some .zargo files ...

Improve this question
6
  • 19
    Diagramming every single thing after-the-fact is not necessarily helpful, and a lot of those diagrams will likely go out of date faster than you can keep them updated. It's a different thing if UML is a part of your design thinking and idea communication utilized from the very start by the whole team. UML stands for Unified Modeling Language - that is, you use it to help you figure out what to build, to help communicate and discuss ideas, not just to document preexisting code. So if you want to have diagrams, focus on small number of things that are key for your readers to understand.
    – Filip Milovanović
    Oct 10, 2022 at 14:54
  • 10
    I've professionally supported well over 100, 2 tiered software products—with database backends. I've never used UML or data dictionaries once. However, I've had a couple customers use data dictionaries and a few contracts require them. If I ever needed a UML diagram, I would use an automated tool to generate them.
    – Nathan Goings
    Oct 11, 2022 at 3:29
  • 3
    Trying to describe a whole system with diagrams can be like assuming you can read, understand and enjoy a book by looking at the cover. Diagrams are good at summarizing information. The key is, what information you put on them. So you can have all the UMLs you want as soon as they focus on what really matters. The big picture I would dare to say. Detailed documentation is useless or becomes useless the day after the kick-off to never catch the actual state of the project again. It's too expensive and worthless.
    – Laiv
    Oct 11, 2022 at 14:01
  • Two comments. 1. You say '0' (zero) documentation. Really? So you have no notes, no diagrams, no memory aids or summaries whatsoever? The code might be self-speaking but reading every line to gain an understanding of what it does is ridiculously inefficient, even for the person who originally wrote it. 2. No use cases? The most important knowledge for a new contributor on a project is how the system solves the business problem it was designed for. If there are no use cases, then the only measure of correctness is what the code does. So you have a bigger problem.. or rather the new people do
    – Colm
    Oct 12, 2022 at 16:50
  • 3
    A good hand-made sketch that gets to the point is better than most UML diagrams, which tend to be awful. Just saying. Concentrate on getting the point across, not the form.
    – Polygnome
    Oct 12, 2022 at 21:40

6 Answers 6

Reset to default
52

Tldr

Documentation should not be seen as something that you simply start doing. It carries a colossal business cost to maintain. It will require additional staff and skills in its own right.

Long read

Beware, if you have written zero documentation so far, and you are in a workplace where zero documentation has ever been written (let alone successfully read), then there is every chance that the documentation you do write will be of a very poor standard and completely ineffective for its purpose.

Technical writing is a skill that most do not possess. Educators and academics - experts on documenting and reproducing knowledge of complicated areas of study - are experienced professionals in their own right.

Once written material exceeds the volume of a long slideshow, or a short article or memorandum, real authorship is required.

There is also very often a risk that the system you want to document has already gone beyond what can be coherently explained to others.

Sometimes, the conceptual weight of a system that has withstood alterations in design, far exceeds the volume of code, and it would be easier to simplify the code and purge historical remnants, than to fully explain that history and all the transitional logic.

One person, working without ever having to explain themselves or reproduce their own knowledge, and supervising their creation constantly for years on end, whose every detail they have devised, can build an edifice of far greater subtlety and complexity than can ever be properly handed over to another person.

The edifice will simply not contain the modularity, the regularity, and the refactorings, that would have been necessary in a team-built effort of similar functionality and complexity.

It will also probably take certain aspects to the extremes of the strengths and weaknesses of your own mind and the finest details of your own changing understandings and obsessions over the years, without ever having had to concede in the slightest to the different profile of someone else's mind coming to the problems afresh.

You might also get a shock about the sheer amount of overhead introduced by team-working. A team of three - if they are of your same calibre - will likely be no more productive in the long-term than you are alone, except that they can keep going once you are gone.

In other words, team-working means productivity will sustain better over multiple generations of staff, but three such replaceable cogs will likely deliver less over a single generation, than one irreplaceable kingpin will deliver in a single tenure.

If they are of lower calibre than you are, then productivity could be worse, because you will be dragged into tasks (like documentation and management) on which you have no experience, and away from the tasks on which you are already proven strongest, whilst they will never grow to your potential on the tasks you are trying to delegate. Also realise, useful documentation will take as much time and effort as writing code.

I would carefully consider whether your business is in fact prepared to pay for the cost of sustainability. Most aren't able or willing to afford it. Bespoke software generally becomes moribund with the departure of sole creators, and then the business either invests in a massive modernisation project, are absorbed by a competitor, buy something "off the shelf" that is widely considered substandard (until made bespoke again), or collapse to a more simple mode of operation that requires less software.

There is always another alternative for most businesses, and that is to avoid much documentation, but find natural modularity in the collective work to allow different aspects to be delegated to staff who cover different areas, or to hire a well-motivated deputy for the principal who will learn like an apprentice over years of working together on the same tasks. These approaches can however fail if there are no natural modules, or if the difficulty of the edifice exceeds the deputy's capability and motivation.

But documentation should not be regarded as the easy option. It's the most effective and flexible option when done well, but it's also the most expensive to do, and the most likely to be useless when done cheap.

Improve this answer
5
  • Thank you @Steve for your thoughts. I understand it very well. As this project is not the only one, my goal is still there in trying to document as well as possible. Even if I don't see a gain in speed with the first 1 or 2 devs joining the team. Though I want to gain in quality in code, teamwork an documentation. It is a good oportunity to train those, also. So my question is still open in the sense of how far I should go with documenting via UML. If I would get a perfectly documented codebase with UML, I would still read through the whole code in order to get a good overview.
    – Edgar Alloro
    Oct 10, 2022 at 12:45
  • 9
    @EdgarAlloro, my main suggestion, then, is be careful that your documentation primarily covers things that the code doesn't say. If you find yourself looking at the code for inspiration of what to document, remember, the code is a listing of things you have already documented. Think high-level maps (especially integration and interactions of disparate systems), relationships that are contrary to the code structure, or business analysis that explains why the code was written, or what the users are doing outside the computer system when they trigger a particular process.
    – Steve
    Oct 10, 2022 at 14:27
  • that sounds more like my current state of mind could be adjusted. The presence of the working code perhaps is more valuable as a detailed documentation for anyone who has to work on it. The code already feels more like reading "inside" the bible or another big book. It is just not possible to read the book in one turn ... Thank you again for all the points from your answer. I will rethink more on that project.
    – Edgar Alloro
    Oct 10, 2022 at 14:45
  • I've had mixed success on projects where we had to have our documentation reviewed by a professional technical writer. On one, the technical writer changed every occurrence of "extravehicular activity" (EVA, i.e., an astronaut spacewalk) to "vehicle with extra activities" (VEA) -- and then the parent organization published that embarrassing document without our consent. On another, the technical writer caught several items that he thought might be mistakes, and several were (e.g., sentences that weren't sentences). On that project, the technical experts had a chance to override.
    – David Hammen
    Oct 13, 2022 at 11:10
  • @DavidHammen, indeed, the problem is that producing documentation is liable to all the same vices as producing code. It sounds more like your "technical writer" was brought in more like a script doctor, to polish what you'd already written, rather than being someone on your team who participates in design and understands the topic as well as you do, and therefore relieves you of the need to write.
    – Steve
    Oct 13, 2022 at 12:22
19

Reframing the question

Indeed, managing documentation is costly. But the bigger the project, the more people involved and the longer the system's lifetime. If you count the time that it takes for every newcomer to grasp the big picture, and aggregate that count over the lifetime of the system, it may well appear that the documentation is much cheaper than no documentation.

Moreover, there are life-critical sectors where no documentation is not an option, and even big spenders with huge projects that mandate use of UML. So it must be possible.

Reframing the question to your context: How to maximize the benefit of the documentation while minimizing its cost?

UML is not about big unmaintainable diagrams

A popular belief is that UML requires a big class diagram that shows it all. But this is considered as a bad practice in the UML community, because such diagrams are not readable, almost immediately obsolete and not maintainable.

The recommended approach is to use several smaller diagrams, that each focus on a particular topic that you want to explain. Like a big paragraph becoming decomposed into smaller, understandable sentences. Also keep the information in the diagram as lean as possible: it will stays accurate longer.

In The UML User's guide, the inventors of UML repeat in every chapter the advice to make small, focused diagrams.

How to minimize costs while maximizing benefits?

If you have a very limited budget, you'd use UML mainly to sketch the big picture: what classes are involved and how are they related? This gives a quick overview. And please, keep the diagram simple: the details for each class can be found in the source code; no need to repeat them.

If you have more budget, add some ADRs to explain the rationale behind the diagrams (the "why", not the "how"), and keep the diagrams up-to-date by including them in the reviews. It’s not a big overhead and can save you a lot of time for growing teams or a large teams with turn-over.

For the interaction diagrams and the activity diagrams, it's similar. Seeing how 4 or 5 classes interact in a sequence diagram helps to quickly grasp the general idea of the choreography, and speedily find out what to look for, in which class. Getting the same understanding from the code is perfectly possible, but just takes more time, as there is more code to discover/read.

The key is to avoid the trap of graphical programming: having a sequence diagram that shows all possible interactions with conditional or looping "fragments" will lead to an unreadable unmaintainable diagram, obsolete at the next commit. The idea should be to illustrate only some key scenarios with the most relevant interactions. As Grady Booch puts it well on Twitter:

Models are meant to be an abstraction of some thing, not just another complete representation of that thing.

So in conclusion, use the diagrams to show the relationships that can't be easily found in the source. Leave the details in the code and the "dark comments" (comments in dark mode?) - no need to repeat anything ;-)

Improve this answer
6
  • 1
    these are some very useful hints. Seems like you had similar problems a time ago and found a solution. "So in conclusion, use the diagrams to show the relationships or the interaction flows that are not quickly found in the code" speaks it out well. A class diagram/ERM seems to be done fast and give much documentation. As well as making activity diagrams or flows of not too obvious processes, if I understand you right. No need to have 100% of coverage, at all. I will do this, and further try to let the code be as solid and speaking as I / we can do. Balance budget/time vs benefit.
    – Edgar Alloro
    Oct 10, 2022 at 20:16
  • When trying to find a balance between budget/time and benefit, I think it is wise to take into account also the capability of each developer's intelligence. Lastly humans try to understand a system, and improve it. The capacity of a human brain is not comparable to a class diagram, that grows too big. I guess it is no problem to know/feel 80 Entities in your head and know what they are for. But putting them into a Class diagram exported as PDF, then looking at it will confuse you and your team-mates.
    – Edgar Alloro
    Oct 10, 2022 at 20:21
  • 2
    @EdgarAlloro There is no contradiction here. To understand how 80 classes are related with each other, a developer has to read at least 80 files, and create a map in the head. Unfortunately, short term memory limitations make us feel any diagram on the screen or mental map in the brain with more than 6-8 items as complex to understand. So it’ll take time to grasp the 80 files. With the diagrams, you’d make 10-12 diagrams, each with less than 10 classes, that will tell the full storry in little weel-thought chunks. What is easier to read: 80 files of code, or 10 simple pictures?
    – Christophe
    Oct 10, 2022 at 21:05
  • Once you’ve seen the pictures, you’ll see how the class telate and go to the relevant source files, in a step-by-step guided manner. This can be much more time effective.
    – Christophe
    Oct 10, 2022 at 21:06
  • word. appreciate it.
    – Edgar Alloro
    Oct 10, 2022 at 21:11
10

If you have 0 documentation, jumping right into code-level documentation is going to be expensive and unnecessary. In the past, when I've inherited projects with no documentation or that unexpectedly grew to a point where more than code was needed, I've had good luck with Arc42 and the C4 Model.

Assuming that your code is well-structured and followed good practices for your language and framework, most of the information that people need to know tend to be in the higher levels of abstraction. They want to understand the context in which the system exists, the fundamental decisions that guide the system, and the major pieces and how to find the code that corresponds to those major pieces. Having to do a deep dive into modeling the code tends to be reserved for the most critical systems or the most complex aspects of a system.

The good thing about the Arc42 documentation system and the C4 Model is that it can be applied recursively, starting at the system level and getting into more detail as necessary and as time permits. Since the higher levels of abstraction tend to be more stable, you can stop building out the documentation at the level at which the costs of supporting the documentation outweigh the benefits of having it for developers in the future.

Of course, there may be other documentation structure or modeling languages out there that you can apply in a similar way, but I haven't found one that's as comprehensively laid out and synergistic as these two. Based on your specific situation, you may want to tailor the formats, but this should give you a better starting point for building useful and valuable system documentation than jumping right into highly detailed UML models.

Improve this answer
5
  • 1
    If I understand you right: Documentation in words would be a first step in my case, before (and If at all) putting every detail into visual representations / UML. I will have a look at Arc42 and C4. Never heard of it ... Thank you!
    – Edgar Alloro
    Oct 10, 2022 at 12:48
  • 1
    @EdgarAlloro No. Graphical models are good. However, UML is probably too detailed. The C4 Model is a graphical architectural model that uses UML at the lowest level of abstract - the Code level. A lighter-weight notation would help at higher levels of abstraction and may be all you need. Words are also important, though, but some things are best expressed with pictures.
    – Thomas Owens ♦
    Oct 10, 2022 at 13:04
  • @ThomasOwens Not to be nitpicking here, "the UML" is not just massive class diagrams with every little auxiliary class spelled out. You can draw a couple of boxes representing components of the system, a couple of lines that detail their interaction and you have a valid UML diagram of a very high-level view.
    – arne
    Oct 11, 2022 at 11:22
  • 1
    @arne I never said that it was "massive class diagrams with every little auxiliary class spelled out". UML has a lot of symbols and rules. It's heavyweight. You need to make sure you are following all of the rules of the language, even for the most abstract diagrams. Using a lightweight notation, like C4 Modeling, has a much smaller set of symbols and rules that is easier to learn, draw, read, and maintain.
    – Thomas Owens ♦
    Oct 11, 2022 at 13:44
  • 2
    While text and pictures are both good, I've found that lists and tables can be really useful. (Easy to edit, too, which is vital.) For example: lists of apps/programs, servers/installations, inputs/data sources and outputs, types of data you process, admin tasks, common issues… All in a summary form, with just enough info to be meaningful, and links to relevant pages/servers/repos/&c. I've found that sort of overview really helps to get a mental model of everything; from there, you can drill down to further info or to the code, with enough context to follow it.
    – gidds
    Oct 12, 2022 at 16:33
3

Documenting everything is certainly possible, but in most cases is just not worth it. Even if you manage to document everything today, you must maintain it for the future. Additionally, show me an engineer who would be excited to read through (and update) a 1000-page long document. Chances are, such a document will be obsolete even before it is finished, and no one will ever read it.

It could be enough to use tests and code as documentation for systems that are not business or safety-critical.

  • Make sure you follow the best practices for the technologies you use. Code is the truth! Thus, anyone proficient in React or whatever other technologies you use could be able to understand the system at a low level.

  • Have good unit test coverage to be used as documentation for classes/methods/functions. A good unit tests suite will demonstrate what inputs a method accepts and what outputs to expect.

  • Have a good acceptance test suite to test (and showcase) the system's main functions on a high level.

  • Have a few diagrams outlining how everything fits together.

This way, you gain 2 benefits for the same effort - A system with low TD and good automated test coverage doubling as documentation.

Improve this answer
8
  • 2
    > "Chances are, such a document will be obsolete even before it is finished, and no one will ever read it." We have tons of this where I work. I keep links to a few specific files with known useful information, but the rest is assumed to be out of date and thus ignored.
    – Yay295
    Oct 11, 2022 at 5:51
  • 3
    We made the same experience: Most devs don't bother to check external documentation, they dive directly into the code. Thus, it's often better to invest the time into improving code readability rather than into writing external documentation.
    – Heinzi
    Oct 11, 2022 at 11:15
  • 2
    @VLAZ: You are completely right. However, in my experience, most devs ignore that and still dive into the code instead of looking for external documentation. :-) And given the state that most external docs are in, I can't really blame them. We try to solve this by putting relevant documentation as close as possible to the code: We prefer code comments to external documents, and if we need external documents, we try to put them in the same folder as the source code, not somewhere else. But even the code comments suffer from becoming obsolete. So far, nothing beats self-describing code.
    – Heinzi
    Oct 11, 2022 at 11:59
  • 1
    @EriksKlotins I agree. My point was that examining the code alone can make you miss the forest for the trees. And while developers like doing that (I'm not ashamed to admit I do it myself), it's not the best tool for understanding larger code bases.
    – VLAZ
    Oct 11, 2022 at 14:12
  • 2
    Acceptance tests, whether manual or automated, are your best approach. Diagrams for the complicated parts are second best. The only thing I would add is that diagrams are a design artifact produced before code is written -- it best describes how the system will be changed not how it is. If the code is already written, passing/failing tests with a summary of the main parts is the best balance. Documents are obsolete the moment the code is written. You cannot account for engineering challenges in a UML diagram.
    – Greg Burghardt
    Oct 12, 2022 at 18:19
1

It's almost certainly not cost-effective to start a project that aims to produce complete documentation. You'll never get to the end of the tunnel.

Write documentation that gives the big picture (what are the top-level components of the system and how do they relate to each other). And write documentation for any key internal components - for example a data structure that's very widely used across the whole system and that needs to be under careful change control because changes can easily break things.

For most of the system, people will prefer to read the code rather than separate documentation, because they know that the code is the ultimate "truth" about how the system works.

Notations like UML are useful to aid communication, but don't get sucked into believing that they are mandatory. If you can find another medium that communicates more effectively (like recording a video), then use it.

Documentation that's actually live-linked to the code is the best sort. For example, write schemas that describe the structure of key data flows, and ensure that the data is actually validated against the schema.

Improve this answer
-1

Not answering your question directly, but your time might be better spent writing tests. Unit Tests for cases that are hard to reach and Integration Tests for everyday use cases.

Even after the fact tests can be used to "document" the as-built behavior. This gives some measure of protection from regression bugs: "You fixed this, but broke that." Your successors may just disable broken tests because they are foolish. Or they may actually be grateful, you saved them from shipping a breaking fix. You could even save yourself from having to remember every detail.

Improve this answer
1
  • I did not downvote. If you go to work on a project as a maintenance team member, you will need to know the requirements, the architecture, the database model, etc. Correct? Using test cases to derive all of the above is not practical or complete solution.
    – NoChance
    Apr 25 at 14:56

Not the answer you're looking for? Browse other questions tagged or ask your own question.