58

When I'm designing a system for a program, I often make misjudgements that will prevent the design from either working, being maintainable, easy to use or all the above. This means I will usually have to iterate several times, refactoring my code until I achieve this.

Recently, I've been trying to think about this as I design, but it has been really difficult and mentally exhausting. The feeling is similar to playing a game of chess and trying to remember all the moves you can make. I'm always coming up with problems with my design as I'm thinking about it and I lose my train of thought. Sometimes I branch off so far, I forget the original problem I was trying to solve!

So my question is, is the iterative process natural? Is me trying to 1 shot a design a bad idea? Are there ways to make this process more bearable?

23
  • 16
    Not writing an answer as there are already great ones out there, but your question is striking at the heart of the differences between waterfall and agile. Waterfall assumes everything is known and can be done right the first time, agile eschews that idea in favor of building things as you go even if that means revisiting/expanding earlier implementations.
    – Flater
    Jun 24, 2020 at 11:35
  • 7
    "Is me trying to 1 shot a design a bad idea?" - yeah. The problem is lack of information; as you develop, you learn more about the system and the problem domain, the rules, the relationships - so you are in a better position to make design choices. The problem is that we tend to build in assumptions too early, and come up with wrong abstractions (this is what's hard to change later on). This is why we have practices like YAGNI and Rule of Three - to minimize assumptions, and things like TDD - to enable refactoring. Jun 24, 2020 at 11:38
  • 2
    Another thing to be aware of is that not all parts of the codebase require the same level of design sophistication. Design takes effort, some research, and it introduces complexity; you want the payoff to be more significant than that cost - and the payoff is greater in those parts of the codebase that change most often. Usually, there's a relatively small core that's the most active. The goal is to identify what concepts are relatively stable in those parts vs what varies, and organize higher-level abstractions around those stable parts (so that you can vary the details). Jun 24, 2020 at 11:44
  • 13
    @Flater it's sad that waterfall was badmouthed so much it's become a complete strawman. In reality, waterfall was iterative approach. Here's the original paper that invented it (www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf), literally the third picture shows the arrows going backwards. Jun 24, 2020 at 20:40
  • 9
    At least under the waterfall model businesses tried to get it right first time, and many of the best systems withstood decades of further development and maintenance whilst being exemplars of reliability and good UI. Now, Agile seems to have given licence to just hack away and keep revising mistakes, and when that threatens to take 10 years to complete the project, the business finally calls time and orders something to be done quickly and dirtily.
    – Steve
    Jun 24, 2020 at 22:43

12 Answers 12

53

Iterating through multiple versions of a design is a great thing to do! It is rare to create a design that has all the good properties at the first try. As software engineers, we should be humble and accept that we will make mistakes or overlook things. It is arrogant to think that you can create good design at your first try.

But as you say, it can be exhausting to work on same piece of code for a prolonged period of time. But there might be practices and disciplines that make it more bearable.

Test automation, preferably TDD

This is the one discipline that enables us to actually change the design. By having a solid and reliable suite of automated tests, the design can be changed drastically without fear of breaking existing functionality. It is that fear which is most exhausting.

Doing TDD also makes it more likely that you create working and 'good enough' design at your first try. This design then requires only small improvements to push it into greatness.

Refactoring

Instead of focusing on changing the whole design, focus on small problems and fix those. Fixing many small problems will result in big changes in the overall design. Making small changes is less mentally exhausting as you get feedback about your design sooner and you can stagger your attention between multiple designs, slowly improving all of them.

Good vs. Perfect

The saying 'Perfect is the enemy of good.' comes to mind here. Knowing when to stop trying to improve the design is a learned skill. If the design is being used and changed, then you will have lots of small opportunities to improve the design, so you don't have to invest all that time in the beginning. As long as you follow the Boy Scouts rule of 'Always leave code cleaner than you found it.', then the design will improve over time.

11
  • 1
    @hanshenrik My take away from that story isn't "A good design on the first try is possible so that's what I should strive for" it's "If you're not Linus Torvalds, don't worry if you can't come up with a good design on the first try." There are very few developers at that level, which is why the industry has coalesced around iterative software development techniques.
    – Jaquez
    Jun 25, 2020 at 15:45
  • 4
    Keep in mind that Linus was intimately familiar with the pros and cons of many VCS architectures, and had detailed, specialized requirements thought up after years of using them full time on a huge project. He was able to fulfill his own requirements after 2 weeks, even if an average developer would likely much rather have used SVN over that early prototype. He's undoubtedly a world class programmer, but I don't think he'd necessarily replicate this amazing feat if given a random project in some arbitrary business domain. Jun 25, 2020 at 17:46
  • 5
    @hanshenrik: Linus Torvalds says a lot of ridiculous things. The original git written by Linus was mostly a bunch of shell-scripts that were pretty much unusable. He transferred ownership of it to Junio Hamano after 2 months (15 years ago), who basically rewrote the entire thing from scratch, and somehow Linus still gets all the credit. Jun 26, 2020 at 0:01
  • 1
    @BlueRaja That's not my recollection. What major data structures or ideas were completely changed between the initial git version and now?
    – Voo
    Jun 26, 2020 at 9:58
  • 1
    I think TDD can be good for quality, but its kind of myth that it can be done just as fast as regular development. It requires more thinking, designing, and maintenance than regular development. And while I think it does produce higher quality, its not guaranteed to prevent regressions. TDD tends to encourages unit tests (though it doesn't mandate unit tests), but some architects feel that unit tests are the least valuable of tests. TDD's wheel house is having lots of junior devs that you can't trust to produce quality. Jun 26, 2020 at 17:57
22

Yes, it is definitely common to have to iterate our designs: We call this emergent design.

There are a number of techniques that can be used to encourage emergent design. Agile is a common methodology of emergent design.

As you have found, humans can't think of everything about a program - even a simple one - at one time, especially when making assumptions about how something is going to work.

Emergent design tackles these problems by saying: "Let's figure out the next right step, and clean up later". Sometimes this means you must go back and re-do a lot of work to simplify or expand upon it (refactoring, as Euphoric mentioned in their answer, is a big part of this).

The reality is the more complex the system, the harder it is to predict what it needs to be. You must create, test and iterate, to see what works. I would say that test-driven development (TDD) is a form of emergent design, and a reliable one in my experience.

Since it is possible to lose sight of your current goal while trying to define the grand design of the whole application, the best thing to do is to try and limit the amount of mental bandwidth you use, focusing on what is relevant right now. That being said, knowing what your goal is helps you to define what you do for your smaller steps. It is an art and a skill to balance emergent design and long-term planning.

An emergent design means the code base is evolving and changing all of the time. The key to successful emergent design is to keep your designs open and extensible. In fact, this is the purpose of the SOLID principles: to make changes to code bases less painful.

17

Ward Cunningham has a great metaphor you can apply to this whole dilemma: Technical Debt.

Imagine, you want to start a manufacturing business. In order to make money, you need machines. But in order to buy machines, you need money! How do you solve this dilemma? The answer is: you take out a loan, use that money to buy machines, use those machines to make money, use that money to pay back the loan.

In your case, you have similar problem: in order to implement the system, you need a design. But in order to know whether the design is good, you need to implement the system.

How do you solve this? You take on Technical Debt. You design the system as good as you can with the knowledge you have right now. Then you implement it. Now, you see some problems with the design.

This difference in knowledge between what you learned about the design while implementing it, and what you knew when you started the design, that is the loan you took. That is your Technical Debt.

And just like real debt, if you don't pay it down, it will accrue interest, and slow you down. So, you need to refactor your system, and the basic idea of refactoring is to make your system look like as if you had known from the beginning what the best design would be.

That is the fundamental idea behind Technical Debt, and the fundamental idea behind Refactoring.

This is very important. Some people will call it "Technical Debt" when they are taking shortcuts and "will fix it later". That is not Technical Debt. When you know that your design is not optimal, then it is not Technical Debt. Technical Debt is when you don't know that your design is not optimal (or at least you don't know in which way it is not optimal), but you need to build it first to see that. If you take shortcuts, it's not Technical Debt. Only when you build the best possible design you can with the information you have, is it Technical Debt.

(Unfortunately, I don't think there is a widely-accepted name for the other thing, so people often call it "Technical Debt" because it seems to kind-of fit, and there is no better term.)

There are lots of ways to make this easier, but this is the foundation. One important way to make this easier is the idea of Baby Steps. You only design the absolute minimal system that is required to take the next step, then implement, refactor, and then, and only then take the next step. And, you make sure the steps itself are as small as possible.

If you only design a tiny bit of the system that takes you half an hour to design and implement, then you only have to refactor and re-design half an hour's worth of work. Whereas if you spend two weeks designing and two months implementing, and only then start refactoring, you will be crushed by a mountain of Technical Debt.

The idea of baby steps ties into the idea of the Minimum Viable Product, which is the smallest, simplest possible product with the minimum set of features required to elicit constructive feedback from the customer.

13
  • 2
    I always use the concept of technical debt, but your explanation is fantastic. Only thing I would criticize is the notion that a shortcut isn't technical debt. The reason why the debt was taken is different (for example there might be a business requirement to get some prototype out of the door as soon as possible), but it's still a technical debt at the end of the day. Jun 25, 2020 at 11:53
  • Hmm, that might be right. When Ward Cunningham coined the term, he was applying it specifically to the dilemma of not knowing what the right design is until after you have built the system, so knowing what the right design is but choosing not to implement it to have an advantage in time-to-market does not really fit that definition. It is, however, still qualitatively different from the situation that I want to describe, which is that I know the correct design but push it off for later because of laziness. I feel there should be three different names for that, but yours is definitely closer … Jun 25, 2020 at 12:29
  • … to Ward's scenario than the my second one. Jun 25, 2020 at 12:29
  • 2
    @peter_the_oak Oh, any form of refactoring is a challenge to explain/sell to customers (internal and external), because that kind of work involves a lot of effort spent on an end goal of changing the system as little as possible, from a broad-strokes enduser perspective. Performance upgrades or big, flashy frontend redesigns are a no-brainer, everybody loves a new coat of paint. But it's tricky when the main benefits are intangible things like, "Once this is done we'll be able to start adding those new features", or "After this we'll have fewer concurrency problems in high-activity periods."
    – FeRD
    Jun 26, 2020 at 8:13
  • 2
    @FeRD: That's one of the reasons why Ward Cunningham chose the particular metaphor of "Debt": he was working at a financial institution at the time, so they understand the dangers of not paying down debts. Jun 26, 2020 at 9:22
8

Yes, you will not get it exactly right the 1st time.

Just like I did with my answer. I completely erased it after writing some 5-6 paragraphs, because I came to understand that my chatter might be answering your literal questions, but it was not really soothing your actual troubles. So I took a different path instead...

  • Define iteration.

What counts as an iteration is almost always dictated by the task(s) at hand. Iterations of larger projects may last months on end, while smaller projects might benefit from weekly iterations. Sometimes you may find yourself improving the design on a daily basis. Make sure you have a realistic timing, otherwise, you end up changing the design too often and this will wear you out in the long run.

  • Define right.

"Right" can be very evasive when designing... requirements change, and you will have to adapt your design at some point, regardless of having gotten it right previously. Yes, the process of design often starts out with a "requirement volatility" handicap. Try to discuss and negotiate what "right" means to you and all people involved in the process. Agree on the level of "rightness" you are all going to accept, so that you know approximately what you are aiming for.

  • Figure out why.

You are not designing a system just for the fun of it (and chances are neither just for your own personal use). More often than not, you are creating code that actual people will use to achieve a desired task. Figure out the standard task (e.g. they want to quickly and easily convert documents to PDF files), make some perturbations to somewhat "amplify" the required functionality, so that you can err on the safe side. Be reasonable, for example it is reasonable to assume the users might want two-way conversion capabilities, or opening the converted file automatically in the end. It is not reasonable to expect that the user would want to make edits to the converted PDF files, that would be a different program, with different needs. Don't be hard on yourself, you can always enforce (reasonable) limits to what they expected.

  • Find the junctions and shape the unknown.

Remember how, sometimes, your favorite development environment and APIs don't really hand you what you would like directly out-of-the-box and you have to write some really dirty code to prevent the MouseDown handler from re-selecting your item on a listbox, while your manual code actually unselects it if it is already selected and you click on it? Notorious methods running two different code paths based on a passed bool argument, one with a default value no less? Or would you prefer to feel the pain of manually doing dynamic dispatch by checking types at run-time? Some huge if-else branch chaos?

While I hate to downplay such important stuff, my point is that those are not your actual design problems and, still, they are quite representative of what you come across when you hear software developers complaining about what a big pile of mess part (or the entirety) of a codebase is and how it needs to be re-written. Your actual problems are bigger, but you can protect yourself easier. The real trouble comes from not building abstractions where they need to be, so that you can, at least, redirect the code to your very own "safe-houses", where you can play your dirty little tricks.

Junctions are those connections in a design, where you might want to do things differently, based on circumstances. The most typical example is the coupling of the GUI with functionality. The job of the Graphical User Interface is, typically, to redirect user interactions to you. This is a perfect example of a junction. Upon receiving user input, you can either couple the interaction to a single code path doing something very specific, you may, instead, define a junction, where you do different things on different occasions.

By "shaping" the unknown, I mean that you need to determine most (or all) holes of your design. And this is where it becomes an oxymoron... The art of designing (in my opinion, of course) lies not in knowing where you are stepping, but in knowing where you are not. So you have to decide and document the parts that you will never ever get to know adequately well, i.e. the parts that may be mobile, volatile, if you will. Which brings me to:

  • Holes are abstractions.

Design a screwdriver. You need to hold it with the hand and screw/unscrew, so you need a piece that goes into the head of the screw, and a rigid body (to withstand torsion), as well as a large part to apply a firm grip. Unless you are designing a screwdriver that needs to be held with something other than a hand, your grip does not really need any significant flexibility, it's a big strong block of material, prolonged, suitable for tight holding. Human hands are not that different after all.

Screws, on the other hand, have multiple different fits, lots of different cases indeed. Therefore, the part at the other side of your screwdriver, if you only design it with a specific shape, you are really only covering a very small range of your requirements and you will need to build another screwdriver with a different head to serve an additional part of the requirements. So, instead, make this piece undefined, build a hole. Create a screwdriver with a hole and expect that, somehow, a head will be magically fitted externally. I think you are getting my point.

In short, to ease the pain of iterations:

  • Try to "need" less of them by...
  • Finding the important things (junctions and holes) and only worrying about those. Don't let other kinds of problems get to you while designing.
  • Focusing on your user stories to know what you are dealing with. If you don't have user stories, or haven't talked with someone that will use your system, stop designing. Imagination is rarely a good guide, we tend to go crazy when left alone designing our "dream system". Be real, talk to potential users, acquire user stories and build on those.
  • Creating enough holes to make your design adequately flexible. Changing the implementation plugged into a hole is not an iteration; yes, functionality may change, but your design stays the same. It's good and adaptable and amenity to such changes is a wonderful attestation to that precise fact.
2
  • 2
    A lot of this is gold. I've got one remark based on my own experience, though: When I design a system, I strive to not introduce any limits to my design that are not strictly required by the problem domain. I try to remove everything that has a somewhat arbitrary look, abstracting the solution as far as possible while still being to-the-point. I find this important because I firmly believe that it's my job to empower users: It's my job to give them tools that they find productive, and which they can use for purposes I never imagined. It's not my job to tell them what they can't do. Jun 25, 2020 at 11:22
  • 2
    Ironically, this strip-all-limitations approach typically tends to yield simpler, faster, more flexible, and thus more powerful solutions than a build-to-fit-exact-requirements approach would. User stories are only a guide to which must be supported efficiently, but they don't tell me what not to support, so I'd rather support it anyways if it doesn't do harm. Jun 25, 2020 at 11:25
7

Some good answers here but I think one thing people often lose sight of is the meaning of 'soft' in software. The fact that we can easily iterate on a design and see it in action is the great advantage of software. It's much more difficult to be successful at something when you only get one shot or the cost of changes is prohibitive.

If you were building a skyscraper, it's really important to get the foundation right. Both the design and implementation need to be nearly flawless. This means that the time and effort and therefore cost of reviewing those plans is high and very difficult to reduce. No one thinks this is a good thing, it's just a matter of fact.

Consider if your program needed to be fabricated on a chip and put into mass-production. A flaw in that design would be catastrophic. Guaranteed, you would want produce multiple prototypes before you went to production.

The history of programming has been a pretty clear trend towards faster iteration. In the old days of punched-cards, programmers would write their code on paper and review it extensively before submitting it because they might only get one shot at running it a day. Now, we can type out a little code and run on the scale of minutes. This is highly advantageous from a productivity standpoint. There are rapidly diminishing returns on the time you spend noodling over a design before trying it. If you don't embrace this kind of iteration, you are losing-out on a major advantage.

2
  • This depends on the domain, right? Looking at a continuum between (a) mobile web games with instant-update capacity, and (b) software to take photos of Pluto from a spacecraft ten years after launch, there's big difference in iteration time between "real" test opportunities. Jun 26, 2020 at 18:32
  • 1
    @DanielR.Collins I think you may have misunderstood my point. The iteration need not include the 'production' release. For example you can write the software for the spacecraft and test it many times (thousands, even millions of times) before you deploy the probe. If you didn't I would question your competence. If I'm designing a rocket that costs $100 million to fire, I'm going to spend a lot more time thinking through and checking my plan. I would even go as far as creating software so that I can 'test' it virtually.
    – JimmyJames
    Jun 29, 2020 at 16:59
6

is the iterative process natural?

Absolutely.

Is me trying to 1 shot a design a bad idea?

Maybe.

Sometimes I may already know the design for a working solution, because I've done something similar before. In fact I may have several plausible designs in my back pocket, and then I can choose which is the best fit for the requirements.

Unless it's an area I've covered really exhaustively, maybe none of my existing designs are absolutely optimal - but often one may be good enough.

If I'm exploring a new problem or solution space though, I'd absolutely expect to write a bunch of experimental sketches and disposable prototypes before settling on a design.

Are there ways to make this process more bearable?

Don't pick a design and start running until you hit a wall, and then start over. Instead, make the process of exploring the design space an explicit part of your work. You should have things you know are experiments, and things you know are disposable (and which you commit to actually throwing away).

Eventually you'll have a good enough overview of the space that you can describe multiple different solutions, and how they differ, and how you chose the correct one for your current needs.

5

Design patterns in general fall more into the realm of guidelines than rules set in stone. The iterative process of software development is quite normal. It becomes less so as you move ahead with your process and learn the restraints and constrictions of your tooling. In the end, you can only discuss about hammers if you know to use one.

I don't recommend you look for the perfect design, as it will never happen. Even design patterns that have been used for the longest time (think the singleton pattern, factory patterns, etc.) still get adaptations to the tools we use to this day. And morph in meaning as software development as a craft evolves.

At the end of the day, it's not about the best design, but the design that manages to solve your particular problem.

4

Essentially, your question is really about development cost. Changing a design retroactively is hard, so you wonder if investing extra effort in the beginning to avoid having to refactor a design in the future is cheaper/easier than dealing with the refactors down the road.

Iterating on a design is inevitable, since it's unlikely that you operate from a place of perfect knowledge about current and future requirements. So it's not even a question of whether you should one-shot a design or do iterative design; you'll do iterative design whether you want to or not because reality will force you to. That doesn't mean that you shouldn't try to get the best design that you can upfront, and there are cognitive tools that can help you with that task, but you should also know not to aim for perfect. In other words, know when to quit predicting future problems and start implementing.

So how should you approach design knowing that you will have to refactor in the future? You become sensitive to design decisions that limit your flexibility, which is why I don't like the guideline that you should design the most minimal and simple thing possible. That's not how you get flexibility. You get flexibility with it sometimes, but sometimes in order to be flexible you need some infrastructure, which certainly isn't going to be the most minimal implementation.

For example, if you're designing a language, you're going to need a parser that breaks things down into tokens, builds phrase analysis trees, and so on. You might feel like you can get by without it if the current specification of your language only has a few simple phrases and try to implement the parsing with regular expressions or something, but if you know that your language will grow past it and is intended to be useful in a wide variety of situations, you better have that proper parser implemented from day one.

As for the mental overload you described, here's a good middle ground between not doing any design and doing a perfect design which you can use to make things easier for you:

Don't try to create a design that addresses future requirements; try to create a design that models the problem domain. You don't have to model the problem domain in full detail, but you should incorporate the main canonical properties of the problem domain in your design right from the beginning. For example, if you're modelling a zoo, the main canonical properties of animals should be first-class citizens in your design, even if you're not going to be using them right now. You can leave them for later only if you know for sure it's going to be really easy to incorporate them in retrospect. Otherwise, code them in.

This is a good compromise because it's much easier to identify this subgroup than to identify all possible problems you might run into, it doesn't require you to guess future requirements, and the main properties of a problem domain are where future requirements will pop up, so it would be very helpful to have proper flexibility around it built-in when they do. If you did something silly like not coding a reference to animal classes into your design because the current requirements didn't require it, it's going to bite you in the ass later because an animal's class is one of the most important properties of the animal, so of course your zoo owning customer will need to refer to it in the future in reports, purchasing decisions, operations, and so on.

2

When I did early development, I often found myself scrapping and rewriting entire segments. Obviously, that's time consuming, and I've developed a number of approaches to solving it:

Assumption Testing Model

A lot of re-writes occur because our understanding or assumptions of how something works is often incomplete or flawed. Absence of documentation, bugs in how a third-party function works, or 'quirky behaviour' are all spanners that can ruin our assumptions, and also our design, which forces rewrites.

So your goal is to find as many incorrect assumptions as possible, as quickly as possible, by writing barebones code that tests your assumption. Does the HTTPS call really accept a 2MB long string? What does the third party library call to DownloadPDF actually do in the event of an error?

The best way is to conduct tests of your assumptions by writing small pieces of compilable code that tests the assumption. I usually do it on the function level, but even short, single lines of code are sufficient. It's a form of rapid prototyping that follows the fail fast, fail often approach.

By doing this, you catch gotchas and quirky behaviour early, which avoids you committing to a design that, at it's foundation, is flawed. Maybe the 2MB call to HTTPS crashes. Maybe it runs too slow. Maybe it throws a bunch of errors. You don't know until you test your assumptions.

Once you know the assumptions are valid, you can write the full, in-depth code around the prototype you've built.

The other approach is:

Measure Twice, Cut Once

Spend longer in design and research than you normally would. Use a piece of paper, doodle down the approximate design. Make sure you understand as much as possible. Any parts you're not sure of, do assumption testing on, or try to find documentation or online working examples of.

Try to avoid reinventing the wheel, use other people's stuff if it works, unless by necessity, and if necessary, try to re-use previous code you've written that you know works. Designing slightly slower, but more stable, more reliable code that you're pretty confident will work is better than trying experimental, faster code with a lot of assumptions.

Finally:

Clearing Technical Debt, The Smart Way

A lot of people, when they see code with a minor flaw or inefficiency, have a nagging OCD to immediately correct the problem, re-write or overhaul the code. Often they'll change one function, that requires another that calls it be updated, and the documentation, so on and so forth.

Instead, if the code is stable, you should document a list of all the improvements you'd like to make, and wait until you have a sizeable enough list, sketch up a design that incorporates as many of them as possible, and then implement it all in one stroke.

The time consuming task isn't finding the bugs or the design, but implementing the re-write, so you should wait until you have something significant to rewrite. If it's a major bug or serious flaw, by all means rush an urgent patch through, but if it's minor and can be dealt with by a quickfix, do the quickfix and save the redesign for the next major overhaul.

Summary

  1. Test assumptions so you don't build on a flawed foundation
  2. Spend more time in design so you spend less time rewriting
  3. If do you rewrite, lump as many rewrites together as possible
0

Have you ever considered doing some requirements engineering and verifying your architecture before your build it? The Requirements Engineering (RE) area of CMMI for development covers this general thing.

A simple method for simple systems:

Draw your proposed architecture as block diagrams with enough detail. Get all your use cases. Manually trace all your use cases through the architecture. Can it do them all? Update the blocks with newly found responsibilities.

Don't forget change of the system as use cases. Networked pieces will need to work cross-version during updates.

All the "ility's" are use cases. Reliab-ility might be best inspected by considering various blocks failing if they have independent failures. Maintainab-ility might involve thnking about making the interfaces between modules better. David Parans wrote a paper about effective design of modules in 1972. You might learn something from it.

Consider lifting the level of rigor to making executable models like TLA+ if the design is distributed or concurrent.

1
0

"So my question is, is the iterative process natural?"

Yes! Getting everything right the first time is difficult and time consuming and, therefore, very expensive. In most development shops (outside of aerospace or defense contractors) it's not cost-effective.

"Is me trying to 1 shot a design a bad idea?"

No! You should just get better at it. Getting better probably does include getting more things right the first time. You can't help that as you gain experience. But that's not the most important thing...

"Are there ways to make this process more bearable?"

Yes! Where there are decisions you are unsure of, your design should make it easy to change your mind and do something different. When you see other stakeholders sweating over choices that they think are "important", you design to make it easy for them to change their minds. Design using the single responsibility principle to separate concerns, recognizing that each of these "deciders" is a source of requirements, and each of these decisions should be isolated from the others.

In this way, you make the most difficult decisions, and the most change-prone decisions, as inconsequential as possible.

Then you don't have to worry so much about getting them right.

Also, read all the stuff about the solid principles in Uncle Bob's blog. Start with these:

https://blog.cleancoder.com/uncle-bob/2014/05/08/SingleReponsibilityPrinciple.html

https://blog.cleancoder.com/uncle-bob/2016/01/04/ALittleArchitecture.html

https://blog.cleancoder.com/uncle-bob/2014/05/12/TheOpenClosedPrinciple.html

0

I've found that writing software isn't that different from writing a book. Sometimes you'll be able to get all the ideas clear in your head before you put pen to paper, and then you can just start at the beginning and keep writing till you get to the end, and when you review it, very little needs to change. It's said that when Peter Swinnerton-Dyer was asked to write the first operating system for the Titan computer, he spent six weeks walking in the gardens thinking about it, then wrote it down, and it worked first time. Sometimes when I write an article it's like that; other times I have to try out lots of ideas and throw them away before anything starts to make sense. I do think that "think hard before you start coding" is often a good motto; but if you're unfamiliar with the technology you're using or don't understand the constraints you're working within, then incremental prototyping (with a lot of throwing-away) works much better.

For the last 20 years I've been developing the Saxon XSLT processor. For the first couple of years there were major changes to the internal architecture at each release: it took quite a while to work out what the key components were and how they should interface to each other, and I was very prepared to make radical changes when I felt them necessary. Since then it's been more a process of preventing architectural decay by fine-tuning of interfaces and data structures as the requirements evolve.

Not the answer you're looking for? Browse other questions tagged or ask your own question.