How much time should you spend planning a commit before writing code?

Question

At the moment I'm spending more time planning out a commit than actually writing code when adding a new feature. Less than two hours would be lucky, and sometimes I'd spend a good part of the day without writing any code. This is making me unhappy, since I don't feel I'm productive enough (I'm living with my parents, and have never been employed as a programmer).

If I don't do this amount of planning, I just end up writing code that will have to be undone before I commit, and this just messes up my project, because I don't like wasting any code I've already written and try to recycle it as much as possible (my precious).

Someone said that programming isn't about how fast you can type; it's about how fast you can think. I'm not very good at thinking fast.

I think I'm overly cautious making my productivity not economically viable, but even still it’s far too easy for me to waste a whole lot of time making a mess of my codebase.

---

Travis asked

Can you explain what you mean by "planning out a commit"?

I guess there's the time spent architecting, i.e., planning out the object hierarchy, which thread will do the work, GPU or CPU, planning polymorphism for m to n relationships, and which asynchronous pattern should I use.

Then there are implementation details and parameter choice for scientific computations. I like to think about how I could iterate, so if I get a bad result it is rectifiable.

Breaking down a feature into a series of behaviours, which you can verify correctness at each step. I suppose I think about how to verify correctness of intermediate steps a lot before I've even started.

Why does the code you write have to be undone before you commit?

Well, it's easy to write code that's unmaintainable, and difficult to debug. So I have to backtrack and write something more structured. I also sometimes overlook some detail that makes my first attempt not viable.

---

"Planning commits" is just what I came up with to communicate when you've finished one feature and are moving on to the next (obviously committing your changes first). You've got no Git changes and haven't yet written any code committing you down one path.

There's one big commit that gets the scaffold in place and need lots of planning, and followed by a couple of smaller ones that don't need any planning. So maybe the commits in question are more like new branches. (It's just that I don't use Git branches.)

Answer 1

Firstly: when coding for a living, especially as a junior in a team, typically not much design work is needed. This is because you'll be working in an existing code base. Chances are, you'll often be working on a feature that is similar to existing features, so you can look at those as an example. This is nowhere near as boring as it may sound; you'll be learning plenty of things, still need to understand the examples and adapt them to suit your needs.

Designing something new can indeed be much harder, but it is also much rarer, especially when you're a junior developer.

That is to say: in a typical junior-level development job, I don't think you'll run into the things you're worried about. Or at least to a far smaller extent.

If I don't do this amount of planning, I just end up writing code that will have to be undone

I would argue that for most developers, this is fairly normal, if they are developing something that is novel (to them and their existing code base).

I do that planning sometimes, and when I start writing the code, I often realise that the plan translates to an awkward implementation that feels forced or over-engineered.

... which is good to know! Coding provides feedback on the plan.

Planning and coding are very much iterative processes. You plan a bit, try to create some code with the planning in mind, which makes you realise that you overlooked something during planning, so you adjust the plan, code some more, rinse, repeat.

I like to think that this is also how artists work. It's a creative journey. Often messy, sometimes boring, sometimes exhilarating, sometimes frustrating. Sometimes you end up with something boring that just works, sometimes with something beautiful that doesn't work. And, every now and then, with something that works and is beautifully elegant.

[I] don't like wasting any code I've already written and try to recycle it as much as possible (my precious).

Throwing code away is fine! The code has already served a useful purpose: it provided feedback on the planning. It helped you iterate.

Someone said that programming isn't about how fast you can type, it's about how fast you can think. I'm not very good at thinking fast.

Neither are most people, especially if they have to do the thinking without seeing any code. Coding helps to make things concrete. It may also obfuscate the bigger picture, so zooming in (code) and out (planning) is part of the iterative process.

It's also worth noting that people, after years of professional experience, develop a kind of muscle memory for specific approaches, and an instinct for applying these. Which is half the battle when making something new. You cannot be expected to have that already (nor should you expect it from yourself).

Put differently, the examples in existing code bases that I mentioned at the top of this answer are in their heads now, and they can apply them in new projects as well.

I think I'm overly cautious making my productivity not economically viable, but even still its far too easy for me to waste a whole lot of time making a mess of my codebase.

I think you're being too hard on yourself (I can relate). Try to change your mindset to allow the iterative process, to embrace the creative journey.

And code that just works (and is readable) is good enough. If later on, new requirements or new features means that the code is no longer good enough, you can adjust it. That adjustability is the reason the world moved away from specialised hardware and embraced software (running on generic hardware).

Edit:

Also, small steps are good, and focussing on making it work first, and only then making it right is also good. If you already have working code, the planning gets easier because there are fewer unknowns an hypotheticals. Relevant quote is relevant:

“First make it work, then make it right…” — Kent Beck

…in the smallest steps you can manage. — Uncle Bob Martin

(source)

As always, everything is more nuanced in practice, and I don't completely let go of design when focussing on making it work.

Answer 2

I would like to change your perspective for a moment. Commits are not something you plan. Commits, especially in the early stages of figuring out a problem, are little more than save-points along a journey. Instead, spend time thinking about the problem.

Break big problems into little problems, and then break those down into smaller problems. Keep decomposing and digesting problems until your mind can see a single line of code. Write that line of code. Keep writing code as you think. Mess up. Change the code. Change it 15 more times, if need be. Stop and commit when you feel like losing that code would be a big setback, or during a natural break in your rhythm of thought and writing code.

There is no general rule or guideline. When to commit is more a feeling than some methodological practice. Basically, if you think, "it would really suck to lose this code while I figure things out," then commit what you have. Just be sure to work in your own branch so no one else has to deal with whatever state your code is in.

Don't worry about whether the code works, compiles, or looks pretty. Many version control systems allow you to combine many messy commits into one clean, cohesive changeset. Version control is a tool just as much as a text editor. Version control just happens to be really good at backing up your work.

So, to directly answer your question, spend zero seconds of your life planning a commit. Write code until you feel like you have something to lose, then commit those changes so you don't.

Answer 3

You mention live stream YouTubers as a standard to live up to, as if they would make up stuff on the spot, type it in and are done. That is not how it goes. They planned and practiced too beforehand or they show you things they have done hundreds of times before, that are in their routine.

When you see a musician play for one minute on YouTube it is safe to assume they spent days practicing, recorded tens of takes and ultimately picked the best one. And that is not counting the years they put in to get up to their level in the first place.

Writing software is a skill like any other that takes time to develop. And different people specialize in different things. And, in your particular specialty, there will likely always be some people who can do things quicker.

Deciding when and where to optimize is another thing. It seems you tend to be obsessively working to satisfy your own compulsive need to eradicate any imperfections. This is fine in the environment you described and how many great creators started their career. Just enjoy it and learn from it while no one is asking you when it's done yet.

Answer 4

At the moment I'm spending more time planning out a commit than actually writing code

No one cares how you spent your time. They care about what you made (if you're lucky).

This is making me unhappy, since I don't feel I'm productive enough

Make something people use. Watch them use it. Learn what they really need.

If I don't do this amount of planning, I just end up writing code that will have to be undone before I commit, and this just messes up my project, because don't like wasting any code I've already written and try to recycle it as much as possible (my precious).

If that works for you, fine. I plan myself. But I also keep a junk folder I dump code in that I'm never even going to check in and will most likely never look at again. It's ok to make a mess if you clean it up. I also find myself planning and debugging when I'm taking a shower. Bathroom tiles can help you visualize arrays.

Someone said that programming isn't about how fast you can type, it's about how fast you can think. I'm not very good at thinking fast.

It's not about how fast you think. It's about what you think. Give yourself time to do it well.

I think I'm overly cautious making my productivity not economically viable, but even still its far too easy for me to waste a whole lot of time making a mess of my codebase.

I hear a lot of worrying about what you yourself think. You need some external validation. Put your stuff in front of other people and learn what they care about.

How much time should you spend planning a commit before writing code?

Planning is like making popcorn. While the ideas are popping, great. When the learning stops, knock it off and get back to work.

Context switching from planning to coding is hard.

The point of both is to communicate your intent. The earlier you do that the cheaper your bugs are to fix. However, guard against spending to much time in either. Respectively that's called analysis paralysis and gold plating.

Immediately I can make sure I have planned out the night before what I will program the next morning. But I guess because I have nearly finished what I set out to build,

It's nice when that happens but often you find problems only once some code exists. Code you have to be willing to scrap or it will drag you down. Bad code can be a good teacher if you let it. I think of it as a sign post pointing me to a better solution.

I am thinking about getting a job, and my actual output is going to start mattering

Don't fixate on productivity. Communicate. Or you'll spend your time working very hard on something no one cares about.

Answer 5

If I don't do this amount of planning, I just end up writing code that will have to be undone before I commit, and this just messes up my project, because don't like wasting any code I've already written and try to recycle it as much as possible (my precious).

This is your problem. There are two reasons to write code:

1. Writing the final version of the code that solves your problem.
2. Writing code to explore the space and discover the most elegant way to solve the problem.

You can try to do the second in your head or on paper, but it's a lot easier to get a feel for where the trouble spots are if you actually write code. In doing so, you'll often realize that some parts are way too complicated the way you tried it, but that if you just did this and that, then you could rewrite this big ugly section this way and cut the code size by 50%. Every problem already has a most elegant solution; you'll have an easier time discovering it once you're actually writing code. (Probably. Everyone's got their own style, and I can only speak for what works for me.)

If you just start writing code without planning it out first, you'll write a bad solution and need to throw it away and rewrite. If you spend hours planning before writing any code, you'll also write a bad solution and need to throw it away and rewrite, but you'll be less willing to do so than you would have been if you just hacked out a quick prototype. Or as Fred Brooks succinctly put it, "plan to throw one away; you will, anyhow."

(I'm assuming here that you're actually solving a novel problem; if you're just throwing together GUI code for MS Word v512 and you've got a specification and a deadline, it's probably not necessary to prototype. But that sort of coding's boring and won't teach you anything; you'll learn more (and have more fun) by working on hard problems than you will by writing boring boilerplate.)

That said, recycling code isn't a bad idea. But the way to do it isn't to refuse to throw anything out. Rather, if you're writing a new program and some parts are similar to something you wrote before, take a look at that code and see if there's anything that you can modify to fit the new problem. And if you end up using the same code more than 2-3 times, look at which parts are always the same and which parts vary from project to project, and write a few library functions that'll let you reuse the code directly in the future.

Answer 6

You're discovering the difference between "programming" and "development". Developing an application is so much more than churning out some code. All the different little units of code that come together to solve the problem need to work closely together and that takes planning!

As you mentioned, if you don't do this planning, then you often end up needing to destroy code you wrote as it wasn't appropriate. Even with planning, this can happen and it's pretty normal.

I actually recommend committing more often. As others have mentioned, version control is there to provide you a history of changes. Obviously you don't want to release unfinished code, so this is why we have branches in version control - you can have a development branch (or many of them even!) where you commit code you're still working on, and then you only commit the changes to the main branch when you're satisfied that it's all working. It's a huge advantage to be able to see the code that you've since deleted, in the context of where you wrote it, by going back through your commits... you never know when you might need that code that you ended up not using, for some future project.

It seems to me that you would find the process flow for Test Driven Development (TDD) quite convenient. People apply TDD to their work in varying degrees, but the general idea is that before writing the actual code that solves the problem, you write the tests for that code, so that you know what needs to be fed in to a method, and what needs to come out of it. It ends up providing you enormous clarity of what needs to be done. The great thing about tests is that you write them in code too, which helps scratch that "but I need to do some real work" itch.

When done effectively, TDD, as well as Unit Testing in general, makes for a more reliable development process. It also aids collaboration with other developers greatly as it serves as a type of documentation of the intention of your code. You (and others) can refactor code with confidence, because as long as the tests pass, the new code is doing the right thing and the rest of the application is going to get along with it. Of course, this is providing that the tests cover all cases - if you find bugs that slip through, then it's time to add some more tests! This is normal too.

My final piece of advice: when researching anything like version control branching, TDD, unit testing, etc., don't get too hung up on doing it "perfectly". Do what works for you. I find TDD very powerful and effective, so I do it - but I still don't do every single thing that the books and sites say to do when doing TDD. I do what works for me, with the goal being to deliver effective, high-quality code that can be understood and maintained by others (including future-me who might still be looking after the same code years down the track!).

P.S. some bonus (potentially controversial) advice: Try to prefer written documentation, tutorials, and discussion over their audio-visual counterparts. It's a lot easier to find the information you're looking for faster by scrolling than by scrubbing through a video, you can paste excerpts into your own notes, and, in my experience, there's a lot more content available.

Answer 7

I did a test. I wrote an average-sized line of code with a timer running. It took about 7 seconds.

Let's do some math.

If my working time is 37.5 hours per week, 47 non-vacation weeks per year, that would be about 0.9 million lines per year. Which is 75 000 lines per month.

The best I have ever reached writing trivial code is 20 000 lines per month (and that was a non-vacation month so the calculation isn't entirely fair). So even then about 27% of the time I'm writing code and about 73% of the time I'm thinking.

However, usually I'm not writing trivial code. Most of the time, I'm doing work that requires careful planning. So I'd say my usual productivity is about at most 5000 lines per month.

I wouldn't say exactly I'm planning what to commit. I'm planning what to write. The commit follows from what I have written, sometimes by committing all, sometimes by breaking it into logical-sized chunks, and even sometimes throwing something away.

Answer 8

I like think of this "planning" as "discovering the solution". The time spent discovering a solution depends on how much you already know about the problem, how much you know about the tools at your disposal, how easy the problem constraints are to satisfy, how complex the problem is. All these things can vary greatly. Sometimes, it takes mere seconds before inspiration strikes, sometimes it takes weeks.

That said, I do have some inputs for you:

• The process of discovery is often iterative

  For any non-trivial problems, discovering the solution consists of several steps, with later steps building on earlier ones. The faster you can iterate, and the more information you glean in each step, the faster you'll be done.

• The process of discovery may involve writing code

  Sometimes, writing code yields information more easily than abstract thinking. For instance, if there is uncertainty about the requirements, it may be more illuminating to get a quick UI prototype into the hands of actual users, than holding lenghty meetings discussing UML diagrams ;-)

• code isn't written perfect, it evolves towards perfection

  As we iterate towards a solution, our understanding of the code we need changes. To reflect our improved understanding in code, we can delete old code, write new code -- or we can edit our existing code! Provided we have written our code in a way that is reasonably easy to change, that's often the most efficient way.

To conclude, software development is about discovering solutions, and efficient software development about discovering solutions efficiently. It's not "writing code" that takes so long, it's figuring out which code needs writing.

(... and knowing that, experienced software developers are quite willing to write experimental code if that speeds up their thinking process)

Answer 9

It's difficult to anticipate things when you don't have lots of experience, because you don't yet have the experience to know what things you need to anticipate.

There is a story about a pottery class where half the students were taken to one side, given pens, paper, as much clay as they needed and a wheel etc. and told they had two weeks to plan the best possible clay vase they could make.

The other half the class were also given as much clay as they needed and a wheel etc. and told they had two weeks to make as many varied clay vases as they could make and the quality was secondary but they were free to learn and practice as many techniques as they could over the next fortnight.

At the end of the two weeks the two halves of the class were brought back together and given a competition in which every student would make a vase and all the vases would then be independently judged.

According to the story (which is probably apocryphal, but you take the point) the students from the half of the class which had made lots and lots of vases (although the quality was secondary) ended up making better vases than the students from the half of the class who had been encouraged to spend a long time thinking and planning out the best clay vases they could come up with.

The point is that - in this story, at least - the real experience came from engaging in the activity repeatedly, not from the thinking through and planning out the activity.

With real experience came real cognitive shortcuts and the ability to anticipate scenarios which really occur when engaging in the activity.

Answer 10

This is not really an answer but a dairy of my thoughts on the topic

Edit 2: I think the answer to the question is you shouldn't work it all out in your mind before writing code. Rather you should work out the pieces on the fly (which requires a leap faith, faith which you need to train up). Rather you should start by coding up pieces that are invariant to your problem (structure and enumeration definitions) first, leaving the pieces that need to fit together with other code (and thus change) to be written up last.

I will claim that orthogonality/separation of concerns is the most important quality of code. I suppose my current approach is to propose a solution (most of the time only in my head). Compare against all my future planned requirements, which reveals lack of orthogonality, and then repeat with a rectified solution. Perhaps this unavoidable for me, and the answer to the question is I do need to plan excessively.

I suppose what makes it hard is that I'm still in the stage of (re)inventing new patterns, rather than using patterns I'm already familiar with. I've started watching Tsoding streams again and find the process of following his thoughts quite therapeutic. It seems as it software development requires some kind of meditative mental state, not required when you are just programming.

---

Edit: I suppose a lot of the planning is a defensive maneuver against yak shaving, the root of all evil. I suppose I have developed obsessive compulsive tendencies about this. Better to think of a way to avoid writing low-level imperative code wherever you can.

Also I think solo-development is highly questionable. It seems to me that software development works by specialization. Having people specialize so that their work is more and less the same, less variation in their work, less working on unfamiliar things.

---

I've spent a year stuck solving my machine learning problem (although I've learned a lot) which I'm now finally deploying, but it took its toll mentally, exasperated by personal issues. I guess this has made me overly cautious in the face of uncertainty, because there's a lot on the line if I can't make it work. Some work on my mental state might help.

On the other hand, live streamers like Tsoding and Andreas Kling demonstrate an amazing mental strength working with high uncertainty and only a small amount surface area of information. It's amazing how little information they need to produce decidability. Also their context switching is far better than mine. I suppose these are senior developer traits that have taken a while to develop.

Also, I believe in IQ, but I come from a mostly lower middle class background, so perhaps I'm not cut out for the more white collar kind of programming I am doing at the moment. Blue collar programming might be more appropriate for me.

Also I think there is a lot of benefit of working for a corporation having them hold a gun to your head for your development. I've never had anyone look to see if I've done enough work. I'm not accountable at all.

My planning mostly involves doodling on a piece of paper, even though I never read any of it. Perhaps my planning should be limited to navigating the code base, visiting places where insertions should be made and writing comments.

This is the first month back to software development and maybe I'm still rusty. When solving a machine learning problem it's a lot quicker to think about what won't work out than to actually train a neural network, because curating a suitable data set is so time-consuming. But programming is different, because you are much more likely to get it right first time, making planning less productive. So I need to recalibrate my patterns.

---

Edit 3: You can always write hacks that get a feature working. This results in bad code. A lot of the time you will be removing code, so its best to not spend the time making it good code until you are certain that the feature will stay. However, good software developers have obsessive compulsive impulses to make the code good when they know it will stay. Inexperienced software developers take more time to make the code "good", so it is no small task to just fix it up when you know a feature will stay. What make good code? I'd probably say 1-1 correspondences atm.

---

Edit 4: I think cognitive division of labour into low stress work and high stress work is important. Certain things like solving compiler errors, and adding features that are decoupled, low surface area, with few moving parts (i.e. text editing work); you know are going to be easy, and you can complete them without running into much trouble.

With other things, you can dig yourself into a hole. You cant understand the complexity before you start, you may spend far too long removing bugs from your fist implementation, you solution may not be robust enough or it's performance scalable. These are the "weeks of work can avoid hours of planning" tasks.

It's important to enjoy the low stress work, while forming better expectations for the complex work to avoid frustration. Difficult work will take time, and you need to accept that and work slowly through it. But slow work is risky, since you could end up wasting you time, and thus demands more planning.

Answer 11

I'm really into just diving in and coding. In fairness I write a lot of crap and constantly embarrass myself looking at my spur-of-the-moment code sketches in hindsight, but I don't write it into our main codebase. I prototype left and right [*]. My philosophy is to try everything that pops into our heads but in a cheap way, away from big codebases, and into small little test projects just like an artist who sketches things every single day in a sketchpad to improve his skills. I'm not saying that's "right" but it's mine.

• [*] One of the first things I did after installing VS 2019 was learning how to make a prototyping project template that included everything I needed from a central library since I'm prototyping left and right.

I also work in a field that demands constant innovation: computer graphics. My best solutions last year that beat everything widely out there become obsolete as someone beats me the next. And I have to beat them again, and I find with this type of field that demands innovation and constantly inventing new solutions to existing problems, you have to take the risk to try things even way outside of the box and also the discipline to throw them away when they turn out to be horrible (which they will be most of the time when you're trying things no one else has tried before or things no else accepted before unless you're some kind of super wizard in which case... please teach me!).

But I'm also a visual arts major besides CS major and I really think programming is almost as much artistic as visual arts, and not so much less scientific than visual arts. And a field that is artistic means it's one where the best solution is the one for which you manage the best results even if your methodology is way different from what's considered orthodox or best, and I subscribe very much to this idea. So when people on here ask me questions like, "What's the best algorithm to sort these elements?" For me, it depends on the programmer and what they find most comfortable and intuitive. It seems almost impossibly unlikely but if they write a bubble sort for 32-bit signed integers that beats my fastest multithreaded SIMD radix sort implementation, then their bubble sort is the best solution since it produces the best results.

And if you're really after best results in a field like this where even an assembly expert can't perfectly predict the dynamic nature of the hardware these days with CPU caches and brand prediction and whatnot, it's counter-intuitive to spend too much time mulling over an idea and getting too attached to it as most of us naturally would without diving in and sketching out the code and testing its robustness, measuring its efficiency with benchmarks and profiler, and having the discipline to toss it away and try something else if it sucks... which is way easier if you just dove in and started coding it sooner than later.

I have many of my colleagues often asking me questions like, "How did you come up with this?" when I come up with an algorithm or data structure that is sometimes as much as 100x faster than our previous solution taking 1/10th of the memory and with even less and more straightforward code than the previous. And I think disappoint them every time with my answer: "I just tried a whole bunch of stuff and measured them all and picked the best solution." All these people are so much smarter than me but they get married to concepts in their minds which don't pass the real-world tests. And I might be too extreme on the opposite side but I never marry myself to anything until my profiler and benchmarks are showing something extraordinary, and I continue with it, and get jaw-dropping results. Anything short of that and I'm ready to delete my solution in my private version control branch and try something else.

So for your question, and especially your personality, I'd suggest taking some of mine. Just dive in and start coding more, but don't do it in a way where it's expensive to throw away your solutions (ex: don't write the code in your product's codebase, do it in your "personal sketchbook").

Answer 12

Tsoding gave a good answer here.

You see how I develop the code. I don't write the correct code right away, character by character. I approach the entire code top down, I sketch the idea; the structure of the code; and I slowly iterate on the details.

I know that I need to handle these tree of cases. I'll outline the tree of cases, and only then start to work on the specific branches (in the context of programming)

You don't really write the code character by character. You're building up the structure top down.

I like to compare programming to drawing, and for that I get so much sh*t because it sounds extremely pretentious. People think I'm being like "the programming is like art. It's like making a painting". I'm saying it's like painting or drawing technically. I'm not talking about artistic expression type of programming. I don't care about artistic expression, it's just a subjective thing in the context of programming.

I'm saying it's like painting or drawing technically. How do you paint or draw technically? You make a sketch; you outline the general idea. Then you go down more and more detailed. You draw top down. You never draw perfectly as you envision it right away, because you cant do that. Once you've started top down, you've gathered more information about what you want. You can see what works, and what doesn't work. And you can make certain decisions.

I think the problems start when you actually have to get the code running before you can see what works, and what doesn't. In such cases, it make be better "run the simulation in you head" (i.e. "planning") depending on how time consuming the implementation is, and how much of a mess it will make of your code base.

But someone also disagreed in the chat:

In my experience, sometimes it's easier to go from top down when solving some programing problem and sometimes it's more apt starting from individual small components and building more complex stuff.

Answer 13

A little story for you. Sir Peter Swinnerton-Dyer, an eminent mathematician, computer scientist, and public administrator, once volunteered to write the bootstrap operating system for a new computer.

He said he thought he could do it in six weeks. He spent the next five weeks walking and resting in the countryside, and his colleagues were getting increasingly anxious. In the sixth week he announced that he'd cracked it; all that remained was to write it down; so he started writing it down, and (the story goes) it worked the first time.

The best programmers have it all sorted out in their head before they write a line of code.