- Both methodologies have real world applications.
- Both methodologies can be overapplied and lead to inefficient results.
- Paul Graham is focusing solely on newbie programmers, overstating himself, or he's overapplying his methodology to the point of being detrimental.
This is classic agile vs waterfall
Agile and Waterfall are two development ideologies that are mostly orthogonal to each other, but they are both valid ideologies in their own right.
One ought to figure out a program completely on paper before even going near a computer
This is waterfall to a tee. You do one thing until it is completely finished and should not be revisited ever again, and then you do the next thing.
Note of course that real-world waterfall still allows for error correction (no one is that perfect), but the point is that waterfall assumes that what you are building is exactly what you will end up needing.
Agile, however, is born out of the realization that when doing waterfall, your assumptions about what you will end up needing are often wrong enough that they cause more problems than they solve. For instance:
- You may have overengineered something and wasted time on doing so, also making the rest of development harder to work with this overly complex implementation.
- You may have underengineered something, and because you assumed you were building the right thing, you coupled things too tightly and are now force to make heavy breaking changes.
- The customer has seen our demo and has tweaked the requirements (added/changed/removed some); which inherently means some of our assumptions about what we would end up needing are no longer correct, and all logic that depends on these assumptions needs to be revisited. If we already overengineered things, that becomes quite the time sink.
The main takeaway here, if you're working agile, is that it would've been better if you had not assumed anything that you didn't need to work out yet.
Fulfill the requirements you were given, but nothing more. Don't make a framework out of a small helper class. Don't implement the entire data structure from the get go.
Agile expects you to revisit and rework/expand it at a later time, trusting that your assumptions today are more error-prone than your better-informed assumptions tomorrow.
Everything has a drawback
Both Agile and Waterfall have their uses, I'm not telling you one is better than the other. But too much of anything is not good, by semantical definition of "too much". Agile and waterfall have different "too much" scenarios.
If you over-apply waterfall, you can run into issues because:
- You've needed to pre-emptively discuss so many things, making the discussing theoretical, abstract, and exponentially more complex with each passing day. As the complexity of the analysis grows, people struggle to communicate clearly more and more.
- You're having to make so many decisions that you end up with a bout of analysis paralysis, where you feel unable to make a decision and therefore waste time deferring that decision to the future.
However, when you over-apply agile, you can run into issues because:
- Developers are no longer making any reasonable long-term considerations, which may have led to clean coding practices slipping.
- Developers don't analyze their own tasks anymore, resorting to shotgun debugging and brute forcing their solutions.
Neither list is intended to be a compelete list.
In either case, the end result is lowered efficiency. Both Agile and Waterfall have a (different) sweet spot for efficient development, and over/underapplication leads to missing the mark and thus being inefficient.
Paul Graham's approach
Now I want to bring Paul Graham's quotes back into the spotlight:
You should figure out programs as you're writing them, just as writers and painters and architects do.
He's not wrong here, if you follow the agile methodology.
If you've ever watched a Bob Ross video, he's the artistic equivalent of agile. He decides where the trees go after he paints the mountains. He doesn't know what the picture will look like before he starts painting, other than a very vague "winter scene" or "seascape". Everything else gets filled in as he goes.
I tended to just spew out code that was hopelessly broken, and gradually beat it into shape. [..] The way I worked, it seemed like programming consisted of debugging.
This is one bridge too far, in my opinion. Unless Paul is talking about his very early days of a newbie programmer (who generally always output broken code on the first pass) or simply expressed himself too strongly here, this is starting to sound like shotgun debugging, which means he took his "act before you think" approach too far and made it more inefficient than it could have been.
Just to be clear here: I'm not advocating a zero tolerance for shotgun debugging. When all else fails, shotgun debugging will always be there as a last resort. But shotgun debugging is inefficient and slow, and you're often better off taking a step back and looking at what you want.
Sketching (the literal artistic definition) isn't the final product but it does imply that you are thinking about what you'll be doing. But sketching is not "gradually beating it into shape", as Paul describes his programming style.
The artistic equivalent of "beating it into shape" would be repeatedly drawing something badly, erasing (part of) it, and trying (that part) again. Paul seems to imply that sketching is "expected failure", which it really isn't.
Sketching is still a thoughtful process of reasonable approximation, but it avoids labeling itself as final and instead keeps itself open to alteration if needed.
Shotgun debugging is valuable for learners, as it teaches them the common mistakes that they should learn to avoid in the future, but that is precisely the point I'm trying to make.
A newbie artist doesn't sketch. They paint the whole picture, fail, and then paint over it. It is only when they start to gather enough experience to know how to (not) paint a picture that they start sketching specifically to avoid that try/retry process.
Sketching is what you do to avoid shotgun debugging. Shotgun debugging is not a form of sketching, it's what happens when you don't sketch.
Which brings me to my final point:
Debugging, I was taught, was a kind of final pass where you caught typos and oversights.
I said that sketching is a reasonable approximation which keeps itself open to alteration if needed. The kind of alterations you need to make to a sketch generally amount to the equivalent of "typos and oversights". If you need to redo your sketch from the ground up, then your sketch must have been really bad or misguided. That's just not good sketching.
While learners should shotgun debug to learn the source of their mistakes, any experienced developer, by their very nature of being "experienced", shouldn't be continually revisiting the basics during their debugging phase.
When you're no longer a newbie programmer, debugging is in fact "a final pass where you catch typos and oversights".