64

I am programmer with 1 year experience, recently I realized I seldom start a project correctly (most of my side project), normally the project cycle goes like

  1. Start with a few use-cases
  2. Start coding
  3. Realize a few things I did not handle well, and does not fit well in current codebase.
  4. Rewrite most part of code

and this might go a few times

So my questions are

  1. Is such practice common, or it implies I am not competent?
  2. How can I improve myself on this aspect?
  • 29
    Perfect! That's called learning :) And competent != efficient on day 1 – user44761 Jul 6 '16 at 7:33
  • 6
    Your interesting question is off-topic, since it looks like career advice. BTW I would also suggest to contribute to some existing free software project, you'll learn a lot (from a community of developers, some of them being more expert than you are today) – Basile Starynkevitch Jul 6 '16 at 7:45
  • 6
    If you find a method to start every project perfectly, let us know. Or write a book about it and become millionaire. – Mast Jul 6 '16 at 14:28
  • 1
    @Qingwei, you said start with use cases, not defining them. Defining them would be a kind of analysis, ie. of user requirements. I think it's better to have a more thorough, detailed understanding of most of the use cases early on. I mean finding just one new use case at a later stage, can often mean substantial re-work. Better to do the re-work on the design than on the implementation. – Brad Thomas Jul 6 '16 at 15:27
  • 1
    I guess it depends on who you talk to but IMO Use-Cases are purely requirements. They should be written completely devoid of any design decisions. Analysis mainly includes designing/defining the system architecture. Thus, Use-Cases don't belong as part of the Analysis Phase. With that said, what usually happens is you write some use-case details, update your architecture diagrams and iterate back to the Use-Cases to make changes you identified while doing the architecture diagrams and update the diagrams based on changes to the use-cases. Then you keep iterating until you are happy with both. – Dunk Jul 6 '16 at 15:42
70

The cycle you describe is normal. The way to improve things is not to avoid this cycle, but to streamline it. The first step is to accept that:

  1. It's near impossible to know everything on day one of a project.
  2. Even if you do somehow know everything, by the time you've finished the project then something (the client's requirements, the market they're in, the tech you're working with, their customers' wishes) will have changed and made at least part of what you knew invalid or incorrect.

Therefore, it's impossible to plan everything up front, and even if you could, following that plan would lead you to build something imperfect or obsolete. Knowing this, we integrate change into our planning. Let's look at your steps:

  1. Start with a few use-cases
  2. Start coding
  3. Realize a few things I did not handle well, and does not fit well in current codebase.
  4. Rewrite most part of code

That's actually a great starting point. Here's how I'd approach it:

1. Start with a few use-cases

Good. By saying "use cases", you're focusing on what the software is for. By saying "a few", you're not trying to discover everything; you're sticking to a manageable amount of work. All I'd add here is to prioritise them. With your client or end user, work out the answer to this question:

What is the smallest, simplest piece of software I could give you that would improve your situation?

This is your minimum viable product - anything smaller than this isn't helpful to your user, but anything bigger risks planning too much too soon. Get enough information to build this, then move on. Be mindful that you won't know everything at this point.

2. Start coding.

Great. You get working as soon as possible. Until you've written code, your clients have received zero benefit. The more time you spend planning, the longer the client has spent waiting with no payback.

Here, I'd add a reminder to write good code. Remember and follow the SOLID Principles, write decent unit tests around anything fragile or complex, make notes on anything you're likely to forget or that might cause problems later. You want to be structuring your code so that change won't cause problems. To do this, every time you make a decision to build something this way instead of that way, you structure your code so that as little code as possible is affected by that decision. In general, a good way to do this is to separate your code:

  • use simple, discrete components (depending on your language and situation, this component might be a function, a class, an assembly, a module, a service, etc. You might also have a large component that is built out of smaller ones, like a class with lots of functions, or an assembly with lots of classes.)
  • each component does one job, or jobs relating to one thing
  • changes to the way one component does its internal workings should not cause other components to have to change
  • components should be given things they use or depend on, rather than fetching or creating them
  • components should give information to other components and ask them to do work, rather than fetching information and doing the work themselves
  • components should not access, use, or depend upon the inner workings of other components - only use their publicly-accessible functions

By doing this, you're isolating the effects of a change so that in most cases, you can fix a problem in one place, and the rest of your code doesn't notice.

3. Encounter issues or shortcomings in the design.

This will happen. It is unavoidable. Accept this. When you hit one of these problems, decide what sort of problem it is.

Some problems are issues in your code or design that make it hard to do what the software should do. For these problems, you need to go back and alter your design to fix the problem.

Some problems are caused by not having enough information, or by having something that you didn't think of before. For these problems, you need to go back to your user or client, and ask them how they'd like to address the issue. When you have the answer, you then go and update your design to handle it.

In both cases, you should be paying attention to what parts of your code had to change, and as you write more code, you should be thinking about which parts may have to change in the future. This makes it easier to work out what parts might be too interlinked, and what parts might need to be more isolated.

4. Rewrite part of the code

Once you've identified how you need to change the code, you can go and make the change. If you've structured your code well, then this will usually involve changing only one component, but in some cases it might involve adding some components as well. If you find that you're having to change a lot of things in a lot of places, then think about why that is. Could you add a component that keeps all of this code inside itself, and then have all these places just use that component? If you can, do so, and next time you have to change this feature you'll be able to do it in one place.

5. Test

A common cause of issues in software is not knowing the requirements well enough. This is often not the developers' fault - often, the user isn't sure what they need either. The easiest way to solve this is to reverse the question. Instead of asking "what do you need the software to do?", each time you go through these steps, give the user what you've built so far and ask them "I built this - does it do what you need?". If they say yes, then you've built something that solves their problem, and you can stop working! If they say no, then they'll be able to tell you in more specific terms what's wrong with your software, and you can go improve that specific thing and come back for more feedback.

6. Learn

As you go through this cycle, pay attention to the problems you're finding and the changes you're making. Are there patterns? Can you improve?

Some examples:

  • If you keep finding you've overlooked a certain user's viewpoint, could you get that user to be more involved in the design phase?
  • If you keep having to change things to be compatible with a technology, could you build something to interface between your code and that technology so you only have to change the interface?
  • If the user keeps changing their mind about words, colours, pictures or other things in the UI, could you build a component that provides to the rest of the application those so that they're all in one place?
  • If you find that a lot of your changes are in the same component, are you sure that component is sticking to just one job? Could you divide it into a few smaller pieces? Can you change this component without having to touch any others?

Be Agile

What you're moving towards here is a style of working known as Agile. Agile isn't a methodology, it's a family of methodologies incorporating a whole load of things (Scrum, XP, Kanban, to name a few) but the thing they all have in common is the idea that things change, and as software developers we should plan to adapt to changes rather than avoiding or ignoring them. Some of its core principles - in particular, the ones that are relevant to your situation - are the following:

  • Don't plan further ahead than you can predict with confidence
  • Make allowances for things to change as you go
  • Rather than building something big in one go, build something small and then incrementally improve it
  • Keep the end user involved in the process, and get prompt, regular feedback
  • Examine your own work and progress, and learn from your mistakes
| improve this answer | |
  • 5
    "Great. You get working as soon as possible. Until you've written code, your clients have received zero benefit. The more time you spend planning, the longer the client has spent waiting with no payback." Can't agree with this whatsoever. The less time you spend planning, the longer the client spends waiting for something that actually works properly, with no payback. – Lightness Races in Orbit Jul 6 '16 at 14:08
  • 4
    @RobCrawford: There's a whole continuum between "no planning" and "excessive planning". Foregoing "planning" or "vision" altogether is likely to get you running... in circles. Agile is not about "not planning", it's about avoiding to rely on uncertain elements and being able to change the goals as you go: you still need some kind of over-arching goal, even if blurry/imprecise, otherwise you cannot measure whether what you develop is progress or egress. – Matthieu M. Jul 6 '16 at 14:30
  • 7
    I think all of you who are objecting to "lack of planning" completely glossed over the fact that the first step is to identify the minimum viable product. This necessarily entails some planning. It seems to me the point of this post is more to discourage trying to get a perfect design up front; instead, so some planning and don't spend forever trying to identify everything up front. – jpmc26 Jul 6 '16 at 15:18
  • 3
    Woah, so this blew up. As commenters have noted, I do NOT advocate doing zero planning. What I'm saying - and what Agile says - is to not do too much planning. I explicitly say "Don't plan further ahead than you can predict with confidence". People saying that I advocate "diving right into coding" should note that coding is step 2, where step 1 is... well, planning. The trick is to do enough planning to determine the smallest product that helps your user, and then give them that product. – anaximander Jul 6 '16 at 15:57
  • 3
    In closing, Agile agrees that there is such a thing as too little planning. It merely suggests that there is also such a thing as too much. – anaximander Jul 6 '16 at 16:02
14

This is normal.

You can take one of two approaches:

  1. Welcome Change

If you assume that you will get it wrong, you must build a code base that is open to change. Mostly this involves taking the code at the end of a book on refactoring, and building your code that way from the start (decomposability, good test coverage, ...).

  1. Avoid Change

In this case you must do a BDUF (big design up front). You have to do a lot of paper prototyping, discussing ideas with potential users or yourself by rubber ducking, trying out various things in prototypes or mockups, writing down a complete specification, and only once you feel you have nailed down the solution then finally you can start coding. Doing all that doesn't actually get rid of unexpected change, it just reduces it a bit, the first year or so. So, you still have to build your code to be easy to change.

So, basically, change is a given. Assume it will happen. Build your code accordingly. In practice you can find a middle ground between the design-up-front and just-start-coding approaches that avoids gratuitous change without getting stuck in analysis paralysis. It just takes a bit of experience.

| improve this answer | |
11

Software development has been described as a series of inherently "wicked" problems.

Horst Rittel and Melvin Webber defined a "wicked" problem as one that could be clearly defined only by solving it, or by solving part of it*. This paradox implies, essentially, that you have to "solve" the problem once in order to clearly define it and then solve it again to create a solution that works. This process has been motherhood and apple pie in software development for decades

This perfectly describes the issue you're facing. Fundamentally, what we do is hard. If there is a part that could be described as "routine", over time we isolate and automate it. So all that's left is either the new or the difficult.

There are other ways to overcome issues like this; some people spend a lot of time considering the issues in advance and not writing code until they're comfortable with the design. Others seek guidance from people who've already dealt with issues like this, either via pair programming or just sites like this.

But certainly your approach doesn't suggest incompetence. The only issue might be if, when you're going back over, you're not reflecting on why you chose to do stuff in this way to begin with, and whether you might have been able to see the "better" way without the rewrite.

In a lot of cases, there was, and you can incorporate it into your design process for next time. In some cases, there wasn't (or the cost would have been as high or higher than the cost of your other approach), and you can just let your concern go.

| improve this answer | |
8
  1. Yes, this is common, except maybe for the "rewrite most of the code" part. You'll never get all requirements right from the beginning, so it's important to deal with change well. That's what the concept of "code maintainability" is all about. Of course it also helps to spend some more time on getting the requirements and design right.
  2. First, think of what changes were required in past projects and how you could have foreseen those or prepared for them better. At the beginning of a project, think more about the details of the use cases. Do some abstract design (what are the main components of the code and how do they communicate, what are their APIs) before you start implementing. Most importantly, try to keep the code as simple and easy to change as possible, read up on concepts such as SOLID, DRY, KISS and YAGNI.
| improve this answer | |
6

Is such practice common, or it implies I am not competent?

Those are not mutually exclusive. The pattern could be common, and you still might be incompetent. Your competence can be determined by measuring your performance against your goals. Are you meeting your goals?

Is this pattern common? Unfortunately, yes. Many people dive into projects with no clear idea of what problem they are solving, how they will engineer a correct solution, and what metrics will constitute success.

How can I improve myself on this aspect?

If you decided to go somewhere and just started walking, and then discovered a day in that what you really needed to do was transport an elephant from Cape Town to New York City, all that time you spent walking was wasted. Figure out what you are doing before you start doing it.

Once you start doing it, consider: what does code that does not have to be rewritten ever look like? That code:

  • Does one useful thing.
  • Does it correctly.
  • Does it with sufficient performance.

So: the more code you write where that code does one useful thing well, correctly, with good performance, the less code you will ever have to rewrite.

| improve this answer | |
1

I think its safe to say you're not so far off from a better way of working, and you're not the only one in this boat.

What I think you're missing is that, although you determine what you want, you don't stop to do the same process for how you'll do it.

This step of stopping and thinking about how to tackle the problem overall is called design, its the step that makes you spend some time in developing the structure or architecture of the system so that everything you do from them on contributes to the final solution, like fitting pieces into a jigsaw puzzle once you've worked out the edges.

Many people don't do this step, yet when I started coding it was mandatory. I think the difference is in development tools - whereas I started with a text editor, you now ave all kinds of features that allow you to jump in and type away.

So, take a little time to figure out the broad areas, components and interoperability between them, and define objects that will comprise your solution before starting to code. It doesn't have to be in immense detail, and understand that you won't get it perfectly right at the start, so it'll evolve over time, but what this does is to help stop you wasting effort revisiting things that shouldn't need to be changed much.

| improve this answer | |
1

You already have some great answers but your question brings a few things to mind that I thought I would try to touch on.

  1. As you noted running into changes down the road I'd suggest thinking about how things impacted your project and how you could have minimized the impact with design/coding choices while at the same time generating a mental map of where people often make late changes. With experience you can anticipate and code in some flexibility for things that you know are going to be important -- though there is a disagreement about this concept in the industry as some will counter you are investing effort in an area not specifically requested, per se.

  2. One the testing front, throwing a prototype up for your project stakeholder can be a great way to refine requirements. However, during development you can look for ways to "see" what is happening in your code without causing a lot of clutter or complexity. There are certainly tools available to help but you can also do a lot without them if you wanted to. Either way, coming out of the code to look at what is happening with a critical eye may provide various types of insight.

  3. Look for common activities. You are going to find yourself handling the same issues over and over again. After a while you should discover the shortfalls or trade-offs of various choices and zero in on a methodology that helps you avoid them. Of course, if you are working in a framework some of these issues may be wrapped up in the tools you are using already.

  4. If you are working within a framework spend the time, if you can spare it, to consider how to do things from scratch. For example, you can easily assemble a request message, open a socket and issue a GET or POST request manually. If you like you can manually parse XML messages. Whatever you do, the questions you generate and the answers you find will grow your skills. Of course, you get to choose which types of underlying issues are important or of interest. I'd consider this personal development and not expect to spend much on-the-clock time here.

  5. Silver bullets, methodologies and high buzz factor issues are everywhere. Basically, the underlying realities of working with information are not changing that quickly. Take note of whether the framework, toolset, or methodology in place is a help or a hindrance in various situations. In large companies I've seen a lot of methodologies attempted though the company was not able to execute them effectively. Just like frameworks, methodologies are not a failsafe way to function at a high level even if they are based on practices of some highly functional teams.

It's hard to boil down experience and have it accessible. I guess a short way to put all this is to keep your eyes open, think about what you see, and never stop learning.

| improve this answer | |
1

I'd like to add a few pointers

1) I personally found it incredible useful to start with visualizing stuff. Draw boxes, arrows, lines ... Never mind whatever modelling language you use. In the first place you are doing it FOR YOURSELF. It should help your flow of thoughts.

2) Find a sparring partner - grab a coffee and the flipchart/diagram etc from above and get to town. IMHO it's even better if you dont have matching tech skills. You bounce back and forth the ideas for implementing the usecase. You find a dead-end or two - you find a solution. With an agile mind there is often less time spent in this stage than if you write code, it doesnt work and your have to kill chunks of your work or redo them

3) Find a boss who can understand that you are likely never done improving your written software. If you always plug in new features / requirements that get dumped on your desk and never care for your software, it will strike back at you with bugs, maintenance unfriendliness and lots of hair pulling a few years down the line. Get this software maintenance time/budget allocated! I good round magic number is about 20% of the time needed to develop the software is needed per year to keep it in shape during the course of its lifetime.

4) This process of incremental learning never stops (and shouldn't)! You will look back in 10 years and smile.

Hope this helps a tiny bit and good luck!

| improve this answer | |
  • Should read "whatever". Edited the post above. – Christian Meißler Jul 7 '16 at 6:42
  • Great recommendations! – Dan Jul 7 '16 at 12:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.