I am programmer with 1 year experience, recently I realized I seldom start a project correctly (most of my side project), normally the project cycle goes like
and this might go a few times
So my questions are
The cycle you describe is normal. The way to improve things is not to avoid this cycle, but to streamline it. The first step is to accept that:
Therefore, it's impossible to plan everything up front, and even if you could, following that plan would lead you to build something imperfect or obsolete. Knowing this, we integrate change into our planning. Let's look at your steps:
- Start with a few use-cases
- Start coding
- Realize a few things I did not handle well, and does not fit well in current codebase.
- Rewrite most part of code
That's actually a great starting point. Here's how I'd approach it:
Good. By saying "use cases", you're focusing on what the software is for. By saying "a few", you're not trying to discover everything; you're sticking to a manageable amount of work. All I'd add here is to prioritise them. With your client or end user, work out the answer to this question:
What is the smallest, simplest piece of software I could give you that would improve your situation?
This is your minimum viable product - anything smaller than this isn't helpful to your user, but anything bigger risks planning too much too soon. Get enough information to build this, then move on. Be mindful that you won't know everything at this point.
Great. You get working as soon as possible. Until you've written code, your clients have received zero benefit. The more time you spend planning, the longer the client has spent waiting with no payback.
Here, I'd add a reminder to write good code. Remember and follow the SOLID Principles, write decent unit tests around anything fragile or complex, make notes on anything you're likely to forget or that might cause problems later. You want to be structuring your code so that change won't cause problems. To do this, every time you make a decision to build something this way instead of that way, you structure your code so that as little code as possible is affected by that decision. In general, a good way to do this is to separate your code:
By doing this, you're isolating the effects of a change so that in most cases, you can fix a problem in one place, and the rest of your code doesn't notice.
This will happen. It is unavoidable. Accept this. When you hit one of these problems, decide what sort of problem it is.
Some problems are issues in your code or design that make it hard to do what the software should do. For these problems, you need to go back and alter your design to fix the problem.
Some problems are caused by not having enough information, or by having something that you didn't think of before. For these problems, you need to go back to your user or client, and ask them how they'd like to address the issue. When you have the answer, you then go and update your design to handle it.
In both cases, you should be paying attention to what parts of your code had to change, and as you write more code, you should be thinking about which parts may have to change in the future. This makes it easier to work out what parts might be too interlinked, and what parts might need to be more isolated.
Once you've identified how you need to change the code, you can go and make the change. If you've structured your code well, then this will usually involve changing only one component, but in some cases it might involve adding some components as well. If you find that you're having to change a lot of things in a lot of places, then think about why that is. Could you add a component that keeps all of this code inside itself, and then have all these places just use that component? If you can, do so, and next time you have to change this feature you'll be able to do it in one place.
A common cause of issues in software is not knowing the requirements well enough. This is often not the developers' fault - often, the user isn't sure what they need either. The easiest way to solve this is to reverse the question. Instead of asking "what do you need the software to do?", each time you go through these steps, give the user what you've built so far and ask them "I built this - does it do what you need?". If they say yes, then you've built something that solves their problem, and you can stop working! If they say no, then they'll be able to tell you in more specific terms what's wrong with your software, and you can go improve that specific thing and come back for more feedback.
As you go through this cycle, pay attention to the problems you're finding and the changes you're making. Are there patterns? Can you improve?
Some examples:
What you're moving towards here is a style of working known as Agile. Agile isn't a methodology, it's a family of methodologies incorporating a whole load of things (Scrum, XP, Kanban, to name a few) but the thing they all have in common is the idea that things change, and as software developers we should plan to adapt to changes rather than avoiding or ignoring them. Some of its core principles - in particular, the ones that are relevant to your situation - are the following:
This is normal.
You can take one of two approaches:
If you assume that you will get it wrong, you must build a code base that is open to change. Mostly this involves taking the code at the end of a book on refactoring, and building your code that way from the start (decomposability, good test coverage, ...).
In this case you must do a BDUF (big design up front). You have to do a lot of paper prototyping, discussing ideas with potential users or yourself by rubber ducking, trying out various things in prototypes or mockups, writing down a complete specification, and only once you feel you have nailed down the solution then finally you can start coding. Doing all that doesn't actually get rid of unexpected change, it just reduces it a bit, the first year or so. So, you still have to build your code to be easy to change.
So, basically, change is a given. Assume it will happen. Build your code accordingly. In practice you can find a middle ground between the design-up-front and just-start-coding approaches that avoids gratuitous change without getting stuck in analysis paralysis. It just takes a bit of experience.
Software development has been described as a series of inherently "wicked" problems.
Horst Rittel and Melvin Webber defined a "wicked" problem as one that could be clearly defined only by solving it, or by solving part of it*. This paradox implies, essentially, that you have to "solve" the problem once in order to clearly define it and then solve it again to create a solution that works. This process has been motherhood and apple pie in software development for decades
This perfectly describes the issue you're facing. Fundamentally, what we do is hard. If there is a part that could be described as "routine", over time we isolate and automate it. So all that's left is either the new or the difficult.
There are other ways to overcome issues like this; some people spend a lot of time considering the issues in advance and not writing code until they're comfortable with the design. Others seek guidance from people who've already dealt with issues like this, either via pair programming or just sites like this.
But certainly your approach doesn't suggest incompetence. The only issue might be if, when you're going back over, you're not reflecting on why you chose to do stuff in this way to begin with, and whether you might have been able to see the "better" way without the rewrite.
In a lot of cases, there was, and you can incorporate it into your design process for next time. In some cases, there wasn't (or the cost would have been as high or higher than the cost of your other approach), and you can just let your concern go.
Is such practice common, or it implies I am not competent?
Those are not mutually exclusive. The pattern could be common, and you still might be incompetent. Your competence can be determined by measuring your performance against your goals. Are you meeting your goals?
Is this pattern common? Unfortunately, yes. Many people dive into projects with no clear idea of what problem they are solving, how they will engineer a correct solution, and what metrics will constitute success.
How can I improve myself on this aspect?
If you decided to go somewhere and just started walking, and then discovered a day in that what you really needed to do was transport an elephant from Cape Town to New York City, all that time you spent walking was wasted. Figure out what you are doing before you start doing it.
Once you start doing it, consider: what does code that does not have to be rewritten ever look like? That code:
So: the more code you write where that code does one useful thing well, correctly, with good performance, the less code you will ever have to rewrite.
I think its safe to say you're not so far off from a better way of working, and you're not the only one in this boat.
What I think you're missing is that, although you determine what you want, you don't stop to do the same process for how you'll do it.
This step of stopping and thinking about how to tackle the problem overall is called design, its the step that makes you spend some time in developing the structure or architecture of the system so that everything you do from them on contributes to the final solution, like fitting pieces into a jigsaw puzzle once you've worked out the edges.
Many people don't do this step, yet when I started coding it was mandatory. I think the difference is in development tools - whereas I started with a text editor, you now ave all kinds of features that allow you to jump in and type away.
So, take a little time to figure out the broad areas, components and interoperability between them, and define objects that will comprise your solution before starting to code. It doesn't have to be in immense detail, and understand that you won't get it perfectly right at the start, so it'll evolve over time, but what this does is to help stop you wasting effort revisiting things that shouldn't need to be changed much.
I'd like to add a few pointers
1) I personally found it incredible useful to start with visualizing stuff. Draw boxes, arrows, lines ... Never mind whatever modelling language you use. In the first place you are doing it FOR YOURSELF. It should help your flow of thoughts.
2) Find a sparring partner - grab a coffee and the flipchart/diagram etc from above and get to town. IMHO it's even better if you dont have matching tech skills. You bounce back and forth the ideas for implementing the usecase. You find a dead-end or two - you find a solution. With an agile mind there is often less time spent in this stage than if you write code, it doesnt work and your have to kill chunks of your work or redo them
3) Find a boss who can understand that you are likely never done improving your written software. If you always plug in new features / requirements that get dumped on your desk and never care for your software, it will strike back at you with bugs, maintenance unfriendliness and lots of hair pulling a few years down the line. Get this software maintenance time/budget allocated! I good round magic number is about 20% of the time needed to develop the software is needed per year to keep it in shape during the course of its lifetime.
4) This process of incremental learning never stops (and shouldn't)! You will look back in 10 years and smile.
Hope this helps a tiny bit and good luck!
You already have some great answers but your question brings a few things to mind that I thought I would try to touch on.
As you noted running into changes down the road I'd suggest thinking about how things impacted your project and how you could have minimized the impact with design/coding choices while at the same time generating a mental map of where people often make late changes. With experience you can anticipate and code in some flexibility for things that you know are going to be important -- though there is a disagreement about this concept in the industry as some will counter you are investing effort in an area not specifically requested, per se.
One the testing front, throwing a prototype up for your project stakeholder can be a great way to refine requirements. However, during development you can look for ways to "see" what is happening in your code without causing a lot of clutter or complexity. There are certainly tools available to help but you can also do a lot without them if you wanted to. Either way, coming out of the code to look at what is happening with a critical eye may provide various types of insight.
Look for common activities. You are going to find yourself handling the same issues over and over again. After a while you should discover the shortfalls or trade-offs of various choices and zero in on a methodology that helps you avoid them. Of course, if you are working in a framework some of these issues may be wrapped up in the tools you are using already.
If you are working within a framework spend the time, if you can spare it, to consider how to do things from scratch. For example, you can easily assemble a request message, open a socket and issue a GET or POST request manually. If you like you can manually parse XML messages. Whatever you do, the questions you generate and the answers you find will grow your skills. Of course, you get to choose which types of underlying issues are important or of interest. I'd consider this personal development and not expect to spend much on-the-clock time here.
Silver bullets, methodologies and high buzz factor issues are everywhere. Basically, the underlying realities of working with information are not changing that quickly. Take note of whether the framework, toolset, or methodology in place is a help or a hindrance in various situations. In large companies I've seen a lot of methodologies attempted though the company was not able to execute them effectively. Just like frameworks, methodologies are not a failsafe way to function at a high level even if they are based on practices of some highly functional teams.
It's hard to boil down experience and have it accessible. I guess a short way to put all this is to keep your eyes open, think about what you see, and never stop learning.
