Project is nearly done, but procedural spaghetti code. Do I rewrite or just keep trying to ship it? [closed]

Question

I'm a beginner web developer (one year of experience).

A couple of weeks after graduating, I got offered a job to build a web application for a company whose owner is not much of a tech guy. He recruited me to avoid theft of his idea, the high cost of development charged by a service company, and to have someone young he can trust onboard to maintain the project for the long run (I came to these conclusions by myself long after being hired).

Cocky as I was back then, with a diploma in computer science, I accepted the offer thinking I can build anything.

I was calling the shots. After some research I settled on PHP, and started with plain PHP, no objects, just ugly procedural code. Two months later, everything was getting messy, and it was hard to make any progress. The web application is huge. So I decided to check out an MVC framework that would make my life easier. That's where I stumbled upon the cool kid in the PHP community: Laravel. I loved it, it was easy to learn, and I started coding right away. My code looked cleaner, more organized. It looked very good.

But again the web application was huge. The company was pressuring me to deliver the first version, which they wanted to deploy, obviously, and start seeking customers.

Because Laravel was fun to work with, it made me remember why I chose this industry in the first place - something I forgot while stuck in the shitty education system.

So I started working on small projects at night, reading about methodologies and best practice. I revisited OOP, moved on to object-oriented design and analysis, and read Uncle Bob's book Clean Code.

This helped me realize that I really knew nothing. I did not know how to build software THE RIGHT WAY. But at this point it was too late, and now I'm almost done. My code is not clean at all, just spaghetti code, a real pain to fix a bug, all the logic is in the controllers, and there is little object oriented design.

I'm having this persistent thought that I have to rewrite the whole project. However, I can't do it... They keep asking when is it going to be all done.

I can not imagine this code deployed on a server. Plus I still know nothing about code efficiency and the web application's performance.

On one hand, the company is waiting for the product and can not wait anymore. On the other hand I can't see myself going any further with the actual code. I could finish up, wrap it up and deploy, but god only knows what might happen when people start using it.

Do I rewrite, or just keep trying to ship, or is there another option that I've missed?

Answer 1

You have stumbled on the achilles heel of most CS educations: they teach you the tools and techniques, but not the trade. Building software is a craft, one which you only acquire through years of practice and the experience of having your software used (users are much harsher critics than teachers). Building software is also quite often a business, one where the business goals may override the technical ambitions.

First of all, ship. If you show the business owner the software, and they feel it's ready to ship, then ship. If it's not to that point, but close, finish it. The only software that matters is that which is actually used. The only software business which earns money is one which has a product.

Secondly, you have learned a lot of valuable things, so you should appreciate the experience for what it has taught you:

1. Slinging code without a plan or architecture is a recipe for disaster
2. There is much more to programming than writing code
3. Non-technical business owners often do not understand the impact of technical decisions (like who to hire), and it is up to the developers to explain things to them.
4. Most problems are already solved much better than you would solve them, in existing frameworks. It pays to know the frameworks that exist and when to use them.
5. People fresh out of school assigned to a big project with little guidance tend to produce a bowl of spaghetti code. This is normal.

Here is some more advice for you on how to proceed:

1. Communicate, communicate, communicate. You must be very open and frank about the state of the project and your ideas on how to proceed, even if you're unsure and see multiple paths. This leaves the business owner the choice on what to do. If you keep knowledge to yourself, you deprive them of choices.
2. Resist the temptation of the full rewrite. While you are rewriting, the business has no product. Also, a rewrite rarely turns out as good as you imagined it. Instead choose an architecture and migrate the codebase to it gradually. Even a horrible codebase can be salvaged this way. Read books about refactoring to help you along.
3. Learn about automated testing / unit testing. You have to build up confidence in the code, and the way to do that is to cover it with automated tests. This goes hand-in-hand with refactoring. As long as you don't have the tests, test manually and comprehensively (try to break your code, because your users will do so). Log all the bugs you find so you can prioritize and fix them (you won't have time to fix all bugs, no software ships bug-free in the real world).
4. Learn about how to deploy a web application and keep it running. The book Web Operations: Keeping the Data On Time is a good start.

Answer 2

This sounds like every other system that has been thrown at me to fix.

Relax, this happens to a lot of people. A junior thrown in at the deep end with no experience, who has no help, no support and no guidance isn't exactly a recipe for success. Hiring and expecting a junior programmer to build a brand new system from scratch that works well, performs well and is maintainable is not realistic whatsoever. Hell you're lucky if all that happens with a senior programmer.

In my opinion you have to come clean. This will not be fun. Tell them that you've done your best, it works (mostly), but you're worried that it might not perform well and that there will be a lot of bugs (there's always bugs). It needs to be reviewed by a senior programmer, and they should be able to fix any glaring performance/security problems pretty quickly. Or they can deploy it and cross their fingers. It'll either go ok, or go up in smoke. Maybe you can fix problems as they come up. If you have a large user base maybe not.

Or you could do what most people do in this situation: take the money, disappear and let them sort it out. I'll leave it up to you to work out what the ethical choice is.

Edit (as this question has a lot of votes I might as well add some more content)

Part of the joys of being a programmer is that non-technical people (probably your manager, definitely the rest of the business) have no idea what you do. This is both good and bad. Part of the bad is that you have to constantly explain how software development projects work. Planning, requirements, code reviews, testing, deploying and bug fixing. It is your job to explain the importance of testing, and to set aside time to test. You have to stand your ground here. People won't understand the importance ("can't we just start using it?") but once they start testing (not in the live environment!) they will quickly understand the benefits. One of the reasons why they hired you is because they don't know anything about software development, so it's up to you to educate them. You need to emphasize the importance of testing and bug fixing here - remember, they are not programmers, they don't know the difference between a divide by zero and a broken html tag.

Often a lot of the problems that crop up aren't actually bugs. They will be usability issues, missed requirements, requirements that have changed, user expectations (why can't I use this on my mobile?) and then the actual actual real bugs. You need to iron these out before you go live - often a lot of bugs can be worked around or fixed a few days later. If people expect a perfect system they're going to be in for a lot of pain. If they are expecting bugs your life will be a big easier over the next couple of weeks.

Oh and don't confuse user testing with unit testing nor with system testing.

• Unit Testing - does my code function return the right value
• System Testing - does it throw an error when I click on X
• User Acceptance Testing (UAT) - does the program conform to the requirements? Does it do what they asked you to make it do? Can it go live?

If you haven't written down the requirements of what they asked you to do, UAT will be much, much more difficult. This is where a lot of people fall down. Having what they wanted the system to do written down on paper will make your life a lot easier. They will say "Why doesn't it do X?" and you can say "You told me to make it do Y". If program is wrong, fix it. If the requirements are wrong, fix the doc, ask for an extra day or two (no, insist on it), make the change, update the documentation and re-test.

Once you've gone through this process a few times you can start looking into Agile.. but that's another world :)

TL;DR Testing is good

Answer 3

Whenever you start from scratch, you'll almost certainly make the same amount of mistakes or more due to the Second System Syndromme. Your new mistakes will be different, but the amount of time needed for debugging will be similar and so will despair about how it's not a good fit. It will also delay deployment into production or deployment of new features if the first version is deployed, which will be serious trouble for the company. Joel Spolsky calls it "single worst strategic mistake" that any company or developer can make.

The recommended approach is instead to clean the initial mess bit by bit during maintenance. And don't even try to refactor it just for the sake of it. Besides, managers usually see that as waste of money (which it often is) and it brings unnecessary risk of introducing new bugs. Once you painfully debugged the code, it may not be pretty, but it will work. So let it be until you need to touch it for other reasons (be it a bug fix, new feature or just a change requested by marketing). Then clean up the parts that are most difficult to adjust. This is often called the Boy Scout Rule.

And at that point you don't have to argue it with the manager. Just include the minimal desired refactoring in the estimate for the request. You'll learn through experience when you may afford to yield a bit because the company is really in a fix and when you don't want to create problems in future and just don't admit any possibility of quickly hacking it.

Last, one more bit of recommended reading: the Big Ball of Mud.

Answer 4

I forget where I first read it, but I just wanted to echo, somewhat more forcefully, what other people have said:

Shipping is a feature.

There is nothing worse than that one guy who keeps "cleaning up" existing (possibly hacky, ugly, dirty) code that works perfectly well, introducing new bugs, etc. What matters in the real world is getting your job done. You've done that. Ship. Don't get lost in redesigns of a project that works perfectly well, even if it's ugly under the hood. When you fix it, do so incrementally, and make yourself a good test suite so that you have as few regressions as possible.

Answer 5

Every project leaves you smarter than you were before. After every project you will have accumulated more experience which would have been very handy when you had it from the start. I know that it is hard to not revisit everything and apply what you have learned. But remember:

Perfect is the enemy of good.

For the client it is always better to have a good software now than a perfect software which will never get released.

This was just your first project. There will be many more projects in the future where you can apply everything you have learned from the start.

Answer 6

I [...] read Uncle's Bob clean code.

I'm having this persistent thought that I have to rewrite the whole project.

This book has a section named, very appropriately, "The Grand Redesign in the Sky".

Don't try to rewrite everything because, in the unlikely event that you have the time to make it, you will face the same problems anyway. When you have finished the redesign you will have learnt new things and will realize that the first parts of it are very unprofessional, so you will want to rewrite it again.

Redesign and rewriting are good, but only if they are done incrementally on a working system. As another user pointed out, follow the Boy Scout Rule, refactoring your code little by little as you work on it.

Answer 7

You're doing good.

You say that your code works, and it's almost ready to ship, right? And you perceive that your code may be vastly improved. Good.

Your dilemma much reminds me of my first experience with freelancing (getting hired while in my 2nd year at uni to create a multilingual POS system). I went through endless questioning as I was never satisfied with the code, wanted scalability, wanted to reinvent better wheels... But I just delayed the project (like, by 12 months approximately) and... what? Once you deploy the thing it still needs a lot of proofing, testing, patching etc...

Do you have experience working with professional code-bases? Many code-bases are full of quirky, hard to maintain code. An alternative to discovering the complexity of software by trying to build a large program yourself would be maintaining/extending equally messy code written by other people.

The only cases where I've seen complete rewrites do much good is when the team concurrently adopted a new toolchain/framework.

If the underlying logic (what the program does, not how it is laid out as functions, classes and so forth...) is sound, it will work just as good, so, you thinking that it's spaghetti code doesn't mean it shouldn't be deployed.

You need to let your customer have something that they can use. Then when they ask you to improve it / add functionality you decide whether a refactor is necessary, and it's okay to let your customer know that "some technical work is needed to integrate said new feature". By which they will understand that it will cost them more money, and they will have to wait longer. And they will understand (or you can pull out).

A better time to rewrite everything would be when another customer ask you to create something similar.

In short, unless you can demonstrate that the whole thing will blow in everybody's face if it gets deployed, delaying deployment would be unprofessional, and will not benefit either you or your customer. Learning to do small refactors while fixing bugs or adding new features, that will be valuable experience for you.

Answer 8

Most of what I would say in response to your question has been said by others. Read "Things You Should Never Do, Part I" by Joel Spolsky (along with some of his other posts about "architecture astronauts"). Remember that "the perfect is the enemy of the good". Learn to refactor incrementally. Shipping is important, etc.

What I would add is this: you've been tasked with something that was considered doable by a single fresh graduate working with a small startup budget / timeframe. You shouldn't need anything much more sophisticated than good, structured, procedural programming. (And, FYI, "procedural programming" is not a bad word. It's a necessity at some level in most cases, and it's wholly adequate for many entire projects.)

Just make sure you actually do structured, procedural programming. Repetition in your code is not necessarily a sign that you need grand, polymorphic structures. It could simply be a sign that you need to take the repeated code and put it into a subroutine. "Spaghetti" control flow may simply be an opportunity to get rid of a global.

If there are aspects of the project that legitimately call for polymorphism, implementation inheritance, etc., I would suggest that maybe the size of the project was underestimated.

Answer 9

If you're really interested in the dilemma you have, you should also read "Lean Startup". A lot of the advice that you're being given here will resonate with you more if you read that book. Basically, resource burn-rate is your worst enemy and nothing is more valuable to you and your organization than end-user/customer feedback. So, get your product to a viable state (the Minimum Viable Product - or MVP) and then ship it out the door (regardless of what the code actually looks like). Let your customers dictate your future changesets and versions, not the other way around. If you focus on that approach both you and your boss will be happier in the long-run.

Answer 10

For reasons that others have thoroughly explained, it is time to finish the project and ship it, painful as that may be.

I would just like to emphasize that testing the app is also part of "finishing" it. If significant pieces of functionality haven't been thoroughly exercised and correct results confirmed, then you're justified in your concern that people will have trouble with this app when it is deployed.

Unit tests and automated higher-level tests are great and are things you should have as much as you can before you try to refactor (or rewrite) this application. But right now you mainly need to test it, even if you have to run every test "by hand" and confirm the correct functioning "by eye". If you can figure out how to automate those tests later, that will help when it's time to start work on the next version of the product.

Some people have a knack for sitting down in front of a new software project as an alpha-test user and making things go wrong. That is, they're good at breaking things. If you are lucky enough to have such a talented person working with you, let them try the app first so you have a chance to fix any obvious bugs early. If you have to do this task yourself, then:

• Be methodical.
• Try every feature.
• Pretend you're a user inexperienced with your application. Make stupid mistakes and see how the software handles them.
• Write down what you're doing so you can try it again after you think you've fixed the problems.

Answer 11

Your question says: "Started wrong, should I start over" while the additional text actually says "Finished project, but did it wrong, should I start over". For the question headline alone: If the amount of programming work that you have done is small compared to the total work needed, then starting all over will make sense. That happens a lot when the problem is complex and badly understood, and quite some time is spent figuring out what the problem actually is. No point continuing with a bad start, if throwing it away and starting all over, this time with good understanding of the problem, means you will actually finish quicker.

Answer 12

This is what I would do personally:

1. Quit, make an excuse and give up (you can even give back part of your salary so you don't look bad)
2. Clean your code as much as possible
3. Make documentation on all the good parts of your work like your good design, good ideas, etc...

Why do I suggest all of these to you?

Because think about the future. There will be time in the future when certain people will get a hold of this code. They will make all sorts of assumption and judgments about you and your ability. They don't care when you wrote it, they don't care the circumstance. They only want to see in it what they want to see to confirm whatever they want to confirm.

You will be branded as whatever bad name, term they can come up with to negatively impact you. And even though the you in the future might very well be totally different in term of technical ability, skills, knowledge, style and the situation will be so different, this code will be used against you in every possible way. It is like a court where they can say all the bad things about you and your code and design and you are not even aware of it so you can explain and defend yourselves. (and you might learn very often that they are deeply wrong, repeatedly) So don't do it. Give up.

Trust me, there are people who made lot of assumptions about you because you did something for whatever purpose at whatever moment. To them, if you did like this in situation A, you will do it in situation B. They think very simple.

Answer 13

Two more suggestions, I'll bet at least one of these you haven't done.

1) Put a bug tracker in place and teach your boss to use it. This can be part of the conversation about how you screwed up, have learned better and are going to fix it in a planned manner

2) Start using version control, although hopefully you are doing so already.

There are heaps of hosted systems out there which provide both the above for free on small accounts. I'm particularly fond of FogBugz which also has a great estimating and task completion system which will give your boss even more certainty that you are tackling things in a well-managed manner.

edit

Wow, someone really didn't like this - a downvote AND a delete flag? Why?

I've been developing software for over thirty years including a lot of consulting work and inheriting other people's code. A huge number of the problem systems I've seen have been where people dug a pit and didn't have detailed notes on how they got there nor did they have version control.

Answer 14

To answer your question: as so many others have said, no. Ship it, and clean it up bit by bit in the process of fixing bugs.

Additionally, while books/StackExchange/webforums are good learning resources, you are likely to find that nothing can match the learning you will receive from working (or even just discussing work) with other programmers. Finding and attending a local group for a technology you are using or would like to learn is a wonderful investment of your time. Besides the technical knowledge to be acquired, it's an easy way to network which is invaluable as you look forward to future gigs.

Answer 15

Your boss was aware of your experience level when he hired you. Just express your concerns, and let him know you're nervous about it. Also let him know how much you've learned and how much better you can do the next project.

Answer 16

You're a beginning web developer, with no good developer present to advice you, your boss hired you knowing this fully well. I think you are doing just as well as anyone could expect you to do. Actually, the fact that you have the insight that the job could have been better, and that you actually learned things that would allow you to do it better, means you are doing better than most. Remember for your own sanity that the version of you who started the job couldn't have done it better. Someone more experienced (and therefore better paid), or you with one project worth of experience, could have done it better.

What to do now: Be happy that you are a better developer than a year ago. The next project you will do better (and at the end you will be more experienced again and not happy with what you've done; that's normal). Changing or rewriting the last project will give the business very little benefit for the cost. What you can do is replacing bad code with good code when you need to make changes anyway; that makes sense because modifying badly maintainable code can be harder than replacing it with good code. And if you get a new inexperienced developer to help, tell them that this code is not an example that they should copy.