How to fix a project with basically no structure?

Question

I've been working on a software project mostly solo for over 5 years. It was a mess to begin with (I am the third or fourth developer to be working on it), and although it's less of a mess now it is still incredibly disorganized. The rate of progress in getting it under control is glacial and I'm starting to feel despondent over the state that it's in. How do I really start fixing it?

Project specifics: It is a sales program written almost entirely in Visual Basic Classic (VB6) with a MySQL back end and a reporting engine written in C#. The C# reporting module is a joy to work on, it was only just written in the past couple years and before that all reports were done in Crystal Reports 9 (yes, we still have some reports that rely on it).

The actual program itself, however, is a complete disaster. There are not quite 90k LOC total, and about 10k lines of comments (mostly not documentation, but old code that's been commented out). 158 form files, and 80 module files. I have no idea how many of those are actually used, because some features of the program are simply deprecated and (uh, sometimes) noted as such without having the associated code removed from the program. I would guess that only 50% of the code is in actual productive use.

I am afraid of touching a lot of the code just because I am not sure if I'm breaking something that one obscure client relies on, it's happened on more occasions than I can count. It's like there are landmines strewn throughout the code.

There is not really any structure to the project. It is not object oriented except in the few places I have had the patience to reform so far. If you need to get data on a form, you instantiate a database object, declare your query right there in the function, execute it and do what you will with the dataset.

When I began working on the project there was no source control in use. I tried to encourage the other people I was working on to use it, but I was the new guy and my attempts to get people to use subversion all but failed. The company's lead developer finally caught a mercurial bug in the last couple years and he's made sure that all developers do use source control on all projects now, so at least that's some progress.

I think if I was able to work on reforming the project full time I would be able to make decent progress and maybe even have an estimate for how long it would take me to fully makeover the project, but it is in active use and I am constantly being asked to put out fires, fix bugs, add features, etc. etc.

So how do I start to really fix this project? Try to tool VB6 with another language? Try and rewrite the program in my spare time? Or is this completely hopeless?

Update

After this post I went back to the project with renewed zeal, but fell back into hopelessness within a few months after seeing such a slow rate of progress. I then repeated this cycle 2 or 3 more times over the next year or so.

I have since moved on to a different job. Although after so many years of vb6, and only peripheral experience with other technologies the search was difficult and I faced many rejections along the way (around a dozen interviews over the course of a year). My advice to others in this situation is to consider leaving for this factor alone. Consider the damage you can do to your career by staying in a dead end position such as this.

Answer 1

Now that it's in source control, you can get rid of the commented out code.

I started here in a similar situation (80KLOC VB6 app with no source control, no real structure, almost everything done in event handlers).

In about 2 years on and off I've gotten more than half converted to C# (usually when significant new features are required). All new C# code has unit test coverage. It definitely takes way more time to convert to C# though. If you're not adding significant new modules, I wouldn't go down that route.

One thing I did was create a rudimentary data access layer that auto-generated itself from the database. That at least caught problems where a table column name changed and I didn't find all the places in the code. Also, I've slowly moved the business logic into modules, out of the form event handlers.

I, however, had the advantage that the application was only internal. Since I only had one site to deploy to, I could take some bigger risks than you could. If I made a mistake, it wasn't usually a big deal to fix it. Sounds like you don't have that luxury.

I really think your best bet is to take the following approach:

1. Learn every bit in excruciating detail. This makes it less likely you'll break something inadvertently.
2. Refactor mercilessly to improve separation and structure, but don't try to shoehorn a new methodology like object-oriented programming in there, since VB6 just sucks at it.
3. Treat it as a learning experience. How would you know a different way is better if you'd never seen the inferior way?
4. Don't bother rewriting in a new language/framework/platform unless you have a major module to write. Even then, consider carefully.

Remember, the goal of the program is to be a product that makes your company money. It doesn't have to be a flawless work of art (and I'm a perfectionist so it's really hard for me to admit that). Sometimes it's better to embrace pragmatism.

There are supposedly a lot of COBOL programmers out there maintaining massive amounts of legacy code. I doubt they're all madly working away at rewriting it in some new language. :)

Answer 2

What you need to do is refactoring. Refactoring is almost impossible in such a situation without unit test that give you the confidence that you haven't broken anything. Therefore, start with creating unit tests. Document the behaviour of the code - but not (only) on paper, but in more code - the unit tests. Once you have the tests in place, you can start restructuring the code.

Answer 3

One rule from David Agan's "Debugging" comes to mind: quit thinking and look. You happen to have at your disposal a machine capable of processing vast quantities of text. Use it to figure out precisely what code is no longer in use, and remove it without mercy. If you make a mistake, that's what source control is for.

That's the easy part. What you need to think about next is how do you eat an elephant? One bite at a time. Pick up Martin Fowler's "Refactoring" and take it function by function, file by file. Don't completely scrap something and rewrite it from what you think the requirements are. Work like a mountain climber, never removing your previous safety until the next one is in place.

Had enough metaphors yet? The point is if you take it slow and steady, with many improvements as simple as renaming or splitting a function, you can improve the bad parts of the code without destroying the good parts of it that were built up through years of debugging and feature requests. You want the code to remain shippable at all times. If it's possible to do that while porting to a more modern language, go for it.

Answer 4

I hate to say it but I'd go with "completely hopeless" beyond just continuing to do an endless cycle of patch/enhance and hope nothing breaks. I've seen far too many VB6 projects like the one you describe and attempts to refactor/rewrite/redo them into C#/.NET fail terribly, often with the people in charge of the attempt being fired.

The thing is that an complete rewrite from the ground up is going to be necessary sooner or later. VB6 and Windows versions that support it well are on the decline. Finding developers who know VB6's quirks and who're willing to work on programs like this are becoming rarer. The internal limits of VB6 will be hit on huge programs like this, causing weird errors.

If there's a good consensus and commitment for a total, ground up, redesign and rewrite at this point, begin work on it and delegate ongoing maintenance to someone else (contractors, junior programmers, whatever works for you). If people aren't convinced enough to begin this yet, just wait around for the pain to become great enough or else move on to greener pastures.

Answer 5

Some good answers here already, I would like to add one thing. Instead of trying to rewrite everything, perhaps you will have a chance to port at least some of that modules mostly 1:1 to VB.NET (of course, you have to be very careful either when doing this)? To my experience, such porting can be done with ~5-10 times less effort than trying to rebuild all existing functionality from scratch. And after you have ported it, then you can start to refactor - with all those tools available in the .NET world, like good unit testing tools, automatic refactoring tools, real OOP etc.

I had been in a similar situation, where we had to migrate an old 16 bit C++ program (150k LOC) into the 32 bit world, where the original GUI framework in use was not available any more. It took us some time to understand that rewriting was unrealistic, but after we decided to port that thing to the .NET world, using C++, C++/CLI and C#, we needed only about 9 months to make that happen (with around 1 dev full time working on that project). And we had no unit tests for the GUI part available, did all that testing of those parts manually.

Answer 6

Although it puts a LOT of emphasis on unit-testing your application which, although a very important thing to do, is something quite hard to do in VB6, Steven McConnell wrote an excellent book about how to tackle on such a projet.

Take a look at it:

Steven C. MCCONNELL: Working Effectively with Legacy Code, Second Edition. Microsoft Press, june 2004.

Basically, his point is to take it slowly, one bit at a time. He also recommends to begin by using refactoring techniques that are very unlikely to break anything, sometimes making it a bit worse in the short term, and to start using more advanced techniques as the code is getting cleaner and has unit tests to support it.