How to reconcile CD with major architectural changes?

Question

(This question is inspired by the SO blog entry Fulfilling the promise of CI/CD and this old question of mine: Why is it wrong to comment out code and then gradually remove it to keep track of what I've already done and what remains to be done?)

The usually recommended software development methodology - if I understand it correctly - is to:

1. Decide on a tiny change in code you intend to make.
2. Write an automatic test that covers this change. (Skip if you only mean to refactor existing code with no intention to affect its semantics)
3. Run tests and see this test failing.
4. Make the intended tiny change in code.
5. Run tests and see them passing.
6. Immediately deploy to production.

Emphasis is put (among others) on tiny changes. It is heavily unrecommended to make large changes that take days, weeks or even months to write and only then deploy. Such changes need to be split into multiple smaller incremental changes that will be written and deployed separetely.

Then, if large architectural changes need to happen (necessiting rewriting a lot of code), then how to make them without falling to the antipattern of big, slow, batched changes deployed infrequently?

Let me provide an example. A few years ago I was trying to make a hobby game. However, unfortunately, after a few months of work I found out I pretty much coded myself into corner. I managed to produce such a spaghetti code that I could no longer work on my own project. Even small fixes requiered inordinate amounts of time and were introducing numerous other bugs.

To remedy this I tried to pinpoint the exact mistakes I made and thought how to fix the style and architecture of the whole game. In the UI code I decided that the issue was that I was programming sloppily, with little or no archicecture nor consistent structure. But in the code responsible for game mechanics I concluded that my mistake was to introduce numerous incorrect abstractions that I had initially believed to be elegant, but that were, in fact, working against me.

As such I decided that the majority of my code had to be, basically, rewritten.

Necessary changes were causing a domino effect. I had implemented about 70 abilities, but their code was incoprehensible. To fix them I first needed to fix the code behind the game engine. But changes to that code were invalidating the existing code behind abilities. Maybe I should have, instead, put the game engine behind an abstraction layer, so that it would expose to abilities both the new, hopefully a bit more sane API as well as the old, IMO fundamentally flawed API? Perhaps; but I thought that this would make the whole code even more messy and incoherent.

My solution from that time was to outright comment out most of the existing code and start rewrting it, gradually replacing the commented out code with new, hopefully more workable code. As soon as no more code would be commented out I would 'deploy'¹ the app. Sadly, I never finished this process: time issues kicked in while I was in the middle of my work and since then I never managed to find enough time to seriously return to the project.

This is only an example; I'm sure more reasons can be provided why large portions of code may need to be rewritten.

If such major changes need to happen, then how to 'deploy frequently', as is often advocated? Especially if changes seem to be contagious, in the sense that each change necessites other changes?

---

¹ _{'Deploy' is in quotes because I've never developed my game to such a point that I'd think it'd be worthy to be published. So I never actually deployed it in the sense of making it available for wide use. But I don't think it matters with respect to the SW development methodology discussed here.}

Answer 1

I think the question is addressed in numerous "what do i do with a pile of spaghetti?" questions on this site.

To summerise; major changes don't have to be done all in one go. Your first task is not making the change, but making incremental change possible.

You do this by gradually refactoring to classes/interfaces/components without changing the actual methods.

Once you have compartmentalised your code into components, you can then start changing the "engine" code to call the interfaces and only then start changing the actual code in the methods behind the interfaces.

All this can be done in small incremental steps, you don't have to have chunks commented out code waiting to be switched in or massive multi month old branches