Does Scrum make sense when implementing a new compiler backend?

Question

I have an existing language that I need to port to a new platform. I'll probably attempt this by changing the backend of the existing compiler.

It is a significant amount of work to re-write the backend. I can't see a way breaking this down into sensible stories without violating INVEST criteria.

I can't see how each story can be Negotiable - they are all required for a working compiler. The stories are all of equal priority and it doesn't matter what order I deliver them. I need to do them all.

There are some parts of the software I'm implementing that are lower priority than others and I can see that we can deliver that incrementally. However, there is a significant core that is Must Have.

I plan on trying to follow Scrum, but am I just going through the motions?

Are there recommended practices for this kind of project?

Answer 1

Bear in mind that the main reason Agile processes were created was to cope with shifting requirements. If requirements are set in stone (truly fixed requirements are rare but I'll take you at your word here!) then some of the best practices for dealing with changing requirements - e.g. Negotiable stories - become somewhat irrelevant. That said, following a Scrum workflow still has a lot of potential value in terms of scheduling and providing deliverables. Even if your deliverables won't constitute a fully usable product for a while, there's still something to be said for being able to demonstrate progress to your customer (and the team!).

Consider that all those 'must have' stories comprise a single epic: "Be able to compile on platform X". In this case the whole epic needs to be completed before any value is delivered to the user, but this is often the case at the start of large projects. Start with a story to compile the simplest possible program, then create further stories to support more and more language features.

Above all, don't get too hung up on trying to force every situation to fit into a highly generalised approach. Agile is supposed to work for you, not the other way around!

Answer 2

Of course Scrum is useful. It's a methodology that does two things for you:

1. It allows your project to adapt to change and
2. It allows you to track progress, and get an idea on when it will be finished

So, there's some value in using it.

I think some of your preconditions are not correct and that's where you are getting lost.

I can't see how each story can be Negotiable - they are all required for a working compiler

This is not true. You can support a subset of the language and still have a compiler that works under certain conditions. Surely less valuable than a full compiler, but still valuable.

Also, you misunderstand what "Negotiable" means: it does not necessarily mean "Optional" and there is no requirement that stories are optional in INVEST. A story is a valuable objective and the negotiation is on how to reach that objective. Surely there's going to be more than way of implementing the backend of each language feature. There's where you need negotiation.

The stories are all of equal priority and it doesn't matter what order I deliver them.

This is not correct, as you say below that some stories are not "must have", so certainly some are less valuable. But even in the "must have" category: some language features are much more fundamental than others, and measurably so.

One way to measure this is "how much more lines of code we can compile on an existing codebase" or "how many more tests pass" if you have a predefined suite of tests.

There are also other options. If you were compiling a C-like language, strictly speaking you only need a if and goto loop to have a (barely) functional language and you can implement while, for and repeat as macros. Assuming it's easy enough to write a use a precompiler, you can have a cheap stopgap solution (hey, are we negotiating? :-)

Regarding, adaptability, supporting a language is a fairly static set of requirements, but languages also change and also your knowledge of your needs change. Do you need to implement everything? Are there things you don't need specifically for your aims? One of the basic tenants of agile is the knowledge of having incomplete knowledge, can you leverage it?

In conclusion, to answer your question more directly: do you need agile processes when your requirements are unchangeable? Definitely not! Are they usable? Probably yes! Are they worth your time? Probably not - but are your requirements unchangeable? In my past experiences, "unchangeable requirements" => "lazy product owner" - not a rule, but worth keeping in mind.

Answer 3

TL;DR

All project management controls add overhead. Don't add overhead you don't need.

Scrum is the Wrong Hammer Here (Don't Be a Nail)

Scrum is a project management framework rather than a set of development practices suitable for an individual developer. Unless you are doing project management, Scrum is probably the wrong choice.

In addition, when you say:

The stories are all of equal priority and it doesn't matter what order I deliver them. I need to do them all.

you are implying a couple of things:

1. The project has zero value unless all stories are 100% complete.
2. The stories are not dependent on one another.

If both of these statements are true, then there's really no point in using a framework or practice designed to prioritize work by value or dependency-order.

Suggested Alternatives

Any development project could potentially benefit from some agile practices. In particular, in your specific case I would recommend:

1. Ensure all your stories have a "definition of done" that includes unit- and acceptance-testing.
2. Integrate and refactor often; don't leave yourself with a giant integration task at the end of the project.

Additionally, even if your product truly has zero value unless all stories are done, I'd spend some time grouping the stories into themes that you can treat as completed milestones. Being able to say "I've completed the foo feature" is often more useful than saying "I've got 23/117 random stories done." YMMV.

Answer 4

I understand your concerns, but I believe there is still value for you in using scrum.

Admittedly, the user stories are much more fixed than in a consumer facing application. So that aspect of scrum will provide less value.

Where I think you will get value is from the iterative and frequent releases. Having a potentially releasable product at the end of every sprint compels you to keep code quality high and technical debt low. It is also a great way to find defects early.

I think you would also benefit from knowing your velocity. After several sprints, you can see how many effort points your team is finishing each sprint. This gives you an objective metric to help determine your ship date.

Above all, remember that agile is adjective. At the end of every sprint, you ought to be holding a retrospective meeting and then tailoring your process to your needs. If there is a part of the scrum process that doesn't apply to compiler development, remove it. If there is some other process element that would benefit you, add it. The most important part of agile in my opinion is to be cognizant of your process and be constantly improving for your specific situation.

(Please note, I've never done scrum on a compiler project; take my advice with a grain of salt.)

Answer 5

Yes it does, as long as you remember that Scrum is not a set of strict rules to follow. You can adjust it to your project. Sprints, standups, weekly scrums will still be useful in order to foster better communication between team members and to make sure the project continues in the direction that was intended.

All stories might seem to have an equal priority, but you need to take into account the relative difficulty of implementing them. You don't want to keep your most difficult items until the end of the project. You will want to start working on them as soon as possible.