3

Currently the software development lifecycle followed in the IT company I work at is:

  • The "Business" works with a solution manager to build a Business Requirement document

  • The solution manager works with the Program manager to build a Functional Spec

  • The PM works with the engineering lead to develop a release plan and with the engineering team to develop technical specifications

  • If there are any clarifications required, developers contact the PM who contacts the solution manager who contacts the business and all the way back introducing a latency of nearly 24 hours and massive email chains for any clarifications

  • By the time the tech spec is made, nearly 1 month has passed in back and forth

  • Now, 2 weeks go to development while the test writes test cases

  • Code is dropped formally to test, test starts raising bugs. Even if there is 1 root cause for 10 different issues, and its an easily fixed one, developers are not allowed to give fresh code to test for the next 1 week. After 2-3 such drops to test the code is given to the ops team as a "golden drop"

( 2 months passed from the beginning)

  • Ops team will now deploy the code in a staging environment. If it runs stable for a week, it will be promoted to UAT and after 2 weeks of that it will be promoted to prod. If there are any bugs found here, well, applying for a visa requires less paperwork

This entire process is followed even if a single SSRS report is to be released.

How do other companies process such requirements? I'm wondering why, the business cannot just drop the requirements to developers, developers build and deploy to UAT themselves, expose it to the business who raise functional bugs and after fixing those promote to prod. (even for more complex stuff)

13
  • 2
    What would you consider efficient? If you're talking about a project with ten million lines of code involving multiple organizational structures, you may need all this infrastructure. If it's just 100,000 lines, not so much. – Robert Harvey Jun 10 '14 at 16:38
  • @RobertHarvey < 10k LOC per release here. 99% of the work is onboarding systems to ETL processes, absorbing changes from upstream systems and requests from downstream systems, and creating SSRS reports. Wondering why a process like I've mentioned in the last 2 lines isnt used in the industry – user87166 Jun 10 '14 at 16:40
  • 1
    Looks like a system where, when the product is done, they know beforehand the features they will get, and that it will stable... it is way more efficient than letting the programmers guess what is needed, and asking the stakeholders about it after finishing the modifications. – SJuan76 Jun 10 '14 at 16:47
  • 11
    You're complaining about waterfall, go read about it and realize everybody else recognizes it's terrible inefficient too. That said it's still used because for all it's inefficiency, in a variety of cases given the correct set of constraints, it actually works to the point of getting well-functioning IT products and services to business users. While it's unarguably inefficient, sometimes the inefficiency is worth it for the decreased risk waterfall has in the right scenario. It sounds like it's working well at your place, even if frustratingly overbearing... – Jimmy Hoffa Jun 10 '14 at 17:13
  • 2
    Agreed that this sounds like waterfall development. I'd suggest you contrast against methodologies like Extreme Programming, Agile, and DevOps. Sometimes waterfall is the right answer (e.g. NASA can't iteratively launch a rocket), but usually it is frowned upon in general software development. – Allan Jun 10 '14 at 19:30
7

How do other companies process such requirements?

Differently.

Different businesses have different processes because they have different needs, different risk tolerance, different resources...

I'm wondering why, the business cannot just drop the requirements to developers

Often, because it's difficult/impossible/expensive to find developers that are good at being developers and also good at talking to business people. The opposite is of course a problem too - finding business people who are good at business and can talk to developers.

developers build and deploy to UAT themselves

Often, because letting Ops do it frees the developers to focus on the code. It also let's the Ops people learn how the system works and maintain/support it. That support can make or break a product.

But all of this is (mostly) devil's advocate on my part. Too often, process is implemented by managers who don't trust their employees to do their jobs rather than documented (and continually re-evaluated) by managers so that new employees can more easily learn how to do what everyone else does.

6

TL;DR

Your experience is typical. As noted in other answers the Agile Manifesto, eXtreme Programming (XP), Scrum, Kanban, Lean and many other techniques are popular to avoid the inefficiencies you point out.

Your question is slightly controversial is it asks "why is a SDLC so inefficient". Yet not all SDLCs are this inefficient. You can have a lightweight SDLC when using, say, Kanban and do continuous deployments into production multiple times each day.

A few very good books for developers that discuss the problems you have identified and possible solutions are:

  • "The Phoenix Project" by Gene Kim and Kevin Behr
  • "Agile IT Organization Design" by Sriram Narayan
  • "Ship it!" by Jared Richardson and William Gwaltney.

I would suggest that you find a job with a company that takes a different approach than your current one to broaden your horizons and improve your job satisfaction.

Why Are Developers Treated As "Mushrooms"?

There is an old joke:

I am treated like a mushroom. I am kept in the dark and when the light rarely comes on for a minute I am buried under another ton of manure.

If you work in a large bureaucracy as you describe it is very frustrating and unrewarding to be a developer. Unfortunately, the problems you describe result from company structure, the annual financial budgeting process, and how the bureaucracy functions. In large companies, this creates many hand-offs to pass requirements from the Sales or Product team back to the developers. As you point out this causes long delays in the feedback loop. Efficiency, quality and features all suffer when there are a long feedback loop and multiple hand-offs.

Having a decision-maker such as a Product or Sales team member spend even as little as half an hour a day talking to developers would be vastly more efficient. They could answer ad-hoc questions and feedback on partial work. To maximise the feedback loop the features should be broken down into small units. A very common approach is to work in two-week sprints. This reduces the window where they can be any misunderstanding on how a feature should work to be less than two weeks. A very common approach is to have a retrospective after every two weeks. If there was confusion about a feature in the previous sprint the retrospective can be used to change the way the team works or documents requirements to avoid the same wasted efforts occurring in future sprints. This could be as simple as "next time we attempt something like X again we will all sit in a room for three hours and sketch it on the whiteboard first". Flexibility and communication are far more valuable than documentation and processes when converting concepts into working software.

The forces that push against creating a single empowered team are driven by corporate politics. The people in the 'hand-off' chain between the developers and stakeholders such as Sales/Product would likely feel very uncomfortable to be bypassed. If your job is to "deal with the development" so that "Sales/Product can focus time on customers" why would you step back and let developers just work directly with the Sales or Product people? The very existence of your job, set up by more senior managers, implicitly states that the company believes that developers should not speak directly to Sales or Product. It is therefore clearly the responsibility of the intermediaries to prevent direct communication. This perversely leads to a very inefficient development process which is harmful to the company and that allows more nimble competitors to beat them.

Understanding why the situation above comes about and how to change it is a large topic. It spans concepts like accounting, corporate structures, employee incentives and many other MBA topics. The book "Agile IT Organization Design" by Sriram Narayan does an excellent job of explaining why these systemic problems occur and how to fix them. The book argues that companies fail to work efficiently due to applying manufacturing management techniques to software development.

Fundamentally software development isn't actually like manufacturing at all. Developers should not be treated like blue-collar workers working on a factory line. Software Engineering is inherently entirely a design process from end-to-end. The act of writing code isn't an act of transcribing a local design into machine instructions. It is more like the act of the developer creating the final business solution within their IDE while seeking inspiration from the written requirements. Developers don't actually "implement features" when they write code. Instead, they interpret the requirements and design the concrete solution as they write code. Applying manufacturing management techniques to this process is extremely counter-productive. It stifles the productivity, creativity and ingenuity of developers and ultimately causes them to quit software development.

To be clear I am not advocating "developer anarchy" as the ideal way to develop software. Teams should use a lightweight governance process and SDLC. I am advocating for small, empowered, mixed discipline teams to rapidly iterate on features and fixes with zero intermediation between them and their key stakeholders.

Why Does Shipping Software Require Delays And Handoffs?

There is an industry concept of DevOps as a culture where developers are empowered to do continuous deployment into production using rigorous automation. Famously highly productive companies release software many times a day. There is a direct correlation between how frequently software is deployed and how successful a company is. For example:

The range of software deployments went from 1,460 deploys per year (calculated as four deploys per day x 365 days) for the highest performers to seven deploys per year for low performers.

As mentioned in the previous section the problem is that software delivery is treated like manufacturing. If you are making a car, and it has a safety design flaw, it can bankrupt your company when you need to recall and fix all the bad cars. At the end of the last century enterprise software was shipped on a physical CD. A factory would print thousands of copies of the CDs that would be physically distributed to suppliers. Shipping a service pack was also done on thousands of CDs. Treating software development as you would treat manufacturing seemed like a very good idea.

It is typical in manufacturing to batch up work to make it efficient. Batching features into monthly or quarterly release appears to optimise the requirements analysis phase (let us not distract Sales/Product too much!). Batching up testing work into two or three weekly drops appears to optimise the testing phase. Deploying a large release mostly by hand using a Tech Ops team seems to be easier than investing in building a fully automated deployment pipeline. Running and monitoring monoliths appears to be easier and more efficient than running lots of microservices. Yet these are all local optimisations. These local optimisations make the end-to-end process of delivering new software poor when compared to high performing companies.

The forces that push against empowering a developer team to do rapid deployments many times a day are driven by both software architecture and corporate politics. If your job is to do "system testing" protect the business from bugs or to "do deployments" to keep developers away from the live environment why would you step back and let developers have a continuous deployment pipeline? The very existence of your job, set up by more senior managers, implicitly states that the company believes that developers should not be empowered to deploy code rapidly into any of Test, UAT or Live.

Then there is the question of whether technically you can continuously deploy small features or bug fixes into the live environment. If you have a monolithic software architecture this is hard. You get all the inefficiencies that you point out. Using feature toggles can help. Better yet a deployment system that lets you do canary releases or blue-green deployments.

One of the main drivers to the adoption of a microservices architecture is so that teams can continuously deploy small features or bug fixes. This makes the company far more productive at the expense of the complexity of running a large number of microservices rather than a simple monolith. If you have a microservices architecture then a Service Mesh solution like Istio will let you tag only a small percentage of traffic to use a new version of a microservice deep within the system. This is known as Traffic Shifting. Swap in a new version of your microservice to a small population of customers, monitor for errors, then gradually roll it out. If unexpected problems occur then shift the traffic back to the old version. Else if the problem is only for one or two customers you can deploy a quick fix just for them and shift only their traffic onto that quick-fix version. That then gives you time to go back and fix the root cause to be able to run a single version of the code for all customers. This may sound like magic but it is possible. Companies that can do this are wiping the floor with competitors in terms of both software costs, quality and time-to-market.

Why Release Little-And-Very-Often?

I have direct experience of working at a large global financial services company that mandated that software teams doubled the number of releases, and halved the number of standard defects, and quartered the number of critical defects, every year. Senior management directly tracked that this was happening across a few thousand people working across a few dozen large scale software systems and over a hundred small systems. Some big teams needed to go from quarterly releases to monthly releases, then fortnightly, then weekly, then mid-week. Some small teams were already doing fortnightly releases at the start so had two years to move to mid-week deployments. This wasn't seen as an optional efficiency drive. It was seen as something necessary to allow the company to survive in the face of stiff competition from more nimble competitors.

Systems that couldn't make the transitions due to architectural issues were then flagged as "legacy" and were no longer investing in. The strangular pattern as used to start the build of replacement systems built as microservices. These new systems aim to ship code into production continuously. Why make this investment? Surely the business could just give the legacy monoliths "a pass" to keep on doing quarterly or monthly updates. Remarkably while releases were doubled across many of the systems their defect rates also did half and critical outages did go down to a quarter in the first few years. So not only was it now very obvious that it was hard to introduce new features into the monolithic systems it was obvious that they were very poor performers. They systematically caused more pain to the team maintaining them and gave a poor customer experience. This was in part due to very long lead times in fixing issues leading to a patchwork of expensive manual workarounds.

It may seem counter-intuitive that you can deploy into production every day and get a higher quality outcome than having a long test cycle. Yet it is the natural result of the more automated testing, shorter feedback loops, more focused and motivated developers, improved software architecture, and improved automation required to do continuous deployments. For example, developers were introducing much smaller changes and rolling any problems back much faster using feature toggles. It is often quoted that research says that a bug that gets into production over a hundred times more expensive than not caught earlier in the SDLC. The answer isn't to run an inefficient SDLC that makes your company uncompetitive such that more nimble competitors put you out of business. The answer is to do everything possible to reduce both scope and duration of bugs in production while making a more efficient SDLC that can produce higher-quality software in much smaller units that can be continuously improved upon.

At the same time as achieving these successes, job satisfaction increased across developers. Usually, the organisational restructuring needed to achieve such changes leads to some of the best people leaving a company. If it actually leads to people feeling empowered and more productive it helps with both hiring and retaining talented staff.

2

Short answer: Other companies that don’t use the waterfall approach that you’re describing to build IT solutions often use an Agile approach, e.g., Scrum or eXtreme Programming.

1
  • 1
    "Just get on and write some code"... "To do what?"... "Stop asking awkward questions... just get on with it!" – Andrew Dec 30 '20 at 12:22
0

•The "Business" works with a solution manager to build a Business Requirement document

Your team must be ready, willing and able to work with the "Business" and create requirements. They may want to keep the same document formats and probably interpret those as some type of agreement or contract between them and the developers. If your development team has a preference to some other type of documentation (sticky notes on a board is still documentation, just not traditional), you may be able to convince the business to accept this. Ideally it is something everyone understands and agrees on.

Now all you have to do is decide what all these other people are going to do. Whether you realize it or not; you not only declared war, but fired the first shot.

2
  • Agreed... it is essential everyone is on the same page, with respects to what is required, before you start! – Andrew Dec 30 '20 at 12:24
  • 1
    are you saying it is the OPs fault that he isn't engaging with the business? please re-read what the OP has written as to what the SDLC is at their company. saying "ignore that and behave like an independent software vendor without any constraints" is basically saying "everything works okay on my laptop fix your own problem". – simbo1905 Dec 30 '20 at 21:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.