I find that some software developers are very adept at this, and often times are praised for their ability to deliver a working concept with abstract requirements. Frankly, this drives me crazy, and I don't like "making it up" as I go. I used to think this was problematic, but I've started to sense a shift, and I'm wondering if I need to adjust my thought (and programming) process when given very little direction. Should I begin to acquire this ability as a skill, or stick to the idea that requirement's gathering and business rules are the first priority?
The skill isn't to write software without requirements. It is instead to elicit requirements from the project owner regardless of whether there is a formal requirements documentation or not.
Gathering requirements is definitely your first priority, but you don't necessarily need to get all of the customer's needs noted up front. The risk is of course is that you might miss some vital piece of information that renders your system architecture useless if you haven't managed to have the right sort of conversations with your customer, however it is not unusual to define a product and even get much of the development out of the way, while deferring the major system architecture decisions until the last possible moment. This is a lean development approach which is meant to ensure that you don't commit to a potentially incompatible architecture too early in your product development until you have more solid information. In the situations the OP has described in his question, this lean approach would be quite important IMHO to avoid major rework and cost blow-out later on, which is when you've finally managed to learn what it was your customer really needed.
Yes, you do sometimes need to crystal-ball-gaze a little to get to the heart of what it is the customer really is asking for, which is where prototyping spikes and the slow - and yes sometimes painful - incremental drawing out of requirements requires that you really develop good customer relationship skills, and also the patience to realize that with any complex software idea, that in the beginning the customer doesn't often know much more than you about what the software actually needs to do. Most often, the customer calls you in early to depend on your expertise to define their requirements as the customer doesn't always have the necessary expertise or knowledge of the software development process.
This is very ambiguous …
Two things I can say:
There are a lot of very gifted technical people whose careers get halted because they wait for perfect requirements. Or they play the, "Sorry, can't do it, wasn't in the requirements." The reality is requirements writing is very difficulty. The precision required for good requirements is unlike anything most business people have ever created. There's a bridge between technology and the business, and people who make the others come 100% of the way to meet them usually lose.
There are software people who learn the domains as good or better than their customers. These people are worth their weight in gold, even if they aren't 100% the best developers. I have seen software people anticipate the quantitative marketing needs of the best brand managers in the country. They weren't the best at coding all the solutions, but they were heroes because they could cross the bridge.
Life's not about black and whites though. If you draw a narrow box around yourself, you'll limit yourself. On the flip side, an organization that dismisses what's needed to create technology is also limited. You'll have to see where in the gray you prefer to be.
Requirements are the steps in the journey, a vision is the direction
For many applications a highly detailed technical specification is just too much up front since a quick discussion could render their carefully typeset document useless. Instead, start with a vision. If everyone understands the overall picture then the requirements can get filled in along the way through discussions.
As a developer you must learn to use these discussions to trawl for requirements. This means asking the customer leading questions that get them thinking about how their decision today fits into the overall vision. The earlier these more detailed discussions take place then the quicker the overall vision will solidify into a coherent design.
You should keep track of the outcome of these discussions in some kind of issue tracker so that others can comment on them if they missed the original discussion. And so that you have a record you, or other members of your team, can refer back to should you need clarification.
So, learn to code against the vision, but be ready to trawl for those requirements when the time comes.
Steve Jobs believed that customers cannot describe exactly what they want the future products to look like, so it is your job to deliver them. So, unless you deliver custom software all the time, forget formal specs and start by creating prototypes and letting the customers play with them and tell you what they think. You have got to put the right person doing the prototyping, and they need to have help. I say this from experience - I am the prototyping monkey who loves creating intuitive interfaces and I teamed up with someone in product who understands what the clients want and can explain things on paper or using Excel.
Neither of us are geniuses, but we think alike - you can almost say we have got chemistry and have had a huge impact on which things are being built and how. Now, only a mid-to-large team can afford to have a prototyper and a non-coder who develops product exclusively, but it is well worth it. Prototyping is the cheapest stage in the software development, so it only makes sense to get the UI and the apparent behavior right. I have not read Code Complete but I think there is something like that written in that book.
Specs are nice to have, but they are never perfect. There exists a theorem about that. You cannot prove that the spec is complete and you cannot prove that the tool has no bugs or that it will halt :)
Yet, software companies do ship software all the time despite these imperfections in the process. The spec will never be perfect. The spec is also UNNATURAL and outdated. A spec to a prototype is like logarithm table is to a single graph - a spec is essentially a boring brochure meant to be printed whereas you could interact with a tool / graph instead. Check out http://www.i-programmer.info/news/112-theory/3900-a-better-way-to-program.html for inspiration.
Now, spec is good if you must have a contract to cover your ass. But a spec should still come after a prototype, not before. It is your job to figure out how to make prototypes cheap.
I've often found that in some situations I need to act as a business analyst, discovering exactly how the business currently works, how people think it works (often very different things), and how they would like it to work.
I've found success by always being clear about the decisions I'm being forced to make in order to build the software. I explain my reasoning, write documentation on what I've discovered, make graphs and distribute them to everyone, etc.
You probably won't make a very good impression by refusing to do any work until complete requirements are handed over. But by gathering good quality requirements yourself (without necessarily drawing attention to the fact), you will reach the same goal of quality software.
And yes, as other commenters have said, always build the software assuming that it will change. Change is the one constant you can rely upon. Always build your software flexible and modular enough so that it won't be painful to update it when some new requirements suddenly appear.
If you want to work as a software developer at a startup, it's a skill to possess.
If you want to work at a consulting company then it's a situation to avoid at all costs. This is because your firm gets paid according to how well you implement the spec/requirements and not how well you solved the customer's problem.
If you are coding for fun in your spare time, then it's your call. If you don't feel qualified to make the call for your spare time projects then try a couple each way and see what works. Also it's not necessary a one-size-fits-all thing, some projects call for one or the other type of a approach. Usually if you pick the wrong one on one of these projects you'll figure it out pretty fast.
A bit of both. You need to satisfy your clients, which means that you need to know what they want. On the other hand, clients are notoriously bad at communicating what they really want.
So you want to avoid scenarios where you don't know what clients want, but you will inevitably run into a scenario where the requirements are 'squishy' at best, and deceptive at worst. A good real world programmer requires adaptability.
It is not possible to write a program without requirements. Even the 'Hello World' has the requirement: to produce output. So, I think you're asking about formal requirements, in form of some big stack of something UML-like. And regarding those, I've met 2 kinds of people:
1) People that need formal requirements. They need to be exactly told what to do, and at best how to do it. They love the sentences like The procedure A taking the argument B will output C, and they hate those: The program should make the work of our deparment more effective. They are usually corporate animals.
2) People who are the oposite to 1. They hate to be told what to do and how to do, they love to be told what should be achieved. They like to talk to the client, analyse what they say and then develop their own solution. They are usually freelancers and don't fit good to corporation.
I won't say which of those is better. Both have their pros and contras. They are simple adequate for the other conditions.
You can NOT develop operational software without knowing the Requirements; but, you can have a jolly good stab at developing what your experience tells you the Requirements are likely to be. Agile development uses a combination of 'intuitive' techniques, including the 80:20 Rule and the 'discovery' of Requirements by prototyping. In other words, an experienced development team makes a best guess at what's needed and produces a prototype of the software. The 80:20 Rule says they'll be 80% correct. The project stakeholders then review the tangible prototype. Their feedback begins to fill the 20% gap in our understanding of the Requirements. So, in effect, Agile isn't about writing software without any requirements, rather, it's about using your prior experience to say, "are you wanting something like this?" Which, in 80% of cases, will allow you to leap ahead and confirm what's really needed quicker than plodding through traditional Requirements processes.