5

I am convinced that software development is essentially a creative process. I also believe that this is the case for all levels, from architecture to coding.

What makes me think so? To put it very briefly, because a software developer is supposed to create something new, not just copy existing stuff. It is more than just grabbing into your toolbox and getting out the right tool for a job, although it definitely helps to have a good toolbox.

On the other hand, when we view software development from the project management side, it is desirable to split a development project into small tasks and assign to each task a certain time in which it is expected to be completed. (I know that there is the concept of story points, but I don't think it makes such a big difference in practice. At the end of the day, a developer is expected to deliver after a certain amount of time.)

From a project management perspective it is clear that these tasks should be small. Depending on whom you ask, an ideal task should be something between 30 minutes and a day.

Now here's my problem: I find it hard to be creative when facing a time limit for a job, even if it is a soft limit based on estimates or story points. The shorter the time limit, the worse it gets. Often I feel that I am much more productive (because I have the freedom to be creative) when I just do what I think needs to be done without thinking too much about the planned times for each task. Some tasks might take much longer than expected but the quality will be higher and on the whole, the project will probably be finished earlier.

Is this just my personal perception or is it a general problem. If the latter, what can be done about it?

Edit
After reading the first comments and answers, I remembered (once again) that the term creativity does not have the best reputation in engineering. By being creative I do not mean doing uneccessary stuff that provides no business value. What I mean is solving problems in new or non-standard ways.

  • 1
    Many developers make the same complaint. It is hard to generalize the problem. It could be the project management is just not very good; it takes estimates literally when the uncertainty is high. On the other hand, your estimates may not be very good. Or perhaps the management doesn't know much about managing developers and they tend to quit. There can be many reasons for this feeling. – Frank Hileman Aug 17 '17 at 22:48
  • 2
    @Frank Puffer: "Is this just my personal perception or is it a general problem. If the latter, what can be done about it?": I know many people who feel the same (and I am one of them). To understand why it is like that you can refer to Samuel's answer's first paragraph. If you want to really be creative, consider joining an open source project, or working for some public institution. In the business the focus is on producing the minimal feature with enough quality that can be sold, and then forget about it and move on to the next task. In such a scenario, creativity gets lower priority. – Giorgio Aug 18 '17 at 5:19
  • 1
    Task estimates shouldn't be seen as a time limit. They are just a way of being open about what you are working on. In the old days devs would be 'adding the big feature' for months and no-one would be able to say when it would be finished or how far along they were – Ewan Aug 18 '17 at 7:04
  • @Ewan: "In the old days devs would be 'adding the big feature' for months and no-one would be able to say when it would be finished or how far along they were": What old days are you referring to? Been programming since the middle 90-ties and did not experience this more often than I do now: simple tasks were and are completed early and complex tasks are more difficult to estimate. Or by old days you mean the 70-ies? I do not have any direct experience of that time. – Giorgio Aug 18 '17 at 8:22
  • 2
    "I find it hard to be creative when facing a time limit for a job" -- All engineering requires some level of creativity. However, because engineering includes not only form and function, but also economics, it's always at least partially driven by time and cost. In some environments, "doing engineering" may be driven less by time and cost than others. But as an engineer, it's your job to balance all of those things. You need to either (1) change your environment to one that is more suitable for you or (2) learn to adapt. I believe both are beyond scope for this community to address. – Thomas Owens Aug 18 '17 at 12:36
6

You are mistaken what "manage" means. In context of software development, managing a project means that the project progresses with necessary features, on budget, on time and with acceptable quality.

Splitting the project into bite-sized chunks is one of the many ways how it should be possible to achieve above goals. And as you imagine, it might not be optimal way for you. Personally, that approach insults me, as it treats me, a developer, as irresponsible and unable to properly progress on the project, unless micro managed.

If you really want to properly manage a developer, you should treat him as a responsible, skilled person who knows what he is doing and is willing and motivated to do anything necessary for a project to succeed (within reason, of course). Your primary management responsibility, would be to clearly communicate goals of your project and leave the day-to-day management to the developers themselves. You should track progress not based on how many tasks were completed, but on how good the software matches your goals.

And if the developers themselves think splitting the project into small tasks and tracking those tasks is beneficial for their work, they will do just that. But this should be completely unrelated to what you, as a manger, do.

  • 1
    The person who should be doing the "splitting [of] the project into bite-sized chunks" is you! The project manager shouldn't be doing this. I don't think the OP is talking about a situation where the project manager or someone other than the development team is doing this. – Derek Elkins Aug 18 '17 at 7:51
  • @DerekElkins: Yes, of course a developer should do this. But does it make a big difference? I find it hard to be creative when creating a task list or WBS. And I find it hard to be creative when implementing the tasks, no matter who defined them. – Frank Puffer Aug 18 '17 at 8:00
  • @DerekElkins It is not question of who, but why. If manager demands that it is done and uses that to track progress/performance, then that is wrong. If it is done, because developer wants it done to track his own work, then that is fine. Some (or most) developers simply don't need fine-grained time/task tracking to be productive. – Euphoric Aug 18 '17 at 12:04
  • "leave the day-to-day management to the developers themselves": Joel described this management style once as "moving the furniture out of the way, so people can concentrate on their work" – Murphy Oct 18 '17 at 13:11
1

From reading the OP, I see 2 questions here,

  1. Creativity: I don't see a separation between tasks & creativity. The dev team should be setting how to complete the task and fulfill the requirements. With that, the dev lead can define task goals and timelines that give developers the freedom to be creative in their solutions, within the limits of meeting the requirements and meeting corporate coding standards/policy.

Yes, project managers and stakeholders will drive towards deadlines, that's part of their responsibility. But if the dev lead has done their job, then there is room for creativity. And frankly, in the end, there will be times when you just have to get the thing done the quickest way you can. Part of the developer world.

  1. Tasks: while the nature of projects requires tasks to have a time estimate, I've seen serious issues caused when tasks are broken down too finely with an arbitrary limit (like 8 hours). That can create a monster list of tasks that becomes unmanageable and unreportable. Better to create tasks that represent a logical unit of work. For example, "code new customer entry screen". The nature of development is that you will have iterations, you will have sticky points that require more time. Trying to define tasks to the level of "write code for customer entry screen required fields", "unit test required fields" and link them in a sequential fashion will be illogical, due to the code - test - debug - fix code - test..... cycles. Project managers need to think of logical work units, not conforming to some arbitrary model of A Task.
0

Now here's my problem: I find it hard to be creative when facing a time limit for a job, even if it is a soft limit based on estimates or story points.

I actually find the opposite, in a way. In my experience, a time limit can be a creative constraint that forces you to question the usefulness of each initial requirement. When you're short on time, you also have to develop workarounds and find new, cheaper or smarter ways of approaching a problem.

At the end of the day, you might have traded off quality, extensibility or scope for time but can you really say you were being less creative?

In other words, does being creative necessarily mean to create a lot of things?

Edit about technical creativity : it shouldn't be something you totally improvise by yourself at the last moment

In my team, much of the "creativity" in code design happens as the devs are collectively breaking down a user story into tasks and estimating them. This is when we'll have technical discussions about the nitty gritty design details, do our white board brainstorming and schema drawing. The idea is that you don't come up with a task estimate at random. It's something everybody extensively thought out and agreed upon.

Another big part of the creativity happens in spikes, technical research and exploration tasks that also have a time limit. Again, the creative constraint of time plays an important role here. It forces you to focus on coming up with a few pragmatic solutions and alleviates analysis paralysis.

All of this narrows down the amount of creative freedom a single developer needs when doing a task. Of course, they still have the liberty of naming things the way they want to name them, designing their unit tests as they see fit, etc. but the broad picture should have been set collectively before.

  • Time limits as such are not an issue. My question is about small tasks that should be completed in hours rather than days. – Frank Puffer Aug 18 '17 at 21:42
  • I edited my answer expanding on technical creativity. Hope it helps. – guillaume31 Aug 21 '17 at 8:29
  • Re:"traded off quality, extensibility or scope for time". IOW, deliver crap that meets the schedule. I can think of nothing that would kill my motivation and creativity more than having to deliver crap on a regular basis. So I can't agree even a little bit that time can be creative constraint. A necessary business constraint, yes. Creative constraint, no. – Dunk Aug 21 '17 at 21:56
  • "IOW, deliver crap that meets the schedule" - no: IOW, find the cleverest spot in the spectrum between delivering crap and goldplating. Quality is not a boolean, it's a continuum. – guillaume31 Aug 21 '17 at 23:34
0

If you're concerned about time constraints, I typically explain the overall situation to project managers in the following kind of way.

I always work as fast as I can, but not faster. Too fast just leads to additional mistakes/bugs that take longer to fix than simply working more carefully in the first place. So working at my maximum >>effective<< speed is what will get the job done in the shortest possible calendar time. And if that's not fast enough for your schedule, then either the schedule's unrealistic or you've got the wrong man (me) for the job.

Please note that (as you can imagine) I always word that last sentence lots more "gently". I'm just telegraphing the idea to you here. Nevertheless, you've somehow got to get that idea across.

Edit (re your edit emphasizing "business value")
Sometimes tight schedules are dictated by "physical/social reality" rather than by the capriciousness of project managers. And in those kinds of cases "creativity" can mean finding some way to make a long job shorter. Here are two examples from my own experience (see my resume, homepage url on profile, for details, including code and documentation samples).

Back (way back) in 1986, I had a contract with CBS as lead programmer to help develop their "Electronic Display System" for the midterm elections that year. So the deadline was Nov 4, period!

Also way back, in 1989, I had a contract with Chase bank to develop a fixed price trading system for government (and later corporate) bonds. While there was a bit more wiggle room in this schedule than CBS's, the system also incorporated various SEC regulatory changes, and Chase would have potentially incurred some significant penalties had the schedule not been met.

Several more recent projects had similar schedule rigidity (but I don't discuss clients after y2k online). In any event, all these kinds of situations require substantial creativity directed towards getting the job done correctly and on schedule. There are all kinds of "creativity", and your toolkit should include these kinds, too.

2

Estimates are not deadlines.

An estimate specifies how long such a task takes on average, but particulars of that task may cause it to be easier or harder that expected.

If you treat estimates as deadlines, either the deadlines will often be missed, or estimates will be padded to something like the 90th percentile. This makes estimates useless for estimating total remaining effort (you may recall from statistics that the average of the sum is the sum of the averages, but the 90th percentile of the sum is lower than the sum of 90th percentiles. In fact, for sufficiently many tasks the strong law of large numbers implies that the 90% percentile of the sum gets close to the sum of averages, so if you want to add up estimates, you need averages, not guarantees).

Project management being concerned with managing the project, having a good estimate for project effort is more important than good task estimates, so a good project manager should allow for considerable variation for task completion times. Over the course of weeks or months though, the averages should come close to the estimate.

1

From my personal view boring projects kill creativity, not software management in general.

An interesting problem (project) typically requires you to perform at your best and utilize what you so far have learned. If you find that you're coming up simple barebone solutions which are boring it's very likely because the project is of that nature.

Imagine that I ask you to create the same web page four times over with only very slight design differences and say that you can't reuse the codebase because of legal reasons. Your creativity is much more likely to be stifled by the repetition of the task rather than any time schedule set.

I have also done maintenance work on projects which could be considered in cryostasis, which is not a very attractive prospect either. It fits the approach of dust off the code, fix it and run away fast and you should do so, because you won't ever get to nurture these products to health.

Your view obviously stems from your own perception of the reality you've worked in but as I've portrayed there are cases in which I find that it holds true; typically when management does not appreciate the value of spending more than the minimum necessary time on any solution or when the solution is pre-determined. The latter of those two points deserves a small paragraph.

I am reminded of some passages in The Mythical Man-Month which discusses the nature of creativity in software. It implied heavily that creativity is possible at low implementation level so long as the implementation is free for the implementer to design. Essentially, if I tell you to write a function in a specific way there is not much creativity involved as opposed to if you would be to write it from scratch on your own though with a predefined input and output.

Time to research new techniques, patterns and technologies is something I value highly. Quite often I can be found researching something unrelated to my task during my working hours. What's important here is that it has proven to provide a lot of value in the long run. We're talking security fixes due to discovered vulnerabilities, design patterns, optimizations to no longer performant code and so on. It's important for there to be a balance though, you obviously can't drain all your time like this.

So yes, I think there are things that stifle creativity. What can we do about them? Frankly most of the things are dependent on the project and/or management, you can highlight your opinions and hope for change or look to take your services elsewhere.

Most importantly though: Accept the non-creative solutions when that's what is best and spend your energy on problems which require it.

  • I am not sure if there really is such a thing as a boring software project in the sense of boring requirements. Even if you have no interest at all in the problem domain, it can be challenging to find a good (simple, maintainable, performant) implementation. This is also true for maintenance work in legacy projects. I woud say that if a project gets boring, this is probably a result of inappropriate project management. – Frank Puffer Aug 19 '17 at 10:08
2

I would advise you to think of your software development in the context of the business. The business' goal is to make a profit. To make a profit the business needs to produce as fast as it can while maintaining a level of quality.

When we're deep in the designs and the code, it's sometimes easy to forget that often the simplest, straightforward, and fastest method is best for the business, but that's not always the case. Sometimes it takes a lot of consideration and creativity to complete a feature.

My suggestion is you take this all into account when estimating. If this is an important feature that spending more time on will dramatically improve the quality or improve the agility of the rest of the project, then estimate higher. If you're consistently struggling to produce a decent result in the time you've estimated, then estimate higher. Propose quality tradeoffs to the project manager and get their opinion. If early into working on the task, you think more time could produce a better result, consult your project manager to get his opinion about whether a better result is worth the estimate slip. In this way you share the responsibility of the schedule and the quality with the rest of your team.

In a perfect world we wouldn't need to put estimates on anything and we would have unlimited time to create beautiful and elegant software, but mostly that's not the case, and we must accept that business needs must be placed foremost.

  • 2
    Surely I think of this in the context of business. My point is that the profit might be higher if project management would be less rigid. – Frank Puffer Aug 17 '17 at 21:45
  • 2
    It's hard for a software developer to make the claim that profit might be higher. Understanding what will increase profit takes research and understanding of the market. There is opportunity cost to consider as well. The project manager will be more familiar with the goals of the business and the project tradeoffs to make that decision. By all means you should pitch it to the PM. My point is you shouldn't make the decision in isolation but with the support of the development team and the PM. – Samuel Aug 17 '17 at 21:51
  • An other business goal for contractors is to reduce costs (time spend) to get a shortterm advantage over the competitors. So "good enough" is "best". Avoid "unneccessary" work like "beeing creative" or invest money to keep the software maintainable – k3b Aug 18 '17 at 7:24

protected by gnat Aug 20 '17 at 18:15

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.