Advice regarding tools and insights from sprints on developer performance

Question

I would like to get some help or ideas regarding the usage of JIRA and getting insights from it. I just joined a small company as a junior project manager. In the last several sprints, Developer "X" burned significantly more points than any other team member.

1. Which parameters I can use use to identify the reason for this exceeded performance, based on JIRA’s capabilities.
2. Which methods (quantitative or qualitative) i can apply outside of JIRA (or another tech tool) to identify the reason for this exceeded performance.

Answer 1

Do not use Jira to pick winners and losers. Coders created Jira and coders can make Jira say whatever they want.

Use Jira to help the team improve its estimating and planning skills. Not for identifying your top coders. It’s not a human resources tool.

Rather, watch the team dynamics. Leaders will emerge on their own. You’ll see who others count on.

This might seem like strange criteria for coders but modern coding is a team sport. Solo heroes aren’t the ones you should fight hard to hold on to. Keep the ones that keep the team working.

Answer 2

Developers are professional problem solvers.

Pick any metric you like and the developers will identify which ones you are using and game them, making the metric effectively useless.

Or they will have pride in their real work and not game the metrics, leading you to punish them/fire them/demoralise them for doing the actual work instead of solving the arbitrary pseudo problem you have created.

Leave story points as the crude estimation technique for forward planning that it is. They are by definition incorrect and only help with capacity planning because the imperfections in each guess average out as more are made.

Answer 3

It's likely impossible to get any insight simply from jira data. Don't get caught up in the numbers.

There are essentially two possibilities

1. The dev in question is simply better than their colleagues.

   This is more likely than it might sound. estimates in the industry put the best devs at several times better performance than the worst.

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968). They studied professional programmers with an average of 7 years’ experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1

2. The tasks were overestimated

   This is virtually guaranteed to happen with every other task, after all they are only estimates. Given hypothetical developers of equal productivity one of them is going to luck out and get (cherry pick) the easy tasks every sprint

As a project manager you shouldn't worry about the productivity of individuals, the team will know amongst themselves what the score is and will tell you over time. Its the performance of the team as a whole that matters.

However, you should try to avoid over estimation and gaming of the system.

1. Ensure that tasks are estimated by the team rather than an individual
2. Use some relative measure of effort rather than hours
3. Make sure tasks are picked up in order of priority, don't let team members cherry pick
4. Ensure tasks are completed by testing them against requirements before marking them complete

Answer 4

You should not be using Story Points to measure individual performance. Consider that Story Points are estimated by the whole team, often without any consideration of who is getting the work. The Story Point estimate may also be for the unit of work in isolation, and the effort to complete a set of work is not necessarily the sum of the Story Points in that set. Since Jira only allows one assignee, the metrics don't account for multiple people contributing to the completion of the work.

I'm sure there are others as well, but this illustrates my point that Story Points are not suitable for measuring the performance of an individual. They are used for sizing and capacity planning.

If you really want to understand, perhaps you should hold a retrospective with the whole team. This may give you some insight into the overall performance of the team as a whole. However, I would also suggest focusing on achieving goals rather than completing Story Points.

Answer 5

Completing a lot of story points isn't always a sign of exceeded performance. It could mean:

• They are specialists in an area of code that has naturally easier tickets.
• They are not worrying much about quality and are leaving other developers to clean up after them.
• They are more pedantic about recording all their work in Jira.
• They avoid more challenging assignments that don't have straightforward solutions.
• They write code that is difficult for anyone but themselves to maintain.
• They avoid spending time helping other developers.
• They are burning themselves out.
• They don't do much off-ticket work like fixing a flaky test, for example, that improves the developer experience for everyone.

Don't get me wrong, there are good productive developers too, but I've worked with too many of the type that boost their own numbers by leaching productivity from others. The only way to know which is which is to talk to the team, probably individually in private to start. I would ask a relatively straightforward but non-leading question like, "I noticed X completes a lot of story points. Why do you think that is?" Then see if you get complaints or compliments.

If they are truly a high performer, I wouldn't count on being able to duplicate their performance, because it is usually due to individual factors that are difficult to replicate. However, you might identify some systemic issues you can address. If you get answers like, "X is the only one who understands SQL so they often work 12 hour days to keep up with the tickets," you can get some training in place and force the SQL work to be spread around more.

Answer 6

At least two possible causes come to mind:

• The developer might just be a genius, a top performer.
• The correlation between task points and effort isn't consistent, and he's picking tasks that yield most points and require little effort (low hanging fruit).

The simplest and probably most exact tool might be to talk with the team about this observation to gain some insights. Technical tools such as JIRA may be helpful in quantifying a model, but first you need a theory which JIRA won't produce for you.

If the overall team productivity is good and there isn't friction between members which would jeopardize introspection and process improvement, it may be best to just ignore this observation, as there isn't much to be gained in terms of productivity improvement and employee motivation.