50

I have noticed a pattern while working on several software projects: the big majority of the bugs reported had a high/very-high priority. I asked some colleagues about why this may be happening, and they mentioned that if a bug doesn't haven't that level of priority it is very rare that the Bug gets developer attention, which indeed makes sense.

So, I wanted to know if this problem is common or if I just had bad luck. I did a quick Google search, and I found that some teams implement Bug Reporting guidelines or have a separate "Bug Triage" team. If you have faced and solved this issue, what was the approach that worked for you?

This question is specifically about the "Priority Inflation" problem: If you face the scenario and what meassures result effective against this problem.

closed as primarily opinion-based by Joel Etherton, GlenH7, maple_shaft Nov 18 '15 at 12:35

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 9
    This is why 'priority' is stupid. Is X a Priority 2 is meaningless, is X more important than Y is answerable and useful. The only thing that matters is order. – Nathan Cooper Nov 17 '15 at 13:50
  • 7
    @NathanCooper Yes, but you see, if we have a bug that's like really important, and we need to really give it that little extra push over to dev you know what we do? That's right - we set its priority to 11. – gbjbaanb Nov 17 '15 at 14:03
  • 6
    @CarlosGavidia thus, you need way to take the priority out of the direct hands of the person submitting the bug and find some other way that is objective to determine the ROI for fixing the bug. – user40980 Nov 17 '15 at 15:39
  • 2
    @JuliaHayward personally I like pef/rev though looking specifically at the 'pain' metric for bugs: Improving Bug Triage with User Pain is also very good. Neither of these let the person reporting the bug set the overall bug priority (the 'priority' in the bug pain metric is about what its blocking - not about how important it is to fix). – user40980 Nov 17 '15 at 16:30
  • 16
    True story: I once solved this priority inflation issue by calling my customers out on it, and threatening to rename the different priorities to "orange," "kumquat," and "orangutang," if they didn't do a better job of differentiating serverities enough to keep the field meaningful. It worked (which was unfortunate, because I actually had the admin privileges to create a "kumquat" severity, and I was rather looking forward to it) – Cort Ammon Nov 17 '15 at 22:45

15 Answers 15

42

I asked some colleagues about what this may be happening, and they mentioned that if a bug doesn't haven't that level of priority it is very rare that the Bug gets developer attention, which indeed makes sense

Actually, if you ask me it does not. The more (used) levels of priority, the more information you have. If you effectively only have one priority, that's the same thing as having no priority at all.

And since you still have the same number of bugs to tackle, and the same amount of manhours in which to do it, it follows that some other heuristic will be used, possibly the null one - "first come, first served". And so you now have a bug priority metric, except that's the time of arrival and no longer under your control.

It can be a symptom of not enough resources being allocated to the bug fixing (there are some policies such as "No new features until the bugs are fixed" that can help there. Joel approves; understanding limits and consequences is a business decision).

In one project I worked, the incoming bugs were accumulated in a "no priority buffer" and every Monday we would review the bug list, estimate difficulty (a very rough estimate; more often than not we just put in, "Average") and sort them by available time. This did tend to bump down the list boring, uninteresting or thought-to-be-hard bugs; to offset that, supervisors and marketing had a certain number of credits per week that they could spend to bump the priority of favorite bugs, and were reimbursed for unsolved bugs (this set a limit on how much a developer-despised bug could be delayed).

It was also possible to merge, cancel and split bugs; I remember one module which was so hopelessly flawed that we sank some twenty or thirty bug reports into a single "rewrite this thing from scratch", which was then split in "clearly state the inputs and outputs of the wretched thing", "write tests to ensure inputs and outputs match the spec", and so on. The last item was "print the old code on recycled paper, bring it out on the lawn and set it on fire" (we did that, too. I remember how good it felt. We took turns on the eulogy; it was quite hilarious).

After some haggling, we had the week's to-do list, which was divided in "will do", "might do" and "can't do" that were bumped to next week. This is where some additional haggling came in: we had say fifty hours to allocate to bugs, and we were 95% sure to fix the first twenty. The management strongly wanted a twenty-first bug to be fixed and had no credits left; we would then offer to swap that bug with one on the "Will do" list, or someone would say "Get me off the FooBazFeature subteam for a couple of days and I'll do it", or we would say "We need more manpower".

The system satisfied no one, really, but this was believed (at least among the developers) to be a good sign.

Some additional negative patterns that turned up were the "Wishful Thinking" on the managers' part ("You stated bug 57212 requires eight hours. That is unacceptable. Make it four") and the "Debug by Fiat" ("Do whatever you want but these forty bugs must be fixed before the big demo next week. You cannot have more time, you cannot have more people"). Also the Boxer Syndrome ("I will work harder"), that tended to work very well for a short time, but usually led to a developer freaking out or leaving for greener pastures.

  • Love the "set on fire" part. We have something similar planned for one of our modules :) – Roman Reiner Nov 17 '15 at 22:26
  • 1
    I'm impressed that you had such an organized/negotiated system and still managed to burn out developers. That must have been one intense project! – thanby Nov 18 '15 at 8:03
  • Actually, we had some intense managers, and not always with good human dynamics. So every now and then there were... issues. Some coped, others didn't. – LSerni Nov 18 '15 at 11:52
  • The original question is "(how to avoid) every bug is high priority". Technically speaking, this (accepted!) answer does NOT really answer it. It merely mentions "the incoming bugs were accumulated in a no-priority-buffer and every Monday we would review the bug list, (roughly) estimate difficulty and sort them by available time". But this answer does give many good observation, good food for thought. – RayLuo Jan 3 '16 at 0:57
  • @RayLuo No, it is an answer. Rather than having the reporters rate the priority, the developers rate the priority. – JAB Sep 18 '17 at 14:39
47

If you have this problem where users are assigning ever-higher priority bugs then the only realistic solution is a triage mechanism. All bugs get reported with whatever priority they like, but some poor manager will have to go through every newly reported bug and reset its priority to a sensible level.

After a while your users will either get the message, or you can change the reporting system so that every bug has a default priority. If they want it escalated they will have to contact someone to bump it, which will require some justification. This fact alone will cause 99% of all bugs to be left un-escalated by the user.

Obviously you have more bugs than you can process, so maybe you need to embark on a bug-fix roundup to clear the backlog. This will show the users that their bugs will get fixed without needing them to be marked as super-super-dooper-really-no-honest-this-time-important.

  • 8
    No. Wait. You seem to sugesst ... But that cannot be ... There's actually devs that don't have more Bugs than they can process? Seriously? :-D – Martin Ba Nov 17 '15 at 12:23
  • 49
    @MartinBa YMMV but I have worked places where we took our time to design and develop our product carefully, and while bugs were found, they were reasonably rare. You kids today, writing code without lots of up-front design thought, no wonder you spend all your time unit testing and refactoring :-) – gbjbaanb Nov 17 '15 at 12:29
  • 5
    Actually, if one spends enough time unit testing the bugs will quickly go down again. In a scrum team, the product owner would be the one reasserting the bugs' importance and product owners are intended to have enough business domain knowledge to assess the true importance of the bugs. If bugs never end up on the sprint backlog, the product owner is not doing their job well enough. – Cronax Nov 17 '15 at 12:48
  • 14
    @Cronax if one spends enough time unit testing, you find that you don't have any time left over to write any functionality. So I guess it does stop any bugs from appearing :-) – gbjbaanb Nov 17 '15 at 13:11
  • 4
    @gbjbaanb as long as you stick to actual unit tests and don't go overboard, the usual agile/scrum metric of spending 10-20% of one's development time unit testing seems accurate to me. You can't test everything but that's the same no matter what kind of testing you're doing and isn't the goal of testing. You're simply ensuring that your code does what it's intended to do, the tester will assess whether it's fit for purpose. – Cronax Nov 17 '15 at 13:16
14

DISCLAIMER: I have not yet had experience with user-reported bug priority shenanigans. I know the question asks for this, but it might help having the perspective of an outsider.

Your problem is not that you have too many high-priority bugs. Your problem is that you have too many people who have direct control over bug priority. If every user can directly assign a priority to their bug, they will almost automatically report their issue as high priority.

You could make it so bug priority has to be configured by a manager or a helpdesk drone, but this could lead to favoritism and social engineering, where a client gets artificially higher priority because of their status or because they know how to craft their messages to make them seem more important. It is also a lot more labor-intensive.

There is a middle ground, where your users have some control over the priority, but in a way that makes it harder to exploit the system. Essentially, you force your users to use a template for reporting bugs. They first select a category:

  1. The program becomes unusable or crashes when I do something.
  2. The program has a graphical defect that affects functionality.
  3. The program does not allow me to do something I should be able to do.
    The program allows me to do something I shouldn't be able to do.
  4. The program gives the wrong result when I do something.
  5. The program takes too long to do something.
  6. The program has a graphical defect that does not affect functionality.
  7. The program has a defect that does not fit in one of the above categories.

To give examples:

  1. My iPhone crashes when I receive a message containing Hebrew symbols.
  2. My Android lock screen is rotated in such a way that half of it falls of the screen.
  3. My Android phone sometimes doesn't accept my lockscreen code, even though I entered the right code.
  4. When I try to navigate to PhoneHub.net, my phone redirects me to an adult site.
  5. When I open the Facebook app, it takes a minute to open, even on fast connections and with no other apps running.
  6. Your app has a spelling error.
  7. I found a security defect in your program and would like to report it.

As you can see, each of these errors has a different grade of severity, and the categories are roughly ordered based on this severity. You can then assign each bug a priority based on the category, who reports it and keywords that appear in the description. Bugs in category 7 should get their priority assigned manually.

Note that this can happen fully automatically, and you should let this happen automatically; in fact automation is the key here. Users are prone to overestimating their own importance to the business, so they don't see an issue in reporting their bugs as a higher priority than they should. they're less inclined to deliberately place their bug in a different category, because that requires them to basically lie about the bug.

Users might still enter their bugs in the wrong category of course. The first thing you should do is, from version 1.0, show a friendly message encouraging users to provide accurate information about the bug to help the developers to find and fix it faster. Most users will understand and stop misreporting bugs. Some users might still continue providing wrong information. When that happens, send those users a gentle reminder via mail that accurate information is important and to please not abuse the system. If they continue falsifying records, you give them a warning that they're willfully abusing the system and continued abuse will result in their bugs automatically being assigned a lower category. If they persist, you can adjust their bug multiplier.

You can see parts of this system in place in high-throughput support situations: giant tech companies like Microsoft, Facebook, Google, Gaming companies with a lot of users like Valve and Blizzard, certain governments,... While I'm not sure of the workings behind the scene, you notice that their user-facing support system has a similar interface to what I suggest here, with a strict category system.

  • Very good answer. Users should never even be allowed to set any kind of priority themselves in a bug report. The categories is a very good advise. Any manual priority setting should be done by a product owner or similar. In our development projects, issues that arise during testing is prioritized by product owner in discussion meetings with the scrum master. – awe Nov 18 '15 at 8:56
11

As people have said, this is why the people reporting bugs don't get to assign priority. Your dev team should be in control of their own task assignment (within the scope set by higher management). So, someone further up says "work on this app, fix this feature, make it better at doing this", and the team gets to decide how to go about that, because they're the ones with the technical expertise required to assess how best to achieve what management wants.

The people reporting the bugs should be assigning impact or severity levels, which have defined scope. You can easily call people out for not sticking to agreed levels of severity, because you have material evidence for those levels. For example:

  1. Complete loss of functionality
  2. Partial loss of functionality
  3. Widespread reduction in effectiveness
  4. Localised reduction in effectiveness
  5. Annoyance or hindrance (workaround exists)
  6. Cosmetic
  7. Nobody actually noticed, was found during obscure exploratory test

To start, you can use these levels as a blunt instrument to point out that a misalignment of some title text isn't a Level 1 bug because it's not rendering the entire application unusable. Once they get the idea, you can make it more fine-grained, and start debating whether the glitch in updating this one textbox is a Level 5 because you can fix it by right-clicking in the textbox a few times, or a Level 4 because it's slowing down everyone in Accounting so they get fewer forms processed per hour.

Once you've got useful, measurable information on how bad the bug is for your organisation, assigning priority becomes obvious. Whatever is currently causing the largest problem for the organisation - lost profits, damage to public image, employee unhappiness, whatever - is going to be High Priority, and you work down from there.

  • This. And the short version for PR purposes is that priority is always relative to other things and therefore can only be decided by comparison with other things in the queue. Just because your thing apparently needs doing doesn't mean it's highest priority. – Steve Jessop Nov 17 '15 at 13:40
  • 1
    Well, you shouldn't discount that the highest-impact issue might be much more difficult to tackle than a slightly lower impact issue. I mean, what would you work on if you could either fix two bugs each costing 100€ per day, or one costing 200$, for the same effort? – Deduplicator Nov 18 '15 at 0:00
  • 1
    That's a good point; effort estimates will come into it as well. – anaximander Nov 18 '15 at 7:12
  • @Deduplicator Sorry did not quite get your 100€ and 200$ analog. Were you trying to suggest a slightly-lower-impact but much-easier issue should be handled before the highest-impact but much-more-difficult one? In other words, you were talking the concept of Return On Invest (ROI)? – RayLuo Jan 3 '16 at 0:27
10

It goes a bit like this:

Mgr: what are you working on? Dev: this low priority tasks Mgr : shouldn't you be working on high priority tasks?

Client: when will my bug be fixed? Dev: it's low priority we have high priority tasks. Client: oh, well then set my bug status to high priority.

This will lead to ever escalating levels of priority. I see you've already gone beyond high priority to very high priority. What will be next are:

  1. Super High Priority
  2. Ultra High Priority
  3. Very Super Ultra High Priority.
  4. Very Super Ultra Mega High Priority.

etc. etc.

Yes, this is a normal process. As long as there is no cost involved with assigning priority, and the caller has influence, of course they will try get their issue resolved in the fastest way and set the highest priority.

There are basically 2 ways to counter this:

  1. Take control away from the client regarding priority levels.
  2. Associate a cost to the client for elevated priority levels.
  • 7
    That is not a normal process. Clients have no business interacting with developers on that level, if that is happening, management has completely and utterly failed to do its job. – Davor Ždralo Nov 17 '15 at 11:12
  • @DavorŽdralo maybe process is not the right word used here. I meant it's the normal thing that happens. – Pieter B Nov 17 '15 at 11:16
  • 3
    It's a normal process insofar as the client will always feel that the bug they are experiencing is more important than it probably is. At the same times, devs are notorious for underestimating the importance of bugs. This is why Scrum has a product owner who sits in between the two with knowledge of the business domain combined with the high-level view of where the application is going. They are uniquely suited to correctly assess the priority of any story or bug. – Cronax Nov 17 '15 at 12:58
5

One can add higher and higher priority levels to the system, especially if it is Jira. Giving the new high priorities increasingly silly but dire names can increase the Hawthorne effect bump in the quality of priority choices made by all parties. If the highest priority has a really outlandish name, the effect could be permanent. Eventually when someone chooses a priority, they have to weigh the social consequences of choosing the "bug of death" priority vs. getting it due attention. So, the highest priority is de facto reserved for something that happened to the CTO at home in front of his guests, or other incidents of equivalent visibility.

4

Introduce a cost to support requests. You can only allow a user to report X number of high priority items in a given period of time, Y number of medium priority items, and Z low priority.

Of course, that also means the dev team and management are going to have to then guarantee that a certain number of these will in fact be fixed - all the high priority items, most of the medium priority items, and (maybe) some of the low priority items within a certain timeframe.

But if this is going to work, management are actually going to have to buy into it, otherwise the whole exercise is a waste of time.

At the end of the day though, your particular situation seems to be a symptom of the a problem that your management aren't allocating enough resources to dealing with support issues. If issues were being dealt with in a timely manner, then I don't think this would be happening.

Something like this was implemented in the first company I worked for as the support process was dysfunctional and led to a situation where if everything is an emergency, nothing is. In our case, after getting the in-house situation under control, the new software development manager put a hard limit on the number of high priority issues a customer could lodge depending on how much they paid in support contracts. If they went over the limit, either they coughed up more cash or the support issue was lowered in priority.

  • 1
    I might have misunderstood your gist but if the software is generally of poor quality, why should the client be penalised for raising a series of high priority bugs? – Robbie Dee Nov 17 '15 at 11:50
  • @RobbieDee who says the software is of poor quality? This isn't a query about how to fix bad code practice, this is about how to stop clients from escalating every support issue to high priority. – user1666620 Nov 17 '15 at 12:18
  • So, you're talking about a notional cost or quota? – Robbie Dee Nov 17 '15 at 12:25
  • @RobbieDee - It depends. Something like this was implemented in the first company I worked for as the support process was dysfunctional and lead to a situation where if everything is an emergency, nothing is. In our case, after getting the in-house situation under control, the new manager put a hard limit on the number of high priority issues a customer could lodge depending on how much they paid in support contracts. If they went over the limit, either they coughed up more cash or the support issue was lowered in priority. – user1666620 Nov 17 '15 at 12:33
  • Ah, OK - that makes sense. – Robbie Dee Nov 17 '15 at 12:34
3

This happens extremely often in large corporations where IT is seen as ancillary, and/or is outsourced. Business people don't understand software and don't care to, all that they care is that "their" bug gets fixed yesterday, regardless of how non-critical it is. They don't realise, or care, that there are a hundred other users also filing bugs, and a team of maybe 5 developers available to fix things.

This is exacerbated by poor management, especially non-IT managers who can't say "no" or who simply let business people set bug priority. (In either case, said manager isn't doing his/her job.) Most managers will prioritise the bug that they were last contacted about because "it's urgent"; the net result is that developers end up jumping from bug to bug, thus fixing a single bug takes longer (context switching), and at the end of the day everyone is unhappy. "When every bug is a high-priority bug, no bugs are high priority."

I've been in this situation, and generally the only way to escape it is to get out. Bug reporting guidelines are always ignored because users don't give a s**t. Attempting to introduce bug triage will either be resisted, or it will be implemented and then ignored the next time a user calls your manager to complain about "their" bug.

Basically, if the developers have no control of the priority, you've already lost.

  • Developers having control of the priorities can be equally problematic. They might jump onto quick wins or only pick bugs in areas with which they are familiar. – Robbie Dee Nov 17 '15 at 12:05
  • @RobbieDee I see nothing wrong with going for low hanging fruit first as a policy. – Pieter B Nov 17 '15 at 12:08
  • 1
    A no-bugs policy is an admirable goal, but IMO a completely unrealistic one - bugs are an artifact of software development because people aren't perfect. This is why you need triage - to figure out what needs to be fixed now, what can be fixed if/when there's time, and what maybe doesn't need to be fixed ever. That way you're able to get rid of the most egregious issues while still delivering features. – Ian Kemp Nov 18 '15 at 11:15
  • 1
    @RobbieDee I've been both a feature developer and a bug-fixer, and I find that the issue is that the feature guys and the fixers each end up silo-ing into their own mini-teams because they aren't working on the same code and hence don't need to interact. That creates all sorts of problems around team cohesion and morale, particularly if the feature guys and the fixers never rotate their roles. – Ian Kemp Nov 18 '15 at 11:22
  • 1
    @RobbieDee And it's even worse if roles are rotated - if you do it on a fixed timeframe, people will get stopped in the middle of work and the "new" people will have to ramp up again; if you do it on the basis of when work is complete, there will be unhappiness because invariably someone will feel short-changed ("why do I always end up spending longer on bugs"). In my experience, the only way that works is to ensure all devs mix feature work and bug fixing - for example, feature development for 4 days of the week, and bugs for 1 day. – Ian Kemp Nov 18 '15 at 11:26
3

As as company, you want to solve the issues with the highest importance/cost ratio. Users decides importance, Engineering decides cost. If users assigns the same importance to all bugs, then cost alone matters.

Practically speaking, this means you define priority as importance/cost, and do not allow users or developers to set that priority directly. Neither side has the full picture.

If users decide to rate all issues equally important, that's OK - but it means that engineering (cost) decides what's done. Explain to them that importance is the only way in which they can influence (but not decide) priority.

  • 1
    That works. As long as all issues impact every user about equally. Otherwise, remember that my bugs always have priority. – Deduplicator Nov 18 '15 at 0:03
  • That works, in theory. But that does not really solve the original problem of "almost every reported bug is a high-priority bug", user can still report every bug as "high-importance" instead, and eventually it becomes "easy bug first". – RayLuo Jan 2 '16 at 23:59
3

A few factors...

  • Often the person report the bug does not know the bigger picture and is unable to decide how significant the bug is.
  • Often low priority bugs never get triaged or considered, therefore it is better to give a priority that is too high and let the person doing the triage lower it.
  • As I developer I have often looked at bug report and found that there is a very high priority problem behind the bug, but the product manager doing triage does not care about the bug.

So I think all bug reports need to be looked at QUICKLY by one to two experienced developers, adding their thoughts to the bug report, then the bug needs to be triaged. Expecting the person that finds the bug to be able to set a useful priority at the point in time when they find it, is just asking too much.

3

It is quite possible that all the bugs mentioned are high priority. I've been on projects that were both underfunded and underspecified, and when testing began QA and users just gave up reporting low priority items, because they knew that spelling errors or glitches in usability were a waste of time if the core functionality of the project was completely hosed. If your automated baggage system is crashing carts together and destroying luggage, who cares if the font on the tags is 2pt too small?

In a situation like this, the project is failing. If you suspect that this is even a possibility, you need a heart to heart with the people reporting defects to find out. If people are inflating bug reports, then the other answers will help. If the bugs are as bad as reported, then you need to take extreme action.

2

Most has already been said, but I'd distill the "bug zoo" list down to something a bit less granular:

A: The bug stops the user dead in her tracks, gives wrong output, generally renders the system/feature/function unusable. That's a "high priority" bug.

B: Everything else. These are "negotiable" bugs.

NEGOTIABLE bugs fall under a variety of concerns (which you'd put in your own particular order):

1: Bugs that impact security;

2: Bugs that impact usability (fitness for intended purpose);

3: Bugs that impact aesthetics;

4: Bugs that impact performance (a subset of usability perhaps?);

5: Bugs that offend your sensibilities as a professional programmer;

6: Bugs that diminish USER TRUST;

So that's the utopian world that none of us actually live in. sigh To complete my answer to the OP's stated question, across the entire software industry, it is utterly common for development efforts to be in a status where every single bug is a priority-one-super-banger-special. And you know what they say about "special": when everyone is special, nobody is special.

2

Basically this issue is rooted in the problem of decentralizing prioritization. There should always be an odd number of people with ability to prioritize a team's workload, and 3 is too many. So that 1 person is responsible for effectively -triage. And it should be a manager/analyst, in consultation with a dev lead/architect. And the process is quite simple, and can be applied to feature requests too. What's the cost of doing the development? What's the value of the expected result to the business. Output of that function is the priority assigned.

Of COURSE every user wants their issue fixed first. And as soon as you allow that, chaos. You have no valid prioritization. You need this done by a SINGLE person with authority (consulting with others where necessary), who has VISIBILITY across ALL the issues and business needs, and competent enough to gather the required business and IT expert advice and thereby generate reasonably accurate estimates of the metrics above.

1

At the risk of stating the obvious I'm going to assume there is no bug tracking software that is setting bugs to high priority as default (or has been set to this setting).

I'm afraid, that in the absence of controls, this is the default scenario for multiple teams, clients etc reporting in. If the status quo is being abused then some sort of triage process is definitely in order.

A quick win which I've seen work very well in the past is that P1 (top priority bugs) spawn a plethora of management alerts. If the system has been abused, it is immediately bumped down. Or if it genuinely urgent, a conference call or physical meeting happens to address the issue as soon as possible.

I'm assuming here that you're talking about all bugs not just those from the initial development. If you're typically a green field development gun for hire, then it certainly isn't unusual that most bugs are high priority following the initial alpha release.

  • In JIRA, the default priority for issues is Major ("Major loss of function") – Carlos Gavidia-Calderon Nov 17 '15 at 13:57
  • 1
    @CarlosGavidia Then the fault would appear to be with the set-up. Bug systems are typically set up for everything to go some sort of medium priority. – Robbie Dee Nov 18 '15 at 10:12
0

You can't just have a priority and expect everything to magically work out. Sometimes people figure it out on their own, but not always.

To correctly assign priorities, there must be a definition of exactly what constitutes each priority. These criteria must be objective, to avoid bug favoritism and political prioritization. If the objective criteria are not followed, you need to make the team follow it.

There's really no way around this - if you can't have objective criteria for what bug goes where, and if people willfully refuse to honor these criteria, then you might as well not have submitter-assigned priorities - either do without priorities, or have a third party assign priority as others suggested. Crowdsourced data only works if the submitters are cooperative, and don't actively sabotage the data collection.

If the difficulty arises from not being able to create objective criteria, you can use relative criteria:

  • Have a simple queue of all bugs. Users are able to insert bugs anywhere in the queue. They should only insert at a given position if they feel their bug is more important than everything below it. Bug fixers start from the top of the queue.
  • Assuming you already have a set of well-classified bugs, tell everyone that the condition for putting a bug in priority X is that it must be more important than all bugs in priority X-1.
  • Tell submitters that at no time can the number of bugs with priority X exceed the number of bugs with priority X-1.

But it sounds like your problem is not the definition, but the belief amongst submitters that low-priority bugs do not get fixed. Presumably, you cannot persuade them otherwise, since (from what you say) their belief is grounded in fact. Then, why do you make them submit these bugs? It ends up being nothing but busywork. You could, for instance, after a certain number of active bugs has been reached, tell everyone to not bother making reports unless they feel they found something more important than most outstanding bugs. Granted, this is just the queue solution with an upper limit on queue length.

Not the answer you're looking for? Browse other questions tagged or ask your own question.