How can you be certain that a controversial change made to your software really is best for users?

Question

Background: we have recently made a significant change to the UI of our software. All previous versions used a MDI interface (windows floating around inside a parent window.) The new version uses a docked interface (windows snap next to each other, fill all available space, and can be tabbed on top of each other, and can be dragged out to another screen.) We thought - and still think - this is a great improvement. However, feedback from our alpha testers is wildly varying. Some love it, some say they find it hard to use and want to use the old system. Many of our users have been using the software for well over a decade and are very used to its old UI behaviour.

Generic situation: I'm sure other software houses have encountered similar problems, ie I think this is a widely applicable problem we will probably all encounter at some point: you make a change (especially a significant change) that you think will be greatly beneficial for the users of your software, and you don't receive universal acclaim.

• How can you be certain that your change really is better for the user, instead of just a change you happen to like?

• In my experience, change is usually liked more when it's voluntary or sought out, not forced. How do you enthuse your users so that it is voluntary? Should you even try? How do you handle this situation?

I have tagged this question 'delphi' because there is an interesting specific parallel: a decade or so ago the Delphi & C++Builder IDE (which we use!) changed from floating windows to a docked UI - exactly the same as us. A small, very vocal group of people complained loudly and some still do to this day. As far as I know, the majority think it was a great change. If there happen to be any Borland / Embarcadero staff reading who feel like chiming in with a historical tale, I'd love to hear your input!

Answer 1

usability testing

before you make the change

You may have just spent a great deal of effort fixing a problem that did not exist!

Answer 2

Testing

In web development, A/B Testing is by far the best way to answer those questions.

To your specific example, have you tried giving it to new users and, better still, then given them the old product for comparison? It's absolutely true that a change to the UI can cause hellish problems in the short-term for experienced users of the old product.

But if it's better for someone who hasn't used the product before then it will almost certainly become better for the old users too, given time. However, sometimes what seems like a great idea to developers isn't great to the average person.

Answer 3

Change is scary

Keep this in mind when you show long time users that you are changing everything about the way you interact with the application. While I personally think that your proposed change will be friendlier to new users, the long time users depend on the way things are. It took them a long time to become proficient, and they've all come up with little systems to make using the application more useful to them.

When you change everything "for the user's good" those users now have to relearn everything they already knew. It's a real disincentive. Think about your gut reaction when you saw Microsoft Office 2007 (or 2010) for the first time. "Where is everything?"

Change is an interruption, then it's done

People will adapt, however you have the user interface organized. To that end, consider the following questions:

• How long does it take to adjust to the new way of doing things? For example, did you completely rearrange the menu structure while changing the way windows are organized? That can be a catastrophic no-no. Limit yourself to only one major UI change at a time.
• Do new users learn the interface faster with the change or without it? You need to demonstrate some tangible benefit to the users. Use an A/B test with new users to focus your results. If you can reduce training costs with the UI change, you'll start to win over the pointy haired bosses.
• Can existing users still do their job? The more a user has to do something, the quicker they will adapt. The problem is that if the change is too different and the user can't find the critical functions they need to do their job, you've failed them.

Can change be good?

Absolutely, yes. The question you have to answer is why is this change good? Are you saving the user steps? Are you reducing the amount of time twiddling with the screen and not doing work?

The next alpha test you do, apply the following:

• Have the users perform a set of tasks on the old interface. Use screen capture to get a better idea of just how the user has found to make the app useful to them. Also, time them.
• Have the users perform the same set of tasks on the new interface. Time them, and use screen capture to get a better idea of just where they are stumbling.
• Once you've observed what they are doing, train them to bring their knowledge of the UI up to where yours is.
• Now, tell them you are doing a speed drill. Have them do the same set of tasks as before on the new interface, but as quickly as they can. Give them some sort of incentive. The incentive doesn't have to be anything grand. It can be something as simple as cookies for the winner.
• After all this, now ask them about their opinions.

What you are trying to do is educate the users how the change is really and truly better.

Answer 4

Thou Shalt Not Surprise Users

A lot of users got used to your software and gained productivity. A major change of UI will be disaster to those who can already blind-type or blind-click inside your software. To them, how you layout the windows is irrelevant to their daily work. All they want is getting work done (faster). So if you slow them down, they will hate you.

Some like new stuff

Geek, developers and casual users usually like new stuff. Exchanging a mouse for a trackpad is fun to them. So this type of user welcome new interface and new themes.

Change or not

It really depends whether you want to sacrifice a group of your users for another.

Solution ?

Option. Allow your users to have an option. The option to use the new wonderfully dock-interface, or old efficient MDI interface.

Answer 5

There's not necessarily going to be a universally accepted approval of the change. Some people might find it better, and others will hate it. You just have to live with the fact that you're going to annoy some users and perhaps chase some customers away. Its about using your judgement - deciding what will help the most people, while limiting the annoyance to others, and most importantly, improving your overall sales.

Also dont just assume those users complaining about the change dont have legitimate reasons for doing so. Particularly if you made the change 'just because', which you see a lot of in this industry. If people are complaining about your switching from MDI to a docked interface, it may be because they cant as easily access the information they want to. Make sure you sit down with them and actually watch how they use the software. Developers and users use software differently. And users may have different hardware. That docked interface may look wonderful on your dual 24 inch 1080p wide-screen LCD monitors, but may be painful on a 17 inch 4:3 monitor, for example. Or the previous arrangement allowed them to minimize mouse movements, but with the new system they're fast developing carpal tunnel syndrome. Or before two items they need could be aligned right next to each other to be seen at a glance, whereas now it requires a head movement.

Answer 6

User acceptance is the key. If the users don't like the change, you should not go for it. They are the ones who decide, not you. Just think about how many users the Microsoft Office engineers pissed when they decided to make major changes to the menus, layouts, etc. Did they think that the change was for good? Certainly. Did they test them on enough or the right users? It seems that not.

Answer 7

What's the point? Is your goal to make the application more appealing to new users? Sometimes you have to move in that direction to grow your product and risk losing existing users. The usability may not improve, but it may "look" newer which gives the impression the product has the latest and greatest. You may have an older user base, so don't assume they have the latest and greatest hardware or operating system. I notice a lot of corporate laptops (stickers) on the train that are still on Windows XP.