3

We have the following problem: at the end of the iteration we give our Android app to our customers to test.

Our customers report bugs that are not reproducible on our test devices.

How can we deal with this situation?
Are there any tools we can use (for example to record their actions)?

Or is it possible to remote debug on their devices?

It really comes down to this: there are so many devices and Android versions and lots of times problems appear only on specific devices (for example we had a problem with multipart SMS not being sent, only on Samsung devices).

  • 2
    What kind of tests do you run during an iteration? Do you have a real tester on the team (someone with the mindset that breaking the application is a reward)? – Bart van Ingen Schenau Aug 7 '14 at 8:45
  • Bart, the problem really is: how to deal with bugs that are reproducible only on customers' devices – memical Aug 7 '14 at 8:59
  • "Your questions should be reasonably scoped..." (help center). meta.programmers.stackexchange.com/questions/6483/… – gnat Aug 7 '14 at 9:23
  • adb allows connections over ip, but the device needs to be in the same WIFI – memical Aug 7 '14 at 9:27
  • Please don't close! This is a very valuable answer. – david.pfx Aug 7 '14 at 12:44
3

Preliminary note:
You are in need of debugging on a specific device after release, but I will also address testing before release.

One resource to consider is vendor loaner or rental programs for development partners (developer partner programs). Some provide short-term access for testing purposes affordably, perhaps even free.
Most of the hardware vendors have developer programs, which offer hardware for reduced costs or even free (e.g. Samsung, Motorola, Apple, BlackBerry (for BlackBerry Alliance Members), LG). Check them out for the devices you want/need.

There are publicly accessible 'physical' test labs that you can just walk into (or reserve time). They are usually called open device labs: starting point, Google search. Check out if there are any in your neighbourhood and what devices they have.

There's the occasional 'crowdsourced' testing community like uTest that you can ask around.

Note that there are also many on-line services for testing your apps on many devices like AppThwack, TestFlight, TestObject, TestFairy, Keynote a.k.a. DeviceAnywhere, or Xamarin test cloud that maybe have the device you want to test on.
You would also use these before releasing your app, and although these are not 'definitive' testing platforms (there's much to be said for testing on real devices), any tests helps.

Note: Useful overview article

  • We get problems like these, often very specific to the UI (keyboard behaviour, hard menu button, vendor oddities). Testing them on anything other than real physical hardware mostly won't work. I had no idea there was such a thing as an ODL, and one here in Melbourne Australia. Thanks! – david.pfx Aug 7 '14 at 12:42
  • Thanks Jan, we are already using testobject. but will take a look at the others too. the open device labs look pretty interesting – memical Aug 7 '14 at 14:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.