31

I'm working in a software development team as software developer. I've been working on the same project for three years now. The software is a 32-bit desktop based C# application in .NET 4. Our target platform in Windows 7 (we had to support Windows XP till last year). The software communicates with various custom hardware for which custom drivers are written. The hardware manufacturing and driver software is written by our client. There is different driver for 32-bit and 64-bit Windows of course.

During our system testing phase, we execute all/most test cases in both 32-bit and 64-bit Windows 7. I can't recall if we got any bug in our software that exist in only one flavor of Windows. Having this experience I've started to wonder, do we really need to test 32-bit software on 64-bit Windows?

What's the industry standard?

  • 1
    Does your .NET application have any dependencies on native DLLs? I've been bitten a few times only testing on one platform mainly because I forgot to package the x86 native DLLs with my software along with the x64 DLLs. If you start to use a new third party library, that library might also try to load native DLLs behind the scenes, and you won't notice until it crashes on an x86 PC. I've also had to write code that picks which DLLs to use based on whether my .NET app is running in 64-bit mode or not, and that code needs to be tested as well. – Phil Oct 7 '14 at 23:26
  • @Phil: Point noted. The DLL does use a lot of external libraries. I believe all of those DLLs are compiled for x86. The application itself doesn't have any dependency on native DLLs, but it calls native win32 API. – Donotalo Oct 13 '14 at 6:32
31

Most of the bugs we encountered with running 32-bit software on 64-bit windows had to do with the location of the software (Program Files (x86) instead of Program Files), locations of registry keys (some were found in Wow6432Node). We had these problems mostly because we needed to communicate with other software (also 32-bit), and so we needed to test the software on both 32-bit and 64-bit...

When you didn't have these problems, I believe it is quite safe not testing on both platforms when you explicitly compile in 32-bit mode. When compiled in 32-bit, the .NET runtime will run everything in 32-bit mode, and it should work the same as the 32-bit mode on 32-bit platforms.

According to 64-bit Applications (MSDN), 32-bit applications are run in Wow64 mode and Running 32-bit Applications (MSDN) explains this mode in more detail.

  • Are you saying that a 32 bit software if run on 64 bit OS, the .NET on that OS will run the software in 32 bit mode - which is same as running the software in 32 bit OS in .NET? Is there any documentation? – Donotalo Oct 3 '14 at 7:56
  • 4
    @Donotalo: you should inform yourself about the basic switch in your Visual Studio configuration manager (or each project's compilation settings) named "platform", with the options "x86", "x64" and "Any CPU". When you found that switch, F1 is probably your friend. – Doc Brown Oct 3 '14 at 8:02
  • Documentation is added to my answer – David Perfors Oct 3 '14 at 8:04
  • 1
    The biggest issue we had was running with other 32 bit libraries, .NET would simply fail to load them when run on a 64-bit machine. – gbjbaanb Oct 3 '14 at 9:20
  • 1
    @gbjbaanb: you surely meant, when you forgot to use "x86" as platform. – Doc Brown Oct 3 '14 at 9:38
23

The hardware manufacturing and driver software is written by our client. There is different driver for 32 bit and 64 bit Windows of course.

So on 32 bit Windows, your software talks to one driver, and on 64 bit Windows, it talks to a different one? Let us assume there are new versions of these drivers from time-to-time. So when you only test your software on 32 bit Windows you cannot be sure there won't be some differences in the 64 bit driver which will make the combination of your software + 64 bit driver fail. And from the viewpoint of your users, it does not matter who is to blame (you or the author of the driver), all they see is a non-working system. So even if your code is free of bugs, a test might reveal a bug in the 64 bit driver, and finding such a bug might help you take the right measures (like sending a bug report to the author of the driver).

Of course, when you have used those two drivers for years, and you are very confident that the behaviour is exactly the same, you might skip the tests for one platform, following the arguments in @DavidPerfors's answer. As a compromise, you could run tests on 64 bit Windows only whenever a new driver version is available. Actually, this depends on the complexity of the drivers, your experience with and confidence in them.

Some additional things to consider:

  • what OS type is your user base using most? 32 bit or 64 bit Windows? If you decide to test only on one platform, choose the one your users use most often.
  • how severe is it when a new release of the software will not work on a less frequently used platform? For example, can your customers immediately step back and install the previous working release? Do they have only some inconvenience or real financial loss by this? If it is the former, testing on only one platform may be fine, if it is the latter, obviously not.
16

The default assumption in enlightened QA circles is "If you didn't test it, then it doesn't work".

As a practical matter that's usually an unreachable goal to strive for in much the same way application engineers might like to have unit-tests for everything; but they don't believe they'll ever reach it and release on schedule.

However your question can only be answered by, or in conjunction with, sales and marketing. You provide them with a cost to test and they provide an analysis of the market benefit. If estimations on both sides were sufficiently accurate, the answer would be simple

if B > C:
    test_32bit_version()

In my experience, everyone's cost estimates are inaccurate. As far as the other side of the equation, Dilbert once parodied decision making there with "I just asked my cat, Mittens". To do much better, they'd need training in anthropological field methods.

  • The default assumption in enlightened QA circles is "If you didn't test it, then it doesn't work". -- and the default assumption in operations circles is "If you didn't test it, then be prepared to be hunted down like a rabid dog by angry sysadmins". It's hard to test all scenarios but it's a good idea to try hard to test all reasonable ones. It's very rare that a project post mortem ends up deciding that time spent testing a system works properly was time wasted. – Rob Moir Oct 4 '14 at 15:04
6

Seeing as 99% of all Windows installations of Windows 7 and up, and a good portion of Vista as well, are 64 bit, why the hell would you even consider not testing for that platform?
It's just a no-brainer, unless you're making it specifically for a very limited group of users you KNOW are using 32 bit Windows and will continue to do so for the life of your product.

So yeah, test for 64 bit problems. In fact develop on 64 bit platforms, and probably supply a 64 bit version as standard with a 32 bit compiled version as an option for those few customers who've not upgraded to a new computer and OS in the last 6-8 years or so.

  • 1
    I concur mostly, but supplying both 32 and 64 bit versions adds complexity and probably shouldn't be done without a good rationale based on performance or functionality. – user82096 Oct 3 '14 at 8:10
  • 4
    I understand the need to supply 32 bit binaries. I just don't know if he should bother to supply native 64 bit ones without a compelling reason. – user82096 Oct 3 '14 at 8:24
  • 1
    If I were releasing desktop software here in 2014, I would probably only release 64 bit binaries. Remember in the 90s when people were transitioning from DOS to Windows 95? We had a similar argument then -- and clearly DOS got left in the dust, just like 32 bit is now (at least on the desktop, not embedded or mobile). – user22815 Oct 3 '14 at 15:13
  • 4
    I must ask @jwenting. Where did you get that it's 99%? Could you improve on that and post the source? – Malavos Oct 3 '14 at 15:52
  • 1
    Yeah, I don't believe the 99% at all. The business world still has a lot of 32 bit Windows installations out there, between old machines not updated (running Win7 from the early days) or out of fear of incompatibilities. "The majority" are probably 64 bit, but I'd be very unlikely to believe any very high percentage without very good evidence. – Joe Oct 3 '14 at 18:55
2

I would test any installer on as many different Windows setups as possible as from my experience the installers are most likely to fail on different systems.

Otherwise, as you know from your experience with the given software, bugs are most unlikely to just show up on 32-bit or 64-bit, and you can take some calculated risk.

Firstly, you should be having many test cycles with very little code changing between the later cycles as you get close to shipping. Any time you can save, you can use to create more test cases, and/or allow more (and hence smaller) cycles, so giving quicker feedback. (The risk of spending time testing X may be more than the risk of not testing Y, because you are testing X too much.)

Therefore

  • Try to test on the “other bitness” to what you used for the proceeding test cycle.
  • If you know the “bitness” your developer use, start with testing on other one.
  • Divide the test cases up between the “bitness” to give come coverage on each
  • But swap them between the “binnes” on each test cycle.
2

Nope. Likewise, when the FDA is done testing new medicines on mice and rats, they skip over testing on monkeys and just sell it for human consumption.

< /sarcasm>

Yes, yes yes yes yes. There is nothing but sadness in store for your software if you do not test every platform you possibly can. Things are always different, and the assumptions in the designer/coder's head during the project usually only come passably close to modelling real life. So please, test your software. Please.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.