8

Currently I'm working on a C/C++ code-base which is fairly portable, it can compile on most Unix like systems as well as MS-Windows (MSVC), using various popular compilers.

Previously I've found testing on different OS's and architectures can help find obscure bugs or bad assumptions.

I worry with the dominance of x86/amd64 our code-base may unknowingly become less portable.

Besides testing on a big-endian system (to find obvious errors with big/little endian), are there some architectures which have characteristics making them better for stress-testing software portability?

Examples of possible differences.

  • different endian.
  • different behavior when threading.
  • behavior of stack memory.
  • size of primitive types (char, short, int, long, float... etc).
  • alignment/padding of structs (which might hide errors).
  • difference in optimizations made by the compiler.

Are there some architectures which have more significant differences to x86/amd64, making them better candidates for exposing code portability issues? (and have C/C++ compilers and libraries - libc, libstdc++).

Asking because its a sizable time-investment to setup a new system, even if its emulated.

in case its not clear what I mean by processor-architectures, eg (x86, amd64, ia64, mips, risc, arm, m68k, ppc, itanium)

Note, I'm not proposing this as a primary way to discover bugs, we run multiple static analysis tools and tests, but in the past we have found errors in code because of differences in less common platforms (SGI, SPARC, Solaris, BSD's etc. However some of these systems are fading out of use)

Improve this question
1

5 Answers 5

Reset to default
10

You are right that using different setups for your tests can increase the chance of accidentally stumbling on some bugs. However, you should consider whether setting up a another testing rig makes sense from a business perspective of things – I suspect you want to sell or distribute useful code, rather than crafting The Perfect Code in your ivory tower.

I worry with the dominance of X86/AMD64 our code-base may unknowingly become less portable.

If your code only ever runs on x86 or AMD64 architectures, then there is little use for testing on other architectures – YAGNI applies here. You'd be better off by expanding your test suite to guarantee all documented behaviour, and by purging the code base of dubious constructs with the help of linters. Using multiple different compilers on different settings is also a low-cost, high-impact strategy for discovering bugs (e.g. GCC and Clang).

If however you explicitly support certain combinations of architectures and operating systems, you should also test those combinations.

If you nevertheless do want to test some more alien setups that are still used fairly commonly, I would recommend:

  • Architecture: SPARC. OSes: Solaris family, Linux, BSD family. Endianness: big, possibly bi. Comments: massive parallelism.

  • Architecture: ARM. OSes: Linux, BSD family, OpenSolaris. Endianness: little, possibly bi. Comments: used in embedded devices, mobile phones.

The features you listed can be tested by varying the following components in the setup:

  • Endianess: architecture.
  • Threading: OS, kernel settings, threading libraries, number of processors.
  • Stack: ?
  • Primitive sizes: preprocessor directives, compiler settings.
  • Struct alignment: compilers.
  • Optimizations: compiler settings.
  • Other: compile using different libc implementations.

My primary exposure to cross-platform programming is reading through the source of the Perl interpreter. Here portability issues are adressed by:

  • … detecting features provided by the used libc implementation, possibly substituting custom functions. This information is then recorded as a set of preprocessor definitions before compilation.
  • … pervasively using macros for numeric types. The sizes can be set during compilation.
  • … making threading optional, as the unthreaded version performs better. In the code, some sections are executed only when compiled with threading support.
  • … mostly ignoring endianess, as this tends to sort itself out. Endianness only becomes relevant when doing something like interpreting a given bit pattern as a 16-bit BE number on a LE system.
  • … explicitly listing supported platforms and also documenting portability issues or differing behaviour on these platforms.
  • … running a huge test suite incl. a ton of regression tests that are executed on a network of CI boxes.
Improve this answer
6
  • Thanks for the answer, - I meant amd64, not ia64 - my bad :S
    – ideasman42
    Commented Aug 3, 2014 at 10:42
  • re: mostly ignoring endianess - worth mentioning file formats (which is mostly where we deal with it), realize you're talking about your own exposure though.
    – ideasman42
    Commented Aug 3, 2014 at 11:24
  • 4
    Testing on non-x86/AMD64 (particularly ARM) can also reveal alignment errors, because the ARM processor gives an error on unaligned access, where the x86 just slows down.
    – Bart van Ingen Schenau
    Commented Aug 3, 2014 at 18:17
  • Unfortunately unaligned access on ARM is a horrible mess. Sometimes it works, sometimes it gives the wrong results and sometimes it gives an error. Which ones it does depends on both the version of arm involved and the configuration of the operating system.
    – Peter Green
    Commented Mar 11, 2020 at 15:56
  • 1
    @gnasher729 please note that this answer is over 10 years old. A lot has changed since then. This answer only mentions ARM for embedded/mobile, now it's a common desktop + server architecture. 32-bit architectures were the norm, ARMv8 was still young. Previous ARM versions definitely had only limited support for unaligned access (ARMv7 assembler manual). Newer versions (ARMv8/AArch64) only require alignment for specific instructions, but otherwise just suffer a performance penalty.
    – amon
    Commented Nov 6 at 18:18
6

I think you've probably covered much of this already, but I'd give some further consideration to:

  • Low memory environment, e.g. mobile with small RAM or embedded devices. While your current customers may not need this, who knows if your library is suddenly in huge demand for engine management systems or home routers.
  • Highly concurrent environment, e.g. more threads and cores than you've ever considered possible. While you may have single-threaded and multi-threaded code paths as a compile option, it's possible that a massive number of threads may expose contention you didn't know you had.
  • Can I supply my own allocators? If not, what happens if I redefine malloc or operator new() to use a slab allocator or otherwise non-standard memory mechanism - can your code still cope?
  • Are you relying on initialisation order anywhere? Any statically-allocated variables?
Improve this answer
2

My employers have been effectively doing this since the late 1980s. I've been doing porting to new platforms, upgrading 32-bit platforms to 64-bit, and updating compilers and operating systems since 1995. My experience on your topics is:

  • different endian.

This is the most noticeable thing, in that pointer arithmetic errors show up quite differently between big- and little-endian platforms. However, in the decade since this question was asked, big-endian platforms have continued to decline, and all the platforms I ship are x86 or ARM, and little-endian.

If your testing load is small enough that you can run testing on an emulator in a useful length of time, then go for it. I can't do that, since the testing takes far too much CPU time.

I have one big-endian testing platform left, Solaris on SPARC, but it's slow and the hardware is expensive. Linux on SPARC is also big-endian. AIX on POWER is big-endian, but Linux on POWER is little-endian. IBM mainframe hardware is big-endian-only, and it has Linux, but it is very expensive.

  • Alignment trapping

You don't mention this, and most architectures don't do it nowadays, but the ability to trap on misaligned accesses is a useful bug-finder. SPARC and IBM mainframe do it, but POWER doesn't. x86 can have it turned on for an individual process, but you'll find that most run-time libraries don't work if you do that.

  • different behavior when threading.

This isn't a big problem, in my experience of working with WIN32 threads and POSIX threads. MacOS threading is a bit weird, and I had to reach below the POSIX layer to get it to work well.

  • behavior of stack memory.

Again, I have not had a problem with this. Just about all architectures have stacks that grow downwards in memory.

  • size of primitive types (char, short, int, long, float... etc).

I haven't had a problem with this, but I've only dealt with relatively conventional C/C++ platforms. The primitive types that have varied in size for me are longs and pointers, which have been either 32- or 64-bit. 64-bit Windows, with 32-bit longs and 64-bit pointers was slightly annoying, but fixed with a few typedefs.

  • alignment/padding of structs (which might hide errors).

I have not had a problem with this, because all the compilers I've ever used pad to natural alignment. This is not required for operation, but it's usually the fastest.

  • difference in optimizations made by the compiler.

Here is where a lot of my career has been spent. These days, optimiser bugs that produce bad code are much rarer, but they still happen. Digging them out, reproducing and reporting them is hard work, but it is worthwhile. You need to do it to a high standard, so that the bugs are as easy as possible to fix, but when you can do that consistently, it pays off. The compiler maintainers, being human, prioritize bugs that are clear and easy to fix, and your bugs get fixed sooner.

  • Floating-point traps.

You don't mention this, but if your code does significant amounts of floating point, the ability to turn on traps for overflows, invalid operations and divides-by-zero is very helpful in digging out bugs. You can do this on x86[-64] Windows and Linux; I haven't met an ARM-based platform that can do it yet.

Improve this answer
3
  • Regarding different primitive sizes, it is somewhat common to have problems when moving between embedded and mainstream or between Unix and Windows. E.g. Windows uses an LLP64 data model (pointer = 64 bit, long = int = 32 bit), whereas POSIX uses LP64 (pointer = long = 64 bit, int = 32 bit). This requires care to avoid, e.g. by using types like size_t/ssize_t for index- or size-like values.
    – amon
    Commented Nov 6 at 18:25
  • @amon: Thanks, you reminded me of a part I'd forgotten.
    – John Dallman
    Commented Nov 6 at 22:58
  • Optimiser bugs are rare. Optimisers turning undefined behaviour into unexpected behaviour are not usual. I remember code detecting overflows by checking say if (x + 1 < x) overflow() which doesnt work on modern compiler because x+1 < x is false unless there is undefined behaviour through overflow, annd the ccompiler can assume it is false as well.
    – gnasher729
    Commented Nov 10 at 21:15
1

At this point in time, from the point of view of high level languages there is very little difference between the most common processors.

At the moment, I write code for 64 bit x86 and ARM processors. For C, C++ and other languages, they are extremely compatible. Little-endian and no alignment restrictions, 8/16/32/64 bit signed and unsigned integers named int8_t to uint64_t. The only difference is long double which is likely not used. Features outside C / C++ are accessed through intrinsic functions. Even vector instructions are portable on a high level. Number of processors, cache sizes, all available portably.

Different when you use very simple embedded processors (but 64 bit ARM is not unusual for an embedded processor).

Improve this answer
0

Use can use user-space QEMU emulator to test programs for different architectures in Linux. The qemu-user package emulates the CPU, and translate syscalls, fcntls, ioctls from the foreign architecture into that of the host.

I recommend an Ubuntu or a Debian distro, and on x86, you just need gcc-{arch}-linux-gnu cross compiler to compile your codes. (of course, if you're using GUI or additional 3rd-party libraries, you need to foreign architecture versions of those.)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.