The difference between 32 bit software and 64 bit software is the size of the pointers, and maybe the size of the integer registers. That's it.
That means all pointers in your program are twice the size. And (at least on an ILP32/LP64 architecture) your
longs are twice the size as well. This typically works out to about a 30% increase in object code size. This means that …
- your object code will take ~30% longer to load from disk into RAM
- your object code will take up ~30% more space in memory
- you have effectively lowered your memory bandwidth (for object code) by ~20%
- you have effectively lowered the size of the instruction cache by ~20%
This has a non-negligible negative effect on performance.
Doing this only makes sense if you can "buy back" those performance costs somehow. Basically, there are two ways to do this: you do a lot of 64 bit integer math, or you need more than 4 GiByte mapped memory. If one or both of those is true, it makes sense to use 64 bit software, otherwise it doesn't.
Note: there are some architectures where there are no corresponding 32 or 64 bit variants. In that case, the question obviously doesn't make sense. The most well-known are IA64, which is only 64 bit and has no 32 bit variant, and x86/AMD64 which are, albeit closely related, different architectures, x86 being 32 bit only, AMD64 being 64 bit only.
Actually, that latter statement is not 100% true anymore. Linux recently added the x32 ABI, which allows you to run AMD64 code with 32 bit pointers, so even though that's not a "proper" CPU architecture, it is a way of using the AMD64 architecture in such a way as if it had a native 32 bit variant. This was done precisely because the performance overhead I mentioned above was causing real measurable, quantifiable problems for real-world users running real-world code in real-world systems.