Is a compiler warning for 64bit to 32bit `size_t` truncation actually useful?

Question

I have been looking at VC++'s C4267:

Compiler Warning (level 3) C4267

'var' : conversion from 'size_t' to 'type', possible loss of data

The compiler detected a conversion from size_t to a smaller type.

To fix this warning, use size_t instead of type. Alternatively, use an integral type that is at least as large as size_t.

I have been looking at this, and - maybe unsurprisingly - when converting a 32 bit code base to 64 bit, there a lots of these errors, mainly wrt. vector.size() etc.

Notes:

• /W3 is the default level in VC++ - so this is a default warning.
• The warning is triggered only for size_t -- even though it is "just" a typedef for unsigned __int64, if you use uint64 directly, the warning isn't triggered as far as I can tell.

Now, this made me wonder, purely on a conceptual level, does this compiler warning actually make any sense for the 64bit size_t => 32bit type?

size_t, is a typedef used for sizes. Mostly of std collections etc. in lower level code for raw memory sizes.

As such, truncating these values from 64bit to 32bit will produce erroneos results IFF the size in question > 4G. That's a lot, I mean really a lot for real world data counts.

Truncating to 16 bit or 8 bit certainly deserves a warning, but this isn't handled separately here. As such, I have the impression that this default warning provides more noise than value to the majority of developers.

Does this warning as-is provide some real benefit that I'm missing here? Or is it really to simplistic? Maybe/possibly I'm missing an important use case for size_t?

Answer 1

The warning is triggered only for size_t -- even though it is "just" a typedef for unsigned __int64, if you use uint64 directly, the warning isn't triggered as far as I can tell.

But uint64_t didn't change size when you moved to 64 bits, and size_t did.

The compiler isn't warning you just because this is a narrowing conversion, although it might on another level.

It's warning you because code that previously used uint32_t for size_t, has now silently changed to using uint64_t instead, and this might cause surprising bugs with no visible change in the code.

Answer 2

I have been looking at this, and - maybe unsurprisingly - when converting a 32 bit code base to 64 bit, there a lots of these errors, mainly wrt. vector.size() etc.

Those warnings are a sign that the developers invoking those methods failed to pay attention to the return types. The fact that there are lots of them means that failure was repeated lots of times.

The question to be asking is why, when the library offers a return type guaranteed to work right every time (size_type in the case of vector.size()), did the callers use something else? To put a finer, folksier point on it, why did they bring a five-pound bag when they knew full well someone might ask them to stuff ten pounds of something into it?

There are legitimate cases where you may well have a larger integer on hand that needs to be stuffed into a smaller space. When you encounter those situations, there are measures you can take to make sure it happens safely and without a compiler warning.

Now, this made me wonder, purely on a conceptual level, does this compiler warning actually make any sense for the 64bit size_t => 32bit type?

If you're writing conceptual code, it doesn't matter a bit. None of the code in my systems is conceptual, and compilers are tools for handling real code you intend to run. For that, it matters a lot.

Do you want your programs to sprout bugs?

size_t func() { return 4294967296; }
uint32_t returned;
returned = func();
if ( returned == 0 ) {
    // This block will be executed.  FAIL.
}

How about security problems?

BugHugeThing thing;
uint32_t how_big = sizeof thing;
BigHugeThing *other_thing = malloc(how_big);
// ...elsewhere...
memcpy(other_thing, &thing, sizeof *other_thing);  // Buffer overflow.  FAIL.

...truncating these values from 64bit to 32bit will produce erroneos results IFF the size in question > 4G. That's a lot, I mean really a lot for real world data counts.

That's a very big IFF. Are you willing to stake your professional reputation on making the assumption that those truncations will never cause problems?

4G may sound like a large number in your world, but it isn't in mine, where data sets with more elements than that are generated and shipped around daily and processed on systems that can inhale all of it. A TV show you watched may well have been shot in 4K, where truncating a 64-bit size_t to 32 bits loses access to all but the first two minutes of footage as it came from the camera.

Bottom line: Return types exist for a reason. Use them.

Answer 3

Your compiler saw that technically there is a potential for error in your program. So the warning does make sense.

It's your job as a programmer to judge if you should fix this potential bug or ignore it.

Personally, I'd say fix it. But that's a human judgement. A computer is not build to do this. It can only tell you the facts and fact is there is a potential mistake in your code for at least one possible branch.

If you don't find the warning useful, then turn it off. It will certainly not vanish from the compiler, because other people with different human judgement may want to make use of it.

Answer 4

One reason for size_t warnings is that if it goes wrong, it will go really, badly wrong. Let's say I create a book with lots of images and the total number of bytes for everything is stored in a size_t. And you convert it to uint32_t. This will go wrong when my book reaches 4 GB in size. In other words, at a point where I have invested a huge amount of work in creating the book.

Or you write software that is just by small customers, medium sized customers, and big customers. The customer that will run into your bug and lose data will be the biggest, baddest, meanest customer that you have, who will chew up you and your company.

(For similar reasons, MacOS X and probably other 64 bit operating systems make the first 4GB of address space inaccessible, that way casting a valid pointer to int or unsigned int and back will never work, which is much better than working until the data gets big).