15

I'm currently working on a library written in C. Many functions of this library expect a string as char* or const char* in their arguments. I started out with those functions always expecting the string's length as a size_t so that null-termination wasn't required. However, when writing tests, this resulted in frequent use of strlen(), like so:

const char* string = "Ugh, strlen is tedious";
libFunction(string, strlen(string));

Trusting the user to pass properly terminated strings would lead to less safe, but more concise and (in my opinion) readable code:

libFunction("I hope there's a null-terminator there!");

So, what's the sensible practice here? Make the API more complicated to use, but force the user to think of their input, or document the requirement for a null-terminated string and trust the caller?

4

Most definitely and absolutely carry the length around. The standard C library is infamously broken this way, which has caused no end of pain in dealing with buffer overflows. This approach is the focus of so much hatred and anguish that modern compilers will actually warn, whine and complain when using this kind standard library functions.

It is so bad, that if you ever come across this question at an interview - and your technical interviewer looks like he's got a few years of experience - pure zealotry may land the job - you can actually get pretty far ahead if you can cite the precedent of shooting someone implementing APIs looking for the C string terminator.

Leaving the emotion of it all aside, there is much that can go wrong with that NULL at the end of your string, in both reading and manipulating it - plus it is really in direct violation of modern design concepts such as defense-in-depth (not necessarily applied to security, but to API design). Examples of C APIs which carry the length abound - ex. the Windows API.

In fact, this problem was settled sometime in the '90s, today's emerging consensus is that you shouldn't even touch your strings.

Later edit: this is quite a live debate so I'll add that trusting everyone below and above you to be nice and use the library str* functions is OK, until you see classic stuff like output = malloc(strlen(input)); strcpy(output, input); or while(*src) { *dest=transform(*src); dest++; src++; }. I can almost hear Mozart's Lacrimosa in the background.

  • 1
    I don't understand your example of the Windows API requiring the caller to supply the length of strings. For example, a typical Win32 API function such as CreateFile takes a LPTCSTR lpFileName parameter as input. No length of the string is expected from the caller. In fact, the use of NUL-terminated strings is so ingrained that the documentation doesn't even mention that the file name must be NUL-terminated (but of course it must be). – Greg Hewgill Jun 12 '12 at 19:44
  • 1
    Actually in Win32, the LPSTR type says that strings may be NUL-terminated, and if not, that will be indicated in the associated specification. So unless specifically indicated otherwise, such strings in Win32 are expected to be NUL-terminated. – Greg Hewgill Jun 12 '12 at 19:51
  • Great point, I was imprecise. Consider that CreateFile and its bunch is around since Windows NT 3.1 (early 90s); the current API (i.e. since the introduction of Strsafe.h in XP SP2 - with Microsoft's public apologies) explicitly deprecated all the NULL-terminated stuff it could. The first time Microsoft felt really really sorry for using NULL-terminated strings was actually much earlier, when they had to introduce the BSTR in the OLE 2.0 specification, in order to somehow bring VB, COM and the old WINAPI in the same boat. – vski Jun 12 '12 at 20:11
  • 1
    Even in StringCbCat for example, only the destination has a maximum buffer, which makes sense. The source is still an ordinary NUL-terminated C string. Perhaps you could improve your answer by clarifying the difference between an input parameter and an output parameter. Output parameters should always have a maximum buffer length; input parameters are usually NUL-terminated (there are exceptions, but rare in my experience). – Greg Hewgill Jun 12 '12 at 20:19
  • 1
    Yes. Strings are immutable on both the JVM/Dalvik and the .NET CLR at the platform level, as well as in many other languages. I would go so far and speculate that native world can't quite do this yet (the C++11 standard) because of a) legacy (you don't really gain that much by having just part of your strings immutable) and b) you really need a GC and a string table to make this work, the scoped allocators in C++11 can't quite cut it. – vski Jun 12 '12 at 21:13
17

In C, the idiom is that character strings are NUL-terminated, so it makes sense to abide by common practice - it is actually relatively unlikely that users of the library will have non-NUL-terminated strings (since these need extra work to print using printf and use in other context). Using any other kind of string is unnatural and probably relatively rare.

Also, under the circumstances, your testing looks a little odd to me, since to work correctly (using strlen), you are assuming a NUL-terminated string in the first place. You should be testing the case of non-NUL-terminated strings if you intend your library to work with them.

  • -1, I'm sorry, this is simply ill-advised. – vski Jun 12 '12 at 19:07
  • In the old days, this wasn't always true. I worked a lot with binary protocols that put string data in fixed length fields that were not NULL terminated. In such cases, it was very handing to work with functions that took a length. I haven't done C in a decade, though. – Steven Burnap Jun 12 '12 at 19:08
  • 4
    @vski, how does forcing the user to call 'strlen' before calling the target function do anything to avoid buffer overflow problems? At least if you check the length yourself within the target function you can be confident about which sense of length is being used (including terminal null or not). – Charles E. Grant Jun 13 '12 at 1:07
  • @Charles E. Grant: See comment above about StringCbCat and StringCbCatN in Strsafe.h. If you just have a char* and no length, then indeed you have no real choice but to use the str* functions, but the point is to carry-the-length-around, thus it becomes an option between str* and strn* functions of which the latter are preferred. – vski Jun 13 '12 at 5:46
  • 2
    @vski There is no need to pass around a string's length. There is a need to pass around a buffer's length. Not all buffers are strings, and not all strings are buffers. – jamesdlin Mar 22 '17 at 10:34
10

Your "safety" argument doesn't really hold. If you don't trust the user to hand you a null-terminated string when that's what you documented (and what's "the norm" for plain C), you can't really trust the length they give you either (which they'll probably get by using strlen just as you're doing if they don't have it handy, and which will fail if the "string" wasn't a string in the first place).

There are valid reasons to require a length though: if you want your functions to work on substrings, it is possibly much easier (and efficient) to pass a length than to have the user do some copying magic back and forth to get the null byte at the right place (and risk off-by-one errors along the way).
Being able to handle encodings where null bytes are not terminations, or being able to handle strings that have embedded nulls (on purpose) can be useful in some circumstances (depends on what exacltly your functions do).
Being able to handle non-null-terminated data (fixed-length arrays) is also handy.
In short: depends on what you're doing in your library, and what type of data you expect your users to be handling.

There's also possibly a performance aspect to this. If your function needs to know the length of the string in advance, and you expect your users to at least usually already know that information, having them pass it (rather than you calculating it) could shave a few cycles.

But if your library expects ordinary plain ASCII text strings, and you don't have excruciating performance constraints and a very good understanding of how your users will interact with your library, adding a length parameter doesn't sound like a good idea. If the string isn't properly terminated, chances are the length parameter will be just as bogus. I don't think you'll gain much with it.

  • Strongly disagree with this approach. Never trust your callers, especially behind a library API, make your best effort to question the stuff they give you and fail gracefully. Carry the darned length, working with NULL-terminated strings is not what "be loose with your callers and strict with your callees" means. – vski Jun 12 '12 at 19:14
  • 2
    I agree mostly with your position, but you seem to put a lot of trust in that length argument - there's no reason why it should be reliable than the null terminator. My position is that it depends on what the library does. – Mat Jun 12 '12 at 19:18
  • There is a lot more that can go wrong with the NULL terminator in strings than with length passed by value. In C, the only reason one would trust the length is because it would be unreasonable and impractical not to - carrying buffer length is not a good answer, is just the best one considering the alternatives. It is one of the reasons why strings (and buffers in general) are neatly packed and encapsulated in RAD languages. – vski Jun 12 '12 at 19:23
2

No. Strings are always null-terminated by definition, the string length is redundant.

Non-null-terminated character data should never be called a "string". Processing it (and throwing lengths around) should usually be encapsulated within a library, and not part of the API. Requiring the length as a parameter just to avoid single strlen() calls is likely Premature Optimization.

Trusting the caller of an API function is not unsafe; undefined behavior is perfectly ok if documented preconditions are not met.

Of course, a well-designed API shouldn't contain pitfalls and should make it easy to be used correctly. And this just means it should be as simple and straightforward as possible, avoiding redundancies and following the language's conventions.

  • not only perfectly ok, but actually unavoidable unless one moves to a memory-safe, single-threaded language. Might have dropped some more neccessary restrictions... – Deduplicator May 9 '15 at 1:44
1

You should always keep your length around. For one, your users may wish to contain NULLs in them. And secondly, don't forget that strlen is O(N) and requires touching the whole string- bye bye cache. And thirdly, it makes it easier to pass around subsets- for example, they could give less than the actual length.

  • 4
    Whether the library function deals with embedded NULLs in strings needs to be very well documented. Most C library functions stop at NULL or length, whichever is first. (And if written competently, those that don't take length never use strlen in a loop test.) – Steven Burnap Jun 12 '12 at 19:33
1

You should distinguish between passing around a string and passing around a buffer.

In C, strings are traditionally NUL-terminated. It is entirely reasonable to expect this. Therefore there is usually no need to pass around the string's length; it can be computed with strlen if necessary.

When passing around a buffer, especially one that is written to, then you absolutely should pass along the buffer size. For a destination buffer, this allows the callee to ensure that it does not overflow the buffer. For an input buffer, it allows the callee to avoid reading past the end, especially if the input buffer contains arbitrary data originating from an untrusted source.

There is perhaps some confusion because both strings and buffers could be char* and because a lot of string functions generate new strings by writing to destination buffers. Some people then conclude that string functions should take string lengths. However, this is an inaccurate conclusion. The practice of including a size with a buffer (whether that buffer be used for strings, arrays of integers, structures, whatever) is a more useful and more general mantra.

(In the case of reading a string from an untrusted source (e.g. a network socket), it is important to supply a length since the input might not be NUL-terminated. However, you should not consider the input to be a string. You should treat it as an arbitrary data buffer that might contain a string (but you don't know until you actually validate it), so this still follows the principle that buffers should have associated sizes and that strings do not need them.)

  • This is exactly what the question and other answers missed. – Blrfl Mar 22 '17 at 10:54
0

If functions are mainly used with string literals, the pain of dealing with explicit lengths may be minimized by defining some macros. For example, given an API function:

void use_string(char *string, int length);

one could define a macro:

#define use_strlit(x) use_string(x, sizeof ("" x "")-1)

and then invoke it as shown in:

void test(void)
{
  use_strlit("Hello");
}

While it may be possible to come up with "creative" things to pass that macro that will compile but won't actually work, the use of "" on either side of the string within the evaluation of "sizeof" should catch accidental attempts to use character pointers other than decomposed string literals [in the absence of those "", an attempt to pass a character pointer would erroneously give the length as the size of a pointer, minus one.

An alternative approach in C99 would be to define a "pointer and length" structure type and define a macro that converts a string literal into a compound literal of that structure type. For example:

struct lstring { char const *ptr; int length; };
#define as_lstring(x) \
  (( struct lstring const) {x, sizeof("" x "")-1})

Note that if one uses such an approach, one should pass such structures by value rather than passing around their addresses. Otherwise something like:

struct lstring *p;
if (foo)
{
  p = &as_lstring("Hello");
}
else
{
  p = &as_lstring("Goodbye!");
}
use_lstring(p);

may fail since the lifetime of compound literals would end at the ends of their enclosing statements.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.