Array or Malloc?

Question

I'm using the following code in my application, and it's working fine. But I'm wondering if it's better to make it with malloc or to leave it as is?

function (int len)
{
char result [len] = some chars;
send result over network
}

Answer 1

The main difference is that VLAs (variable length arrays) provide no mechanism for detecting allocation failures.

If you declare

char result[len];

and len exceeds the amount of available stack space, your program's behavior is undefined. There is no language mechanism either for determining in advance whether the allocation will succeed, or for determining after the fact whether it succeeded.

On the other hand, if you write:

char *result = malloc(len);
if (result == NULL) {
    /* allocation failed, abort or take corrective action */
}

then you can handle failures gracefully, or at least guarantee that your program won't try to continue to execute after a failure.

(Well, mostly. On Linux systems, malloc() can allocate a chunk of address space even if there's no corresponding storage available; later attempts to use that space can invoke the OOM Killer. But checking for malloc() failure is still good practice.)

Another issue, on many systems, is that there's more space (possibly a lot more space) available for malloc() than for automatic objects like VLAs.

And as Philip's answer already mentioned, VLAs were added in C99 (Microsoft in particular doesn't support them).

And VLAs were made optional in C11. Probably most C11 compilers will support them, but you can't count on it.

Answer 2

Variable-length automatic arrays were introduced to C in C99.

Unless you have concerns about backwards comparability to older standards, it's fine.

In general, if it works, don't touch it. Don't optimize ahead of time. Don't worry about adding special features or clever ways of doing things, because you often aren't going to use it. Keep it simple.

Answer 3

If your compiler supports variable-length arrays, the only danger is overflowing the stack on some systems, when the len is ridiculously large. If you know for sure that len is not going to be larger than a certain number, and you know that your stack is not going to overflow even at the max length, leave the code as is; otherwise, rewrite it with malloc and free.

Answer 4

I like the idea that you can have a run-time allocated array without memory fragmentation, dangling pointers, etc. However, others have pointed out that this run-time allocation can silently fail. So I tried this using gcc 4.5.3 in a Cygwin bash environment:

#include <stdio.h>
#include <string.h>

void testit (unsigned long len)
{
    char result [len*2];
    char marker[100];

    memset(marker, 0, sizeof(marker));
    printf("result's size: %lu\n", sizeof(result));
    strcpy(result, "this is a test that should overflow if no allocation");
    printf("marker's contents: '%s'\n", marker);
}

int main(int argc, char *argv[])
{
    testit(100);
    testit((unsigned long)-1);  // probably too big
}

The output was:

$ ./a.exe
result's size: 200
marker's contents: ''
result's size: 4294967294
marker's contents: 'should overflow if no allocation'

The overly large length passed in the second call clearly caused the failure (overflowing into marker[]). This doesn't mean that this kind of check is fool-proof (fools can be clever!) or that it meets the standards of C99, but it might help if you have that concern.

As usual, YMMV.

Answer 5

Generally speaking the stack is the easiest and best place to put your data.

I would avoid the problems of VLAs by simply allocating the largest array you expect.

There are however there are cases when the heap is best and messing around with malloc is worth the effort.

1. When its large but variable amount of data. Large depends on your environment > 1K for embedded systems, > 10MB for a Enterprise server.
2. When you want the data to persist after you exit your routine, e.g. if you return a pointer to your data. Using
3. A combination of static pointer and malloc() is usually better than defining a large static array;

Answer 6

In embedded programming, we always use static array instead of malloc when the malloc and free operations are frequent. Because of the lack of memory management in embedded system, the frequent alloc and free operations will cause memory fragment. But we should utilize some tricky methods such as defining the max size of array and using static local array.

If your application is running in Linux or Windows, it is no matter using array or malloc. The key point lies in where you use your date structure and your code logic.

Answer 7

The call stack is always limited. On mainstream OSes like Linux or Windows the limit is one or a few megabytes (and you could find ways to change it). With some multi-threaded applications, it could be lower (because the threads could be created with a smaller stack). On embedded systems, it could be as small as a few kilobytes. A good rule of thumb is to avoid call frames larger than a few kilobytes.

So using a VLA makes sense only if you are sure that your len is small enough (at most a few dozens of thousands). Otherwise you have a stack overflow and that is a case of undefined behavior, a very scary situation.

However, using manual C dynamic memory allocation (e.g. calloc or malloc & free) has also its drawbacks:

• it can fail and you should always test for failure (e.g. calloc or malloc returning NULL).

• it is slower: a successful VLA allocation takes a few nanoseconds, a successful malloc could need several microseconds (in the good cases, only a fraction of a microsecond) or even more (in pathological cases involving thrashing, much more).

• it is much harder to code: you can free only when you are sure that the pointed zone is no more used. In your case you might call both calloc and free in the same routine.

If you know that most of the time your result (a very poor name, you never should return the address of an automatic variable VLA; so I'm using buf instead of result below) is small you could special case it, e.g.

char tinybuf[256];
char *buf = (len<sizeof(tinybuf))?tinybuf:malloc(len);
if (!buf) { perror("malloc"); exit(EXIT_FAILURE); };
fill_buffer(buf, len);
send_buffer_on_network(buf, len);
if (buf != tinybuf) 
  free(buf);

However, the above code is less readable and is probably premature optimization. It is however more robust than a pure VLA solution.

PS. Some systems (e.g. some Linux distributions are enabling by default) have memory overcommitment (which makes malloc giving some pointer even if there is not enough memory). This is a feature I dislike and usually disable on my Linux machines.

Answer 8

Something that nobody has mentioned yet is that the variable length array option is probably going to be vastly faster than malloc/free since allocating a VLA is just a case of adjusting the stack pointer (in GCC at least).

So, if this function is one that is called frequently (which you will, of course, determine by profiling), the VLA is a good optimisation option.

Answer 9

This is a very common C solution I use for the problem which might be of help. Unlike VLAs, it doesn't face any practical risk of stack overflow in pathological cases.

/// Used for frequent allocations where the common case generally allocates
/// a small amount of memory, at which point a heap allocation can be
/// avoided, but rare cases also need to be handled which may allocate a
/// substantial amount. Note that this structure is not safe to copy as
/// it could potentially invalidate the 'data' pointer. Its primary use
/// is just to allow the stack to be used in common cases.
struct FastMem
{
    /// Stores raw bytes for fast access.
    char fast_mem[512];

    /// Points to 'fast_mem' if the data fits. Otherwise, it will point to a
    /// dynamically allocated memory address.
    void* data;
};

/// @return A pointer to a newly allocated memory block of the specified size.
/// If the memory fits in the specified fast memory structure, it will use that
/// instead of the heap.
void* fm_malloc(struct FastMem* mem, int size)
{
    // Utilize the stack if the memory fits, otherwise malloc.
    mem->data = (size < sizeof mem->fast_mem) ? mem->fast_mem: malloc(size);
    return mem->data;
}

/// Frees the specified memory block if it has been allocated on the heap.
void fm_free(struct FastMem* mem)
{
    // Free the memory if it was allocated dynamically with 'malloc'.
    if (mem->data != mem->fast_mem)
        free(mem->data);
    mem->data = 0;
}

To use it in your case:

struct FastMem fm;

// `result` will be allocated on the stack if 'len <= 512'.
char* result = fm_malloc(&fm, len);

// send result over network.
...

// this function will only do a heap deallocation if 'len > 512'.
fm_free(&fm, result);

What this does in the above case is use the stack if the string fits into 512 bytes or less. Otherwise it uses a heap allocation. This can be useful if, say, 99% of the time, the string fits into 512 bytes or less. However, let's say there's some crazy exotic case you might occasionally need to handle where the string is 32 kilobytes where the user fell asleep on his keyboard or something like that. This allows both situations to be handled without problems.

The actual version I use in production also has its own version of realloc and calloc and so forth as well as standard-conforming C++ data structures built on the same concept, but I extracted the minimum necessary to illustrate the concept.

It does have the caveat that it's dangerous to copy around and you shouldn't return pointers allocated through it (they could end up being invalidated as the FastMem instance is destroyed). It's meant to be used for simple cases within a local function's scope where you would be tempted to always use the stack/VLAs otherwise where some rare case might cause buffer/stack overflows. It's not a general-purpose allocator and should not be used as such.

I actually created it ages ago in response to a situation in a legacy codebase using C89 that a former team thought would never happen where a user managed to name an item with a name that was over 2047 characters long (maybe he fell asleep on his keyboard). My colleagues actually tried to increase the size of the arrays allocated in various places to 16,384 in response at which point I thought it was getting ridiculous and just exchanging a greater risk of stack overflow in exchange for lesser risk of buffer overflow. This provided a solution that was very easy to plug in to fix those cases by just adding a couple of lines of code. This allowed the common case to be handled very efficiently and still utilize the stack without those crazy rare cases that demanded the heap crashing the software. However, I've found it useful since then even after C99 since VLAs still can't protect us against stack overflows. This one can but still pools from the stack for small allocation requests.