0

Freeing unused memory is a major issue, causing unwanted GC delays in some languages, use-after-free vulnerabilities in others, and Rust has significant complexity to try to fix the issue.

Many server-side applications are request (or transaction) based, with no state shared between requests. An idea I had was that memory allocation could be based on requests/transactions. Roughly speaking, each allocation is associated with a request, and when the request completes, the associated allocations are freed. This would provide automatic memory management from the application's point of view, without GC delays.

Has this approach been attempted in any real-world systems? What are the potential limitations?

4
  • 1
    Region-based memory management (wikipedia.org). Note that there are several names, e.g. arena allocator, or simply "allocator" (referring to a custom allocator implementation that uses this approach).
    – rwong
    Sep 8, 2016 at 12:33
  • How do you ensure that nobody stashes away a pointer to some request-allocated memory and keeps using it after the request is over?
    – user7043
    Sep 8, 2016 at 16:01
  • @delnan - You'd have to have no mutable global state. One of the answers mentions Erlang actors use a private heap - which sounds like a similar setup.
    – paj28
    Sep 9, 2016 at 11:14
  • @paj28 Not only must there be no shared state, you must deep copy every piece of data that's crossing from one actor to another. This can be feasible, but it requires a quite different programming model, and is problematic for some applications (which, admittedly, aren't that common in case of web servers).
    – user7043
    Sep 9, 2016 at 11:26

2 Answers 2

2

This idea is not new:

  • CGI will fire up a completely new instance of the web app for each and every request. After the request ends, the process is terminated by the OS and memory reclaimed just like any other program.
  • PHP in its most simple mode of operation will do the same (modern PHP web frameworks won't, however).
  • The standard model in Erlang is to use one actor per request. In Erlang, each actor has their own private garbage collected heap. So, if the request is very long-running, the garbage collector will only collect dead objects in that single request, but more likely, the request is so short that the garbage collector will never need to run at all and the memory will just be reclaimed by the Erlang VM when the actor ends.

CGI and the per-request mode of PHP have been abandoned because processes are often very expensive. But that's a problem with the implementation of processes, not an inherent problem with the model as such: an Erlang process, for example, weighs only 200 Bytes, and can be started and destroyed in just a few instructions.

I also wanted to point out that "causing unwanted GC delays" is not so much a problem of some languages but rather some garbage collector implementations. For example, the Metronome garbage collector which is available as a paid option for IBM's J9 JVM has configurable maximum pause times down to 1 ms. Azul's C4 (Continuously Concurrent Compacting Collector) has no noticeable pause times since it is fully concurrent: not only can its internal phases run concurrently with each other, it can also run concurrent with the user code.

1

The pattern of allocating a bunch and only cleanup at the end is actually pretty common. Though the main goal is to avoid the cost of the allocation itself in many cases. That the allocation pattern allows for a bulk free is often a happy coincidence.

One of the disadvantages is that the allocated memory could become very large even if most of the memory in it is dead for example by reallocing frequently. Though that should be solvable by being smarter about the allocations like reserving enough space to avoid needing to do a realloc.

Another disadvantage is that RAII destructors will become tricky. Destroying a struct pushed onto this pool would attempt to destroy other objects on the pool which could result in a double destroy. A fix for this would be to not run the destructors for objects pushed into allocated memory but instead link cleanup of their external resources (database connections, files,...) to the request so when the request is finished it can clean up its resources that haven't been already.

Perhaps the main caveat with this however is when you want to allocate memory that stays alive after the request is done like for a session cache or the result of a large computation that uses such a bulk free allocator.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.