0

I'm writing a web server as a test project in node.js

Whats really bothering me so far is the lack of control or even awareness of memory usage. Naturally I want to cache some stuff in RAM for faster operations but how much are my objects using? There seems to be no reliable way to tell.

Best I can think up is estimating worst case scenarios, each reference 8B, each character in a string 2B but what is the base cost of an array or an object? Could be nothing or could be something depending on how they are implemented, I've got no idea.

I tried measuring the sizes of things by creating 10M of them and measuring the difference before and after with forcing GC and process.memoryUsage().heapUsed but got some really odd results, Numbers would be 10B, each character in a string 5B, an empty array 42B, and empty object 66B, I doubt its correct but its the only result I have.

At this point I'm thinking that creating a huge blob and manually converting js@v8 native types into bytes and storing them there is the only way to actually have a clue, sounds pretty lame.

Is there a practical way to actually utilize the physical memory my server near its limit without the fear of running out?

|improve this question
  • 1
    I'm not sure node.js is really the right environment if you want that level of control. That's more the domain of C/C++ where you know exactly how every data structure is organized and stored in memory. Somewhat by design, data types are abstract in node.js as is garbage collection. In fact, arrays in V8 can be stored multiple ways depending upon how they are structured to optimize for speed or compactness based on how it's being used. Likewise objects have various fast access optimizations that may or may not be present depending upon circumstances. – jfriend00 Feb 15 at 0:29
  • 1
    You can't even really know when a bunch of objects you declared, used and then no longer have references to have been fully garbage collected. Another thought is that you may consider out of process storage mechanisms such as a database (that does its own caching and manages its own memory usage for you) or if you don't need persistence, you can use a memory store like redis that manages things for you. – jfriend00 Feb 15 at 0:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.