Do threads use virtual memory or real memory?

Question

I was trying to optimize my Linux server to handle 10,000 threads per process while it does just 382 right now. As per this article the following formula is used to find out total possible threads:

number of threads = total virtual memory / (stack size*1024*1024)

This means threads store all their data in virtual memory. And to the best of my knowledge, virtual memory is swap space in a Linux machine which is stored on harddisk than RAM or cache.

So my question is does our threads uses harddisk to store for processing/store its data.

If yes, then doesnt this effect performance? Can we enhance the performance by putting in them on RAM or cache? How?

If no, how exactly do threads works?

Update:

According to useless's answer, virtual memory is a system comprising roughly:

• physical memory (RAM)
• any swapfiles you have attached
• hardware support for translating virtual to physical addresses and issuing page faults when a virtual address isn't available in physical memory
• (kernel) software support for: managing the lookup tables used by that hardware handling those page faults by pulling pages in from swap on demand

Thus, Everything that is on virtual memory is collectively on RAM(Real Memory) and Hard Disk(Swap Files). And as James explain in his answer decision on Ram vs HDD is taken by Kernel using algorithims such as LRU.

Answer 1

to the best of my knowledge, virtual memory is swap space in a Linux machine

Nope, virtual memory is a system comprising roughly:

• physical memory (RAM)
• any swapfiles you have attached
• hardware support for translating virtual to physical addresses and issuing page faults when a virtual address isn't available in physical memory
• (kernel) software support for:
  • managing the lookup tables used by that hardware
  • handling those page faults by pulling pages in from swap on demand

It's up to the kernel to make sure the virtual memory you want is cached into RAM when you want it - unless you're writing your own userspace VM layer (such as databases often do, iiuc), just don't worry about it.

Answer 2

If the thread is actually running then the current instruction , and, any variables the thread is using must be in physical memory.

Most (in fact nearly all) programs reside in virtual memory, and, most programs use virtual memory for storage of variables.

Virtual addresses organized into chunks called pages (these are usually 4096 or 8192 byte blocks).

At any given time each block of virtual memory is stored somewhere in real memory or on the disk in the "swap space" reserved for this.

Your program code deals with virtual addresses, when you branch to a virtual address, or, request access to storage at a virtual address the system (usually at hardware level) locates the current location of the address request and maps it to your virtual address, if the address currently resides on the disk it pages it into real memory and then maps the address.

Obviously when all physical memory is in use if something is paged in then something else must be paged out, so the system looks for the "Least Recently Used" page and copies this out to disk before copying the page you requested in.

In modern systems there are several optimizations and tricks associated with virtual storage.

• Addresses are mapped on a "per process" basis so for instance all C programs in a Linux box start the "main" process at the same address.
• This can enable several 32 bit processes to occupy and use much more than 4GB on a machine as a 32 bit virtual address can be mapped to a real 64 bit address.
• When processes end or memory is otherwise "free"ed the the system just marks the pages as free, they are never copied back to the swap disk.
• Similarly when a new block of storage is requested the system just grabs a free page in real memory, no, disk IO takes place.
• The sleep and hibernate functions force all the memory to be copied to the swap space so that all current processes and there current memory contents can be recreated on wake-up.

Answer 3

First of all, you need to read more on computer memory, because you seems to lack the knowledge in that field.

A thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources.

So, threads are going to use available memory - whatever kind of it is available. How many threads you can start depends on the memory size and how much memory is needed per thread. If thread uses heap (not only stack), then it needs more memory, and in that case you can start less threads.

Answer 4

The simple answer to your question is, they use virtual memory. everything uses virtual memory except a handful of processes related to the OS.

On the other hand, when your thread (or any thread, in any process) is actually running, it is using physical memory. The memory pages associated with that process are swapped in to physical memory which is where the processor does its work.

Answer 5

Virtual memory is your RAM plus your swap space. Virtual just means the address your program sees is different than the address the RAM chip sees. If you need to access memory in swap, the OS will move it into RAM first. If you don't want any swapping, just disable it. If you have enough RAM you don't really need it.

That being said, unless you have a 10,000 core processor, increasing to 10,000 threads isn't really an "optimization." Once you have enough threads to consume all the cores, plus a spare or two for when those threads are blocked, adding more threads decreases performance due to the switching overhead and cache misses. You might still want to use more threads if it makes your program logic simpler, but you will be trading off performance.

Answer 6

optimize my Linux server to handle 10,000 threads per process

As others explained, this is generally wrong. A thread is a costly resource, notably because it has its own call stack (typically, a megabyte) and because it is a task schedulable by the kernel. Threads are even more costly than opened file descriptors.

Read Operating Systems: Three Easy Pieces (freely downloadable textbook).

As a rule of thumb, you don't want to have many threads, and certainly not many runnable threads. The number of runnable threads should generally be at most the number of cores (or a small multiple of that), so about a dozen at most. The number of threads in a process could be slightly bigger. So unless you have a very expansive server (with many processor sockets and cores), you don't want to have more than a dozen runnable threads, and a hundred threads (most of them being idle) in your process (on your desktop).

On Linux, threads and processes are very similar (since both can be created by clone(2)) and both are tasks scheduled by the kernel. Actually the kernel scheduler is scheduling tasks which can be threads inside some multi-threaded process, or the single main thread of a single-threaded process (in that case, you'll name "process" that single thread), or kernel threads. You probably don't want to have more than a thousand schedulable tasks in total on your desktop system.

On Linux, a process is simply a group of threads sharing the same virtual address space (and sharing some other things, such as file descriptor table, etc...). Some processes have only one thread.

A virtual address space is defined by Wikipedia as

"the set of ranges of virtual addresses that an operating system makes available to a process"

^{(but see also this answer explaining that the terminology is not universal, and some Microsoft documentation uses a different and incompatible definition).}

On Linux, proc(5) is useful to understand the virtual address space of some processes. Try both
cat /proc/self/maps and cat /proc/$$/maps in a terminal. See also this, and pmap(1) & ps(1) & top(1).

All user-space programs are running in some process and using virtual memory so every process has its own virtual address space. The physical RAM is a resource managed by the Linux kernel, and applications don't have direct access to RAM (except by mmap(2)-ing /dev/mem, see mem(4)).

So a process don't use directly RAM. It uses virtual memory and has its own virtual address space. The kernel uses paging to manage physical RAM pages and provide the virtual address space and the process abstractions. At any time (even when your process is idle, or when it is running) the kernel could page out some pages (e.g. swap them on the disk). The kernel is configuring the MMU (and handling page miss hardware exceptions in some interrupt handler, either by fetching the page from disk or by propagating a segmentation fault to the process, see signal(7))

You could have green threads above system threads (but green thread libraries are difficult to implement and debug). Look into goroutines used in Go for a fancy example. See also setcontext(3).

Sometimes, your system may experiment thrashing. This happens when the total virtual memory (needed by all processes) exceeds -by a large factor- the available physical RAM. Then your computer becomes unresponsive. Read about resident set size, demand paging, working set, memory overcommitmment, ASLR.

See also -for Linux- fork(2), clone(2), mmap(2), madvise(2), posix_fadvise(2), mlock(2), execve(2), credentials(7), pthreads(7), futex(7), capabilities(7).