0

I am trying to build software in JavaScript that emulates an operating system. For all intents and purposes, it is an operating system, even though it is JavaScript and we are running in the Node.js or the browser basically. But that shouldn't matter, the fact that it is JavaScript. It could be in any language or platform for that matter.

What I'm stuck at is the beginning. I am used to writing code in higher level languages that are functioning on top of the operating system, so the process architecture is already built, so I've never had to really deal with it.

The broad question is: how do I create a process architecture?

The specific question is: at a high level, how do I create securely isolated processes in theory?

I essentially have this:

var all_processes = []
var specific_process = -1

function set_process(index) {
  if (all_processes[index]) {
    throw new Error('Process taken')
  } else {
    specific_process = index
    all_processes[index] = {}
  }
}

function call_function(name, ...args) {
  all_processes[specific_process][name](...args)
}

But I am imagining doing async and multithreading, etc. All the fancy features of processes. I would like to know how you can keep processes isolated basically, in a secure way. How does the OS do it? Because you need to at some point create and manage the processes, but then the processes themselves can't communicate (unless through a specific protocol perhaps). So it feels like on one level, I need to pass the specific process around to each function call, but then you lose the clean API of not having the process in every parameter. But then if you do that, then what's to say you don't grab another process and pass that around (i.e. how do you maintain security of only accessing that one process?). Also, checking the process on every call would add performance overhead.

Essentially how to create secure processes, at least at a high level what I should be searching/looking for, if not a description of how to do it.

Wikipedia says:

Process isolation can be implemented with virtual address space, where process A's address space is different from process B's address space – preventing A from writing onto B.

I don't understand what this means though, what to do exactly. What does this make my code look like in theory?

Perhaps sandboxes offer some insight, but Wikipedia doesn't offer much, delegating to the operating system without details:

A sandbox is implemented by executing the software in a restricted operating system environment, thus controlling the resources (for example, file descriptors, memory, file system space, etc.) that a process may use.

Privileges also doesn't reveal much.

This paper says the following:

Most operating systems use a CPU’s memory management hardware to provide process isolation, using two mechanisms. First, processes are only allowed access to certain pages of physical memory. Second, privilege levels prevent untrusted code from manipulating the system resources that implement processes, for example, the memory management unit (MMU) or interrupt controllers. These mechanisms’ non-trivial performance costs are largely hidden, since there is no widely used alternative approach to compare them to. Mapping from virtual to physical addresses can incur overheads up to 10–30% due to exception handling, inline TLB lookup, TLB reloads, and maintenance of kernel data structures such as page tables [29]. In addition, virtual memory and privilege levels increase the cost of inter-process communication.

As a solution they present software isolated processes (SIPs), and say:

The design and implementation of a system based on SIPs is a major contribution of this work. A software isolated process is a collection of memory pages and a language safety mechanism that ensures that code in a process cannot access another process’s pages. A SIP replaces hardware memory protection with static verification of program safety. Singularity uses language safety and a fast communication mechanism built on channels [15] to enforce a system-wide invariant that neither the kernel nor any other process contains a reference into a given process’s object space. Because different process’ object spaces always reside on disjoint memory pages, memory reclamation is straightforward when processes terminate.

I don't understand how this could work at all. Though this is helpful:

Moreover, the system maintains the invariant that there exists at most one pointer to an item in the exchange heap. When a process sends a message, it loses its reference to the message, which is transferred to the receiving process (analogous to sending a letter by postal mail). Therefore, processes cannot use this heap as shared memory, and messages can be exchanged very efficiently through pointer passing, not copying.

  • I do not think you need to do much in this regard, as JavaScript already have means of encapsulation. – Theraot Dec 1 '19 at 1:49
  • @Theraot I don't want to cheat and use language specific features, I want to reimplement everything myself using only 1 array! – Lance Pollard Dec 1 '19 at 1:52
  • 1
    "When a process accesses an address, the processor looks up that address in the page table. If the page table indicates that the process shouldn't have access to the address, a page fault occurs, which generally kills the program." I would like to know what this would look like in JS with 1 array, sort of thing. – Lance Pollard Dec 1 '19 at 1:55
  • May I suggest to use language features to emulate hardware? I mean, consider that stuff like DMA and protected memory, which in modern computers have hardware facilities, are not part of your operating system. – Theraot Dec 1 '19 at 3:22
  • 2
    You may find this useful: Virtual Memory and Address Translation. Also get a copy of the book "Operating system design and implementation". – Theraot Dec 1 '19 at 3:34
4

Basic hardware architecture

First thing first. If you are emulating a Von Newman architecture (instead of a Harvard architecture). You have the code in the same array as the data. I am assuming Von Newman. Then your big array is both for data and code.


What the operating system does

Ideally an operating system should create the illusion that the process is running alone in the system. The memory, the CPU time, and every device, will be abstracted. They will not be accessible directly, instead the operating system provides an API that both makes it easy to use them and prevents the process to interfere with others.

Ah, by the way, that also means that interruptions should never go directly into process memory. Instead they will be handled by a driver, and dispatched appropriately. Will get back to this.


Creating a process

As you already know, you are going to break the memory into pages. To load an executable in memory, you need to put the code of the executable there... somewhere. So, you will create an empty process, assign pages to that process with execute privilege (you need to store in a table, what process created what page and what privileges they have), and load there (from IO device, yeah, with that driver) the code of the executable.

Once the code is loaded, you create a thread for the process (do not forget to allocate pages for its stack) , and set the pointer to the next instruction of that thread at the start of the program (you may have a header in the executable where it is specified, for example, or it could always be at the same virtual address), and it will be scheduled.

Allowing the execution pointer to go into a data page would be an easy way to allow the process to modify its own code. It would also allows for vulnerable processes. That is a process would have to implement its won measures to prevent the execution of data. If a process is vulnerable, a malicious user software could do a well crafted sequence of operations that results in the process allocating and setting data to the instructions the attacker wants, and then executing it. This is why we want pages with and without execution privilege.


Allocating memory

Now, the process needs to allocate data memory to work. That is, the process needs pages with read and write privileges, which is where the variables it needs will be stored. Either have the process do the calls to get these pages, or declare them in an executable header.

We can imagine a routine in the code of the executable that keeps track of much data memory is available to the process and if the program asks for more, it will ask the operating system for more pages (this would be part of any standard library/runtime worth its salt, so that the developer does not have to write it). Think “malloc”.

The operating system has to return a handle to the page to the process, so that the process can free it. Or, if the process ends with pages that has not been freed, the operating system can reclaim them.

About the address space... each page will have an address in real memory and an address in process memory. And we need to do translation from one to the other.

When the process wants to read a pointer, the operating system has to look up in what page it belongs, and read from that page at the right offset. Given that this routine would not look in pages of other process, there would be no way – unless an exploit is found – for the process to access pages of another process…


Inter-process communication

Consider if you to support sharing pages. That is, a process could give privileges over its pages to other processes, via a system call. This would put give an address in the other process memory to the same page.

Another common means to provide inter-process communication is named pipes/channel. It is another type of operating system object. It also has a handle and system calls to use it. It should work as a queue to push data… the trick is that it can be found by name, allowing a producer process to create it, and the consumer to find it by name and receive.

Do this without copy? Have pages that do not belong to any process. They are exchange pages. The producer process would write a message by a system call, and the operating system would share read privilege (but not write) with the consumer. Therefore, there is no need to make multiple copies to deliver it to each consumer.


Some security concerns

Make sure your operating system is not vulnerable under buffer overflow on any system call.

Also anything that calls into process code from the operating system (such as sending a message, a timer, starting a thread, hooks, etc…) need extra care. In particular when they can be triggered by another process. For example, if another process can load a page, give execution privileges on it to another process, write data to it, then set a hook to that page for the other process, you have code injection. So, you know, don’t allow that.


On Virtual memory

The operating system can choose to do swap and virtual memory, the process does not need to be aware of it. Instead the operating system will exchange pages with the swap/pagination file to make the pages the process wants available.

In case a page is not available (because whatever), we have a page fault.

If we have reached the end of the process memory address space, or if the operating system is unable to allocate more pages (for example, run out of swap or swap is disabled), we an an out of memory error.


The driver

Alright, let us talk about the drivers. Let us say we want to do an async read from disk (let us say from an open file). This is more or less what you want:

  • The process has a file handle.
  • The process allocated a buffer to read.
  • The process creates a custom event.
  • The process passes the file handle, the buffer and the event to an operating system call.
  • The operating system delegates the call to the device driver.
  • The driver gets where to read from the file handle.
  • The driver the asks the device to read, via interruptions.
  • If you have DMA (Direct Memory Access) the device can write directly to the buffer (without using the CPU) and sets an interruption handled by the driver. Otherwise the driver needs code to copy the data from the device.
  • Either way, when the copy is done (either the driver got the interruption or did the copy directly), it does a system call signal the event.

Now, what is the event? It is something a thread in the process can wait on. That is, the operating system will signal it, to resume the thread (the thread will not continue execution right away, it would be moved to a ready to run state).

3

As wikipedia says, you allocate a process to an address space, and don't let the address spaces overlap.

For one, you could use a javascript array for an address space.  Then have a different such array for each process.  Though a process would be able to use it's indexes internally within itself to access its own memory, it would never be able to index into another process's address space (the other process' array).

Async and multithreading are orthogonal to processes.  Neither of these terms implies inter process communication, which you will need in some form for a complete OS, whether network packets, shared memory, or shared files.

  • "For one, you could use a javascript array for an address space." I would like to only have 1 array, and thus reimplement the logic of having multiple arrays which cheatingly handles the isolation. – Lance Pollard Dec 1 '19 at 1:51
  • 1
    Yes, that would work. You might give each process a base position in the shared array, and add any index they use to access their content to that base, so that they each think they have location 0, for example; denying access below their zero and above their limit (where possibly a next process lives in the real array). This is similar to how modern operating systems/processors work, except that they use hardware features to specify the base address (akin to having a location 0 at a different physical memory address for each process). – Erik Eidt Dec 1 '19 at 2:25
  • 1
    Modern os/hardware allows expanding a process' local address space without running into the next process, that is one of the features of virtual memory. To simulate in software (infinitely expandable address space for processes, yet all allocated from same array) you'd have to implement some form of paging (page mapping) in software. – Erik Eidt Dec 1 '19 at 2:31
  • Paging has performance/space costs -- it is basically tables of multiple indirections. Let's note though that hardware mitigates some of these cost via the TLB mechanism, which does caching of mappings for recently used pages. – Erik Eidt Dec 1 '19 at 2:39
1

It may help to highlight the basics here. Note that memory protection is usually something built-in on the hardware level of modern processors. For the Intel family this started with the 80286 processor. Memory management features were further enhanced with the 80386 architecture which lived on for a long time. The point is, to implement these features in software you would basically need to write an interpreter. For you that is the only way to stay in control.

A microprocessor goes about it a little differently. Although it is basically an interpreter, it would be inefficient to perform access checks for each instruction before it is executed. It can however set the stage for execution of code such that when memory is addressed outside of the range appointed to a process, a hardware interrupt will occur. This is cheap (efficient). You have no means to go that route in your JavaScript program.

So although it may be good exercise, don't expect it to come close to a common implementation as far as popular operating systems are concerned.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.