3

For my basic kernel I refuse to implement the dreaded confusion of virtual memory scheming, so I want only real memory addresses for everything. Some people have argued with me that since virtual memory is hardware supported, and sometimes the MMU hardware is integrated with the CPU, is that a rule that virtual memory is mandatory by hardware-design?

I have at least one good reason to not care about virtual memory at this point, being that the real memory I have is plenty (32 GB), and that I don't care to go through trouble and workarounds with virtual addressing and such(and that I honestly don't like the idea, regardless of the fact that it helps in many areas).

So if my question wasn't clear, is it possible to write a dedicated kernel and not at all implement virtual memory and use only real addresses for everything, with a MMU?

(Some of these low-level programming and hardware-concepts are mind-boggling, even after hours and hours of studying and implementing).

PS: I'm guessing the answer to some point is "yes", but hearing someone with more knowledge on the subject would definitely be a good pointer for me and others who read this.

10
  • 1
    Yes, I don't want to swap or page to disk. Virtual memory works along those concepts, and I don't want to bother with that really at this point, but everything I see is bombarded with the subject with kernel-development strategies and/or OS design implementations. I just want to simply work with RAM for now, and I'm wondering if that's 100% possible given the hardware-design now suited to virtual memory kernels, which is mostly all of them in modern use. Commented Jan 22, 2013 at 22:17
  • 2
    So don't. What's stopping you? Commented Jan 22, 2013 at 22:55
  • 2
    OK, keep in mind that this is a community of volunteers, attempting to answer what is arguably a vague, unanswerable question in a Q&A format. Try to have some respect for the platform and the people who are freely giving their time to try and help you out. Otherwise, there are plenty of books, blogs and other resources if this kind of hand-holding is not your cup of tea. Commented Jan 22, 2013 at 22:59
  • 2
    In regard to 32 Gb never being used, that is what they said when 120 Mb harddisks where introduced in the nineties. In addition, in data analysis jobs you could easily chew through 32 Gb. Assuming that something is so improbable that it cannot be expected to happen, and basing design decisions on that, does not lead to a robust system. Commented Jan 22, 2013 at 23:24
  • 1
    I worked on an old Honeywell system like that once. Memory allocation was managed using a big hard-bound log book. There was a space in the book for each 24-bit word of memory. There were only 64K words, so the book wasn't too big. We allocated code space from the bottom and data from the top. All programming was in assembly language. To install an application, you found enough space for the code and marked that off, then assigned the data locations, and then hard-coded the data addresses into the application. Commented Jan 23, 2013 at 16:42

5 Answers 5

19

You can certainly do that, but it may not save you as much complexity as you think.

One of the main benefits of virtual memory is that it keeps different processes from needing to know which parts of memory other processes are using.

On a system with virtual memory, one of the main jobs of the MMU is to give each process the illusion that the whole machine is theirs. As an example, if you have three programs you want to run on your OS, each can be compiled to start at memory address 0 -- and none of them has to be changed if you want to run all three at the same time. You are right, however, that this adds complexity to the kernel.

If you decide to save this complexity by making it the program's job to not use the same addresses used by other programs, there are two main ways you can go:

  • One option is to make each program use a different range of memory -- if program1 only uses addresses 0 .. 8191, and program2 only uses addresses 8192 .. 16383, then (barring a bug), they won't interfere with each other, but -- and this is a big deal -- you need to plan in advance what range of memory each program will use, so you need to know beforehand all the programs which will run on your OS. This may be workable for an embedded system (and may be the only choice there), but is obviously a show stopper for an OS which you hope other people will build software for. Historically, a method like this was also used for some early shared library implementations, and proved very hard to get right.

  • Another option is to compile each program using what's called position independent code -- code where no absolute addresses are compiled into the program, and all access to variables and code within the program is done by first checking a register (possibly the program counter) for a value pointing to where the program actually is in memory. This allows multiple programs to coexist, but requires the compiler and linker to do more work to make the program work this way, requires the operating system to do more work when loading each program, and has a cost in performance every time access to a variable or function has to be computed using this method. Historically, some operating systems, including classic MacOS before version 7.1, have taken this approach, and something like this is still used in modern shared library implementations.

So, you can move the complexity from the kernel to each running program, or from the kernel to your record-keeping of which program gets loaded where -- but you can't get rid of it altogether.

5
  • I agree with your answer the most, and find it the most helpful. I hate that virtual memory is such a pain to work with when writing a kernel. There should be more light shed on the subject. I'll have to do trial-and-error and see what I can get done without it. Thanks for your help! Commented Jan 22, 2013 at 23:02
  • You don't need to plan in advance. Just keep a list of what memory has been used (and potentially how much space an application uses). Then you allocate the memory at the point when the application is compiled. If it is recompiled it must fit in the same space as before (or less (or you have to reassign new memory from the top (unused space))). I once read of an OS that used this strategy (long since forgotten in time (by me)). Commented Jan 22, 2013 at 23:58
  • Absolutely -- if you build all of the apps which will run on your OS at the same time, in the same place. Of course, this scales... Poorly.
    – jimwise
    Commented Jan 23, 2013 at 0:36
  • there's a third option, have the program keep meta data so the kernel can move all absolute addresses after it is loaded but before it starts to run Commented Jan 23, 2013 at 1:07
  • Yes -- this is delaying the linking step until later, and is a good example of how you can move this complexity, but not really get rid of it. Note that a JIT compiler (as in the JVM) is actually doing just this -- and delaying a lot more of the traditional compile - link - load cycle to run time as well.
    – jimwise
    Commented Jan 23, 2013 at 1:12
12

It's fairly common for embedded kernels not to use virtual memory, because protecting processes' memory spaces from each other isn't as important, and you generally have a relatively small amount of memory. In that case, the MMU is primarily used for protecting appropriate parts of the address space as read only, so you get segfaults when you want them.

With 32GB of RAM, though, you'd require a 64-bit machine to address it all natively, or some other paging scheme. Also, you still have to understand the MMU in order to initialize it properly, so it might not save you that much effort. Unless you're building a dedicated machine that only runs a handful of processes, I would highly recommend biting the bullet and implementing virtual memory.

0
3

An MMU plays two major roles: it translates virtual addresses into physical addresses, and it allows the possibility of trapping (also known as faulting and various other names), i.e. executing code (in the kernel context) when a process dereferences certain virtual addresses.

There is a third related capability, that of having different mappings and different traps for different processes. This is a consequence of executing code to change MMU settings during a context switch. This doesn't require anything special about the MMU (but most processor designs have features in the MMU to make this more efficient).

Some processors (typically high-end microcontrollers, such as the ARM Cortex-M3) do not support translations but nonetheless support trapping: they have a memory protection unit (MPU). An MPU allows the kernel to prevent certain addresses from being accessed. Thus you can enforce memory separation between processes at runtime with only an MPU, you don't need an MMU.

If you have an MMU, you can use it as an MPU. It's up to you to design your kernel such that every virtual address is either always mapped to the corresponding physical address or unmapped. (An unmapped address triggers a trap.)

There is a version of the Linux kernel that supports processors with an MPU but no MMU: µClinux. The code is integrated in the Linux source tree under nommu architectures. The lack of an MMU brings some limitations, in particular the impossibility of implementing fork.

An important thing that you give up if you have no virtual memory is the ability to load every program in the same address space. This makes it considerably easier to compile programs. For example, with virtual memory, you can choose to always load the code of a program at a certain predefined address, store global variables at a predefined address, etc. If every process is stored at a different address, you need to generate position-independent code all the time.

Another limitation of having no virtual memory is that memory management becomes more difficult, because of fragmentation. Suppose you have four consecutive physical memory pages of 4kB each, numbered 0,1,2,3, and you first allocate four page-sized objects, one in each page; then you free the objects in pages 1 and 3. With virtual memory, you can allocate an 8kB object made of pages 1 and 3 by mapping them at consecutive addresses. With a direct physical mapping, page 1 is lost until you need a single-page object or the surrounding objects are freed.

If you want, you can use your processor without activating its MMU. All processors boot with the MMU turned off, because you can't do anything useful until the MMU tables have been initialized to something sensible. With no MMU, you can't enforce any isolation between processes. You also can't swap, or have memory-mapped files.

With address 0 mapped, take care if you program in C or most other languages. There are a lot of runtime environments and non-fully-portable C code that assume that an address of all-bits-zero is the NULL pointer, which does not point to any object.

Depending on the architecture, there may be no way to access some hardware without mapping it into memory. This requires having the MMU up and going. You can still pretty much ignore the MMU by mapping the physical RAM from address 0, mapping devices above the last RAM address, and never changing the MMU mapping.

All in all, by eschewing virtual memory, you're simplifying some parts of the kernel, but making it a lot more difficult to write programs that do anything useful. There's a reason why all high-end processors have an MMU: it's difficult to do anything sophisticated without it. MMU management is not the most difficult part of writing a kernel if you keep it simple; if you find it “mind-boggling”, you have a lot to learn about low-level programming. Writing a kernel with MMU management would be a very good learning exercise.

3

Since you don't take Karl's answer, I will try to answer your question: with no virtual memory, a useful MMU will be based on memory segmentation, pretty much like what the DOS did in the old (good) days - you allocate memory in segments and addresses are always local to the corresponding segments. Another approach is to go with no MMU at all (for embedded systems) - all processes uses a pre-allocated address schema and you will quickly find out 32G is not big enough.

Again, as Karl suggested, you should bite the bullet.

Added more detail to Robert's comment

For a system that needs to dynamically load and unload modules, the free physical memory is always dynamic. To a module that assumes direct physical memory access, there is a hard way to locate it's data and code: when the module is loaded, scan the machine code and rewrite all pointers to the allocated addresses. I have not read anything in detail about this technique but it just sounds pain to me (while windows does this for dlls). An easier way to address this is to use local addresses to segments - the location of a segment may change but the address relative to its beginning never changes.

Whether segmentation is supported in modern computer is not relavent, it's just an memory management solution that is easy to implement.

2
  • Why does the elimination of virtual memory preclude a flat memory model? Why would segmentation be required? Segmentation is an artifact from the old 16 bit Intel processors, I doubt that it is even used anymore in modern computers, not in that form anyway. Commented Jan 22, 2013 at 22:39
  • @GumBall If the answer doesn't help you, clarify your question and say "thanks for trying". How 'bout a little respect for those who are trying to help? At this point, if I knew much about kernel development, I wouldn't help you.
    – Phil
    Commented Jan 22, 2013 at 23:03
1

You certainly don't have to, a lot of small systems didn't. Early versions of Windows, PalmOS and Apple's pre-OS X systems used "handles", which were basically indexes into a table of pointers to memory regions. The idea was that programs would do all their memory referencing via the handles, giving the system the ability to move the memory regions as needed (to prevent fragmentation). It's hard to prevent memory corruption with such a scheme, and it can lead to strange bugs if you create a pointer to a region that gets relocated.

I believe IBM's OS/MFT (going wayyy back) simply partitioned memory into fixed-size regions (defined at 'system generation' time), then loaded a single program into each region. Each program was only supposed to use the memory in its region, so all programs had to know their memory requirements. Depending on the tasks you'll be running, maybe something simple like this could work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.