0

This question already has an answer here:

  1. As far as I read, files are introduced in OS books. Before an OS is installed on a bare computer system, does the computer use files?
  2. In particular, executable files have formats in OSes, e.g. executable files have ELF format in Linux, which can guide the loader in the OS to load them into memory when running them. Before installation of an OS, does the computer system have the concept of executable files or something similar for storing programs?

If so, what is their format, for example, on a X86 or X86-64 bit bare computer machine?

Thanks.

I am not asking about booting but I am curious how a program is stored on a bare computer. A program may be for booting or not for booting. In particular, I am interested in how a user program is stored on a bare computer.

marked as duplicate by gnat, durron597, user22815, Bart van Ingen Schenau, Robert Harvey Oct 27 '15 at 2:56

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 2
  • I am not asking about booting. BUt currious how a program is stored on a bare computer. A program may be for booting or not for booting. – Tim Mar 14 '15 at 23:37
  • 5
    The question is still, nevertheless, a duplicate, in that the question's answer can be found by a keen reader after the reader has gone through and understood all of the answers posted in that earlier question. There is no new information to be added by any answer to the current question. I understand that you may have one or more puzzles, but to answer those puzzles you just need to read more, and think more. – rwong Mar 14 '15 at 23:41
  • 5
    You are asking about booting. Booting is the answer to the question "How can a computer operate when the operating system is not yet loaded?" A computer having files, but not having an operating system, still has some sort of software that can read said files. – Robert Harvey Mar 15 '15 at 0:04
  • @Robert: what are the file formats for those executable files? Do ELF and other formats for executable files still make sense? – Tim Mar 15 '15 at 0:16
15

Since you're not talking about booting, your answer is actually that the concept of "executable," or even "files" is rather irrelevant.

The concept of files is an OS concept. Without an OS, all you have is a harddrive, with some bits on it. It is the OS that gives structure to that information that makes it meaningful to think of the data as a file.

For a "bare" computer, without considering booting, the idea of "storing user programs" is meaningless besides the very boring definition of "it stores the files by putting the 1's and 0's on a harddrive." At this level, these 1's and 0's are indistinguishable from those of text, or images, or anything else because there is no "defined" structure.

People are focusing on booting because that is where your concept of "executable file" becomes meaningful for the first time. When a BIOS boots (from ROM), one of its last tasks is identifying a media to boot from (i.e. a hard drive), and "executing" something from it. This is usually implemented as "the bios will command the harddrive to produce the 1's and 0's from a specific numbered location on the harddrive. These 1's and 0's will then be stored in memory, and the Instruction Pointer of the 1st CPU will be pointed at those 1's and 0's."

This is the point where we get closest to an answer to your question. This block of 1's and 0's is formatted in a way that is defined by the CPU as a set of instructions. I can't quite call it a file, but is definitely a continuous block of CPU defined values loaded from a harddrive.

Now this particular set of instructions gets to take ownership of the computer. It will command everything from here on out. If I wished, I can now go ask the harddrive for a different block of 1's and 0's, store those in memory, and "run them." This suggests that this block of 1's and 0's known as "machine code" would be an answer to your question "how are user programs stored." However, it is an incomplete answer. I could also write my block to go read from the harddrive, then invert every bit it read, before jumping to that data as a set of instructions. In this case, now the 1's and 0's are not "machine code," but rather a sort of encoded machine code. This shows that there is no one "defined" way to store user programs on a bare computer except for the first program, which is special.

Now eventually we're going to run some programs that bring in the concept of a file system on the harddrive. Now, for the first time, we have a concept of a file, so we can talk about "executable files." However, because we're so far along in the process, there is no one way they get stored. As shown before, we could arbitrarily choose to invert every bit.

So the reason why your question involves terms that don't really go together is that the concept of an "executable file" is at odds with the idea of a "bare computer." The two words simply do not go together. However, in the initial moments of booting, there is a section of the harddrive, which is not a file but gets really close to being called a file, which is in an executable format defined by the CPU.

  • downvoted: a harddrive is a peripheral and would require some kind of operating system to enumerate it before any attempt to access even raw bits/bytes on it. – James Snell Mar 15 '15 at 10:40
  • 3
    @JamesSnell - The BIOS enumerates the hard drives, not the OS, and then it knows the boot order (due to configuration) so it then knows to look for a master boot record (MBR) on each device, in sequence, and if it finds one, it loads the MBR into memory and executes it. At that point it's whatever's in the MBR that now has control of the computer. This is the correct answer, I think. – Scott Whitlock Mar 15 '15 at 12:06
  • The BIOS is a program supplied/stored on an EPROM (or EEPROM to be more exact). It is not a hardware device and therefore cannot enumerate a thing... – James Snell Mar 15 '15 at 13:09
  • 1
    @JamesSnell, I think we're edging into hair splitting. I think you'll find that catagorizing the code in the classic PC BIOS as an operating system is a minority viewpoint. Most folks would say a motherboard with a BIOS in ROM was bare hardware. Heck, back in the 80s many knowledgeable people wouldn't even categorize MS-DOS as an operating system, since it left so much of the management of the system up to the application programmer. – Charles E. Grant Mar 15 '15 at 19:34
  • 1
    @JamesSnell I think the splitting of hairs is going on because the OP explicitly stated they were not thinking about booting. We've been assuming that means he is referring to after booting. I think the version you are using for bare (no code which enumerates hardware) is meaningful, but I don't think its the version the OP was going after. – Cort Ammon Mar 15 '15 at 23:12
4

To get an OS up and running, there are various things being executed which are typically called "loaders" and "bootstrappers".

Their collective purpose is exactly as their names imply: to get the OS up and running, one step at a time.

Some of these executable code are stored in read-only memory (ROM), or some type of reprogrammable memory (such as Flash ROM), or a certain predetermined location on the storage disk (located on a so-called "boot sector", which is a predetermined location on a "disk partition") from which the machine can be started up.

Your suspicion that at least some of these executable code must exist outside the concept we call "files" or "file system" is correct. Not all of them are; but at least some of them must be.

The computer hardware-firmware, and operating system itself typically are able to access these "non-file" storage areas to perform infrequent maintenance upgrades to these executable code.

The purpose of allowing these upgrades from software is an engineering choice driven by the consumer market - it is cheaper to maintain computers if is it possible for customers to perform the upgrades, than to require machines to be shipped back to a factory or maintenance center to do the same.


Are these executable code different from the code found in "executable files"? Maybe, maybe not. There are certainly some differences.

A 64-bit machine may spend 99.99% of its time executing 64-bit machine code once booted; but during the initial seconds it may have to execute some 32-bit machine code.

Likewise, a fully-loaded OS will execute in privileged mode and the rest of the code execute in user mode. But during bootstrapping most of the code may execute in privileged mode - or rather, in some "simply unprotected" mode, because the CPU mechanism for software fault handling may not have been fully initialized.

Code that needs to execute without the OS, obviously cannot make OS calls. They can still make jumps and subroutine calls to other code - as long as they are located within the same piece of code that is loaded into RAM in one swoop.

This brings up the "size limitation" of bootstrapper code. The previous stage has to know exactly how big is the code size of the next stage. That amount of executable code/data would be brought into RAM before any of it can be executed. In the absence of an OS (and also of any memory-management mechanism), the bootstrapper cannot make subroutine calls located outside what has been brought into RAM - unless if the bootstrapper knows how to load that the extra code.

The BIOS may have to provide a way to facilitate (simplify) the loading of extra data from the storage disk. Performance or security is not the concern - the only concern is simplicity for the caller (which is the bootstrapper). Otherwise, that would be a very tedious task, and would eat up the precious space the bootstrapper code has to fit in.


What formats are used for such bootstrapping code?

These code must follow some kind of conventions in order to work correctly. Whether these conventions qualify as "a format" is a word-play.

For example, the starting offset of the first machine instruction in the bootstrapping code must be specified. Otherwise, how else do you start executing it?

Some bootstrapping code are required to carry a checksum, or even a digital signature (i.e. become digitally signed), to prevent trying to execute a bootstrapper whose data is corrupted, or spoofed or tampered with by malicious intent.

These conventions must be agreed upon by the machine manufacturers, operating systems vendors and/or maintainers, and so on.

  • 1
    It may be worth noting that many current computers use UEFI, which is really just a more sophisticated BIOS. UEFI does have some very basic support for reading and writing files in a few common formats, but it doesn't fully implement a filesystem. This article on UEFI may be helpful. – Charles E. Grant Mar 15 '15 at 0:47
4

Back in the stone age of computing, one of the common modes of booting up was to read an executable program into memory completely under hardware control and then start running it.

The PDP-10 for example, could command the paper tape reader to read a block of data, take the data presented and stuff it into memory, and then start running it when the block was complete. This was used to read a 16 instruction program which was enough to read more data from the tape and place it in controlled locations of memory, and eventually to start the bigger program which could run the console tty, read the hard disk, and load the operating system.

  • As did the IBM System/360 family of the early 1960s. You set an input device address using front-panel switches, pressed the "Initial Program Load" (IPL, aka boot) button, and it read 24 bytes from the device into locations 0-23, continued the I/O command program with the Channel Command Word (CCW) it read into locations 8-15, and when the I/O program ended, loaded the Program Status Word (PSW) from locations 0-7, launching whatever code it read in. – Ross Patterson Mar 16 '15 at 1:39
4

Files are a low-level abstraction provided by the operating system thru some file system layer. Another important abstraction provided by OSes is processes (running instances of programs). The OSdev site has interesting practical information. The underlying bar metal computer (e.g. your x86 laptop or desktop) does not know about files.... It is reading and writing sectors (often blocks of 512 bytes or of 4096 bytes) on disks (and the OS builds the file abstraction above that). Your usual OS (Linux or Windows) is using the file abstractions to store programs. But the bar metal computer is just executing machine code instructions (and some of them are read from the disk by the OS). At the lowest level, close to the hardware, programs do not really exist (only machine code)! Executables (hence programs) and processes are also an abstraction (& convention) provided and handled by the OS. On Linux, most executable binaries are in ELF.

But you could imagine an operating system without any files. You'll have to write such an OS from scratch, running on the bare metal (and of course you'll have to define some notion of program, care about how that thing boots, run programs and uses the disk). It could have some orthogonal persistence model (like Phantom, EROS, etc...)

In the 1980s & 1990s some Lisp machines did not have any files. They persisted their entire state on the disk.

In practice computers are becoming increasing complex. You can look at UEFI and at the Grub boot loader as having some limited form of files. and some firmware (like Open Firmware) also provide a limited form of files.

Another amusing way of looking at this is to imagine that aliens might have computers which do not use any files (but something else). Files are an invention (somehow related to technology and social issues of the IT sector in the 1960s)

And of course some small microcontrollers don't have file systems. My washing machine probably has some 8 bit microcontroller, but no file system.

I guess that files are becoming less relevant than before (databases being increasingly important). Notice that a plain Unix-like filesystem might not be the best way to organize your data on some multi terabyte disk. You probably want to store more meta-data about it than a native filesystem offers (and you might use some database for that).

  • Just out of curiosity: Is there any database system which stores its data on disks rather than on files? I mean it would receive one or more dedicated disks at configuration time? I am only aware of filesystem backed databases – but I am not that much into that kind of things. – Michael Le Barbier Grünewald Mar 15 '15 at 22:34
  • Certainly, it was the case at least on Suns in the 1990s. Databases required a specific disk partition. – Basile Starynkevitch Mar 15 '15 at 22:35
2

Before an OS is installed on a bare computer system, does the computer use files?

No. At startup some code is copied from an EPROM into memory at a known location and the processor executes those instructions. All it does is load execute instruction at the 'current' memory address increment it and repeat forever. The CPU is unaware that a harddisk controller or anything else is even connected let alone do anything useful with any of it - it requires instructions on how to do that and those instructions are usually the lowest layer of an operating system.

does the computer system have the concept of executable files or something similar for storing programs?

With nothing to put files on there is no file system and no file format. Initially everything is loaded from an EPROM into RAM (main memory) and then executed instruction by instruction. So far as the CPU is concerned they're just an array of bytes.

The BIOS is an example of code that the EPROM contains - it is the code for a (very simple) operating system which does what it needs to (like wake up disk controllers and other peripherals) and look for a another piece of code to run. The same mechanism that boots your system and allows it to operate enough for you to load/install your OS of choice is the one that kickstarts everything time you power up the system.

If you felt that inclined and had the right information about the exact hardware on your motherboard you can write your own EPROM and run code from that. Of course at some point you'd need to reuse functions and as that need increases you'd find yourself writing your own OS and all the headaches that go with it...

  • Why copy the EEPROM into RAM? Why not just put the EEPROM (or ROM) into an addressable spot in memory, say at location 0? – Scott Whitlock Mar 15 '15 at 12:09
  • They do exactly that in some systems but not in the x86 from what I recall. I've not done bare metal x86 in quite some time though. – James Snell Mar 15 '15 at 13:05
  • @ScottWhitlock, different kinds of memory run at different speeds. ROM is generally slow compared to dynamic RAM. If you are going to be referring to the contents of ROM a lot it makes sense to pay the one time cost of copying it into RAM. – Charles E. Grant Mar 21 '15 at 23:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.