2

I asked this question on an IRC channel, sadly I am going around in circles.

I am aiming for a high level overview (but with some technical details if necessary) on how a function such as printf() from stdio.h "talks" to the Windows operating system.

I know a bit about MSVCRT.dll, bits of the Windows API such as how it communicates with kernel32.dll which goes to the Native API in ntdll.dll. I think VisualStudio uses msvcrt100.dll for C Standard Library...

Somebody recommended using an open source C standard library to understand what happens under the hood. Aside from not knowing how to use one in my Visual Studio project, I would still not know HOW this communicates with the OS.

My terrible understanding which misses several steps is as follows:

1) Syntax of printf() is checked against header stdio.h

2) When program is run it uses msvcrt100 for printf

3) msvcrt100 then loads the necessary Windows library such as kernel32

4) Kernel32 then passes it onto ntdll.dll

This question focuses on C, but if C++ is the same feel free to post.

Windows only.

  • You can use the stepping debugger to trace deeper into the DLLs. However, that will not reveal what happens once the string (characters) are passed into the OS. The short answer is that, once a whole string (terminated by a newline and a null, or explicitly flushed) is received by the OS, the OS will need to pass that to the console window, which will then display the string on the text window. Flushing the buffer is a key step. Incomplete strings do not get displayed; they remain in a buffer. – rwong May 30 '18 at 11:33
  • @rwong Just to clarify - I am not interested into HOW the DLLs "talk" to the OS, but the high level steps of how my program "talks" to those DLLs via c standard library. – user5623335 May 30 '18 at 11:36
  • Then compiling the project in Debug configuration, setting a breakpoint, and starting the test application stepping into each function call should be sufficient. – rwong May 30 '18 at 12:10
  • @securityauditor Are you just wanting to know how any program "talks" to a dll? – candied_orange May 30 '18 at 12:55
  • I make a simple vanilla Hello World program that uses printf(), how does that program "talk" to the OS? Just a high level overview. I assume there is a wrapper that loads a DLL, maybe kernel32.dll or what? – user5623335 May 30 '18 at 14:40
4

While the specific details vary between operating systems, you probably want to start somewhat with an understand of

  • an object file format (.obj), created by the compiler/assembler, and
  • an executable file format (.exe), formed by by the linker
  • a dll file format, also formed (and consumed) by the linker

These (disc-based) file formats contain sections for machine code, initial program data values, relocations and a symbol table as well.

Before the code can be executed by the hardware, all symbolically described values (i.e. main, printf) have to be fully resolved to memory locations.

The compiler (and assembler) have incomplete information at compile time about the final memory locations of various pieces of code and data — so, they share this information in the object file symbolically rather than as final memory addresses — this is where the relocations and symbol table come into play.

The symbol table has both imports and exports — the exports gives string names to offsets within sections (code/data) the object file; whereas the imports are associated with just string names (externals for later resolution).

Relocations tell the consumer of the object file how & where to fix up the machine code and data, once the memory addresses of the symbols is known; thus, relocations typically have references to entries in the symbol table (they also reference sections in the object module...).

A .exe file for a program has one special location main that is marked as the entry point in the headers for the executable.  A .exe for a program also has all external references resolved, one way or another — however, the linker that produces the executable file still does not have complete information about the memory addresses of all of the symbols as that usually doesn't happen until load time.  So, the .exe file still has relocations and a symbol table.

The .exe combines all the .obj file's into a single file: the code sections are concatenated, as are the data sections.  As compared with the .obj file, some of the obj file's relocations can be resolved and thus removed from the .exe file; other relocations can have their form simplified (referring to the code or data section rather than referring particular symbols — this makes for a shorter relocation entry).

Typically the operating system loader ultimately determines the locations of code and data sections, and this completes the assignment of memory address to symbols, meaning that any outstanding relocations can now be performed.

A DLL (.dll) file is like a .exe file except that it has specific exports.  The .exe file created by the linker may have references to .dll files.  These reference are read by a dynamic linking loader, so that the operation of loading an .exe file require also loading a .dll file, which may further require additional .dll files to be loaded.  All of them are cross linked to each other as per their relocation entries.

The hardware then executes machine code instructions like call printf, where the reference to printf is specified by its memory location.  There are many approaches, some involving jump tables or code sequences of several instructions; however, suffice it to say that during execution of the machine code, the hardware only sees/knows about memory addresses and not symbol names.  All symbolic references are eventually resolved to memory addresses by the system of relocations and symbol table entries.

printf, when appropriate (e.g. local buffer full), will ultimately invoke a system call of some sort to perform i/o such as flushing the local buffer to disc or device.  A system call is a way for a user process to request an operation of the operating system.  System calls similar to regular calls except that the caller is in the user process and the callee is an operating system entry point.  System calls provide a controlled method of raising the privilege (from user to kernel) as needed to provide access shared devices.  System calls don't use memory address, but instead each system call is associated with a simple integer index — this allows the .exe's and .dll's that call into the operating system to remain unaware of kernel memory addresses.

  • So printf() will eventually make a system call to print something to screen, I assume via WinAPI (usermode) which goes to Native API (Kernel mode)? – user5623335 May 30 '18 at 14:53
  • (sprintf prints to a char buffer, fprintf goes to a file handle parameter.) – Erik Eidt May 30 '18 at 15:25
  • @securityauditor, yes. When you call printfit will call a routine in a dll, which will buffer your content, and eventually flush that to stdio via a call to the kernel (by making a system call). During initialization, this dll will ask the kernel if stdio is a printing device or a file, and will alter its later behavior based on that (for files it will buffer 4k or so, for printing devices like screen, it will flush the buffer on newline (\n) character. – Erik Eidt May 30 '18 at 15:28
  • Hello World.exe -> MSCVRT100.dll -> kernel32.dll -> ntdll.dll -> kernel stuff happens here which prints "Hello World" to screen? – user5623335 May 30 '18 at 15:55
  • I think so. That kernel stuff is that it will ultimately send that character string to the appropriate device driver. For example, if stdio is redirected to a file, then the driver will be a file system driver, which will do its own buffering and ultimately result in disc I/O to a file. I don't know the exact flow for how those characters would be rendered on the screen, and it probably depends on the environment running the command line program. – Erik Eidt May 30 '18 at 16:17
1

There are two ways a windows program interacts with a dll.

The normal method is that part of the process startup involves windows mapping the content of the dll into the process's address space, and associating the (loaded) addresses of the exported functions (and data) with the (default) call targets in the program's code. By the time that main has begun, everything behaves as if the functions in the dll were in the exe all along.

The other method is that the loading of the dll is delayed until after startup, and pointers to the dll's functions are asked for. The two important functions here are LoadLibrary and GetProcAddress.

0

You probably want to read the documentation for WriteConsole. It's the native Windows function that most closely corresponds to printf. As you can see, it requires a formatted string - it works more like puts. You may need to call sprintf first, before calling WriteConsole.

C++ has a similar split of functionality. std::cout eventually calls WriteConsole, whereas operator<< does the formatting of individual arguments.

  • Are you sure it uses console-io instead of file-io? – Deduplicator Jun 18 '18 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.