I'm a self-taught programmer, just in case this question is answered in CS 101. I've learned and used lots of languages, mostly for my own personal use, but occasionally for professional stuff.

It seems that I'm always running into the same wall when I run into trouble programming. For example, I just asked a question on another forum about how to handle a pointer-to-array that was returned by a function. Initially I'm thinking that I simply don't know the proper technique that the designers of C++ set up to handle the situation. But from the answers and discussions that follow I see that I don't really get what happens when something is 'returned'.

How deep a level of understanding of the programming process must a good programmer achieve?

  • 3
    My advice: Learn some x86 assembly (DOS or otherwise). Then learn to read some of the assembler output of some small pieces of C code. Ask questions if you don't understand the output. Repeat. This will force you to understand what is happening at the CPU level
    – Earlz
    Commented Oct 17, 2011 at 22:32
  • 2
    – Job
    Commented Oct 17, 2011 at 23:38
  • Earlz - Do you mean that I should learn to program using the x86 instruction set? Is that the 'CPU level'?
    – bev
    Commented Oct 18, 2011 at 0:22
  • Job - thx, that was fun. He's actually made a few errors, tho, just FYI.
    – bev
    Commented Oct 18, 2011 at 0:32
  • Bev, and all, the answer by @ScottWhitlock is correct. The answer marked as correct is false. Lots of people understand hardware. And, understanding hardware and what your code translates to, can help you use the machine efficiently. The opposite, you can bring the machine to its knees, so to speak. Look up the problem of cache-oblivious algorithms, for some examples. Concurrency and latency provides another set of examples.
    – DrM
    Commented Jan 2, 2022 at 17:08

10 Answers 10


No. Nobody understands what's going on at the hardware level.

Computer systems are like onions -- there are many layers, and each one depends on the layer underneath it for support. If you're the guy working on one of the outer layers, you shouldn't care too much what happens in the middle of the onion. And that's a good thing, because the middle of the onion is always changing. As long as the layer or layers that support your particular layer continue to look the same and support your layer, you're good.

But then again...

Yes. I mean, you don't need to understand what's really happening inside the onion, but it helps a lot to have a mental model of what the inside of a typical onion looks like. Maybe not the deepest part, where you've got gates made up of transistors and such, or the next layer or two, where you've got microcode, a clock, instruction decoding units etc. The next layers, though, are where you've got registers, the stack, and the heap. These are the deepest layers where you have a lot of influence over what happens -- the compiler translates your code into instructions that run at this level, and if you want you can usually step through these instructions and find out what's "really" happening.

Most experienced programmers have a slightly fairy-tale version of these layers in their head. They help you understand what the compiler is talking about when it tells you that there was an "invalid address exception" or a "stack overflow error" or something like that.

If you're interested, read a book on computer architecture. It doesn't even need to be a particularly new book -- digital computers have been working in approximately the same way for a long time. The more you learn about the inside of the onion, the more astounded you'll be that any of this stuff works at all! Learning (approximately) what's going on in the lower layers makes programming both less mysterious and, somehow, more magical. And really, more fun.

Another thing you might look into is embedded onions. Er, I mean embedded systems. There are a number of embedded platforms that are pretty easy to use: Arduino and BASIC Stamp are two examples. These are basically small microprocessors with a lot of built-in features. You can think of them as onions with fewer layers than your typical desktop PC, so it's possible to get a pretty thorough understanding of just what's going on in the whole system, from the hardware on up to the software.

  • 2
    Thanks. This basically answers my question. I'm an EE who has done chip-level (i.e. transistor-level) design of registers, adders, multiplexers, etc, so I get the lowest level (unless we're talking quantum mechanics). I also can use the languages I know fairly well. I just have a huge gap in the middle level (stack, heap), where you say the compiler does its work. Since, as you put it, I want my programming experience to be "less mysterious and,..., more magical." it seems like i should study the levels that are still unknown to me.
    – bev
    Commented Oct 18, 2011 at 0:04
  • @bev: In that case, you really should check out a platform like Arduino.
    – Caleb
    Commented Oct 18, 2011 at 3:32
  • sorry to be dull, but I checked out Arduino, and I can't really see how using it would help me understand how a compiler treats pointers and arrays differently. What am I not seeing?
    – bev
    Commented Oct 18, 2011 at 6:13
  • 1
    @bev: If you just want to find out how functions are called, you can probably spend 30 minutes reading about that and be done. If you want to get a better sense of how everything works together, it'll be easiest with a small system. It's the best way to get the whole onion in your head at once. AVR, the family of chips on which Arduino is based, is a nice, general purpose, easy to use system with an instruction set that's small enough to learn without too much trouble.
    – Caleb
    Commented Oct 18, 2011 at 7:16
  • Ah, OK. The home page is a little murky on that aspect of their products. I'll look again.
    – bev
    Commented Oct 18, 2011 at 23:04

You're not talking about the hardware level, you're talking about what the compiler really does with what you tell it to do.

You most certainly do need this level of understanding in order to figure out what went wrong when it's not obvious, especially when dealing with a memory stomp situation.

  • Loren - Yes! Thanks for the simple truth. Now I need to figure out the best way to learn what c++ compilers do with their data types. BTW, as an EE, I know that it's not the hardware level literally. I just didn't know what you CS geeks call it. (Still don't for that matter. Compiler level?)
    – bev
    Commented Oct 18, 2011 at 0:18
  • BTW -- memory stomp?
    – bev
    Commented Oct 18, 2011 at 0:34
  • @Bev: You just proved my point here--if you don't even know what a memory stomp is you'll have an awful time finding a bug due to one. A memory stomp is when something writes to a location that it's not supposed to and erases (stomps on) whatever happened to be there. If you're lucky whatever you hit was immediately vital and at least it blows up. If you're unlucky the program just keeps going with some holes in it's data. Commented Oct 18, 2011 at 1:03
  • thanks for the clarification. It also shows me how much I don't know, since as far as I know I just write to the heap or the stack, with no finer control.
    – bev
    Commented Oct 18, 2011 at 6:16
  • @Bev: The problem comes when you write someplace you don't think you're writing. You have something on the stack and you make a pointer to it. You leave the routine--the item goes away, the pointer doesn't. Now what happens when you write to that pointer?? Or you have an array of 100 items--what happens when you write to item #200? Commented Oct 18, 2011 at 21:04

Understanding Program Memory != Understanding Hardware

Understanding the Memory Hierarchy == Understanding Hardware

To answer your generic question: It depends. It can't hurt to understand hardware, but understanding it will not help in all cases.

Based on your example, you just need to understand more about how memory is divided up and how it is organized when you are running a program. Understanding hardware will not help you in this regard, because memory (as visible to a program) does not even truly represent the hardware thanks to the magic of virtual memory.

If you were curious about performance issues based on the order in which you access memory, NOW you would benefit from understanding hardware, the memory heirarchy, cache misses, page faults, and all the glorious wonderful goodness that comes from hardware.

  • Stargazer - I'm not at the point yet where I can worry about performance issues. Soon, hopefully. Thanks for you comments.
    – bev
    Commented Oct 18, 2011 at 0:20

If you do decide to learn a bit of assembler, you should probably learn something like 6502 assembler on a Commodore 64 (emulated, of course), or 68000 on an Amiga.

You can get some idea of the Commodore 64 here...


The classic everything-you-need-to-know book is the one described here...


You can probably find a PDF scan if you look around.

IMO, 6502 is easier than Z80, and 68000 is easier than 8086 - more regular instruction sets etc.

But the CPU is only one aspect of the hardware. Also, a modern CPU is a massively different beast, and it does things that are transparent even from the point of view of compilers - such as presenting a virtual address space.

A particular advantage of the 6502 on the C64 is that not only is the CPU simple, but there's some very simple to hack-around-with hardware too. I used to have great fun playing around with the SID music chip.

So - it's probably a worthwhile exercise if you don't spend too much time on it. I learned 6502 assembler as my second language when I was about 14, right after Commodore Basic. But mostly it's getting that very simple working model so that you can add more sophisticated ideas to it with a minimum of misunderstanding.

Some useful things you can learn working in assembler...

  • How CPU registers work.
  • How memory addressing works, including indirection.
  • How the CPU stack works.
  • How bitwise logic works.
  • How the CPU controls I/O devices.
  • How interrupts work.

One particular reason I'd recommend it is to get a better intuition of the way simple steps operate entirely deterministically and mechanically and utterly without intelligence or common sense. Basically getting used to the imperative execution model in it's purest and most stubbornly ignorant form.

Precisely how useful it is to know most of those things now, though, is a difficult question.

One thing you won't learn is how to play well with a memory heirarchy. Those old machines mostly had a simple memory model with no layers of cache and no virtual memory. You also won't learn much about concurrency - they were certainly ways to handle that, but it mostly meant interrupts. You didn't need to worry about mutexes etc.

Sometimes, a mental model of how these things once worked, or of how assembler works, can even mislead. For example, thinking of a C pointer as an address can lead to undefined behaviour issues. A C pointer is normally implemented as an integer containing an address, but there's no guarantee that that's strictly true. For example, on some bizarre platforms, different pointers may point into different address spaces. This becomes important when you want to do arithmetic or bitwise-logic with two pointers.

Unless you have one of those bizarre platforms, you may not think you care about that - but compilers these days are more and more likely to exploit standards-undefined behaviour for optimisation.

So a mental model of the system architecture can be useful, but it's still important to code to the language spec., not to a hypothetical model that your language and platform may not respect.

Finally, a lot of useful mental model stuff comes from getting an idea of how compilers generate code - and code generation for modern languages is very different from the quite trivial compilers available back then.

This is a favorite book of mine for that...


Along with the stuff about parsing and ASTs etc, it covers code generation for a range of language paradigms - imperative, OOP, functional, logic, parallel and distributed - and also for memory management. If you want to know how polymorphic method calls work without getting bogged down in CPU instruction set details, a book like this one is your friend - and there's a new edition due out soon.

  • Steve - wow. I'm near speechless with the completeness and focus of your answer to my question. Thanks so much for taking the time to write this whole thing. I'll definitely take your suggestions.
    – bev
    Commented Oct 18, 2011 at 6:23
  • 1
    I would suggest that PDP-11 assembler is a bit nicer to learn than all of the others mentioned. What all of the others teach are the limitations forced by more limited hardware resources, and/or by more limited hardware design and forethought. Something like one of the all-too-common 8051 family teaches how really bizarre the programming model can get on such limited hardware (where Steve's mention of different address spaces, for example, come into play). Commented Nov 18, 2012 at 3:06
  • @Greg - I never got to play with a PDP-11, I'm afraid. Nor an 8051 - I did some embedded work for a while, but that was using an 8096-family chip. I just had a look here though - interesting. I heard about the Harvard architecture before some time, but I had no idea there was something like this that was very popular and is still in use.
    – user8709
    Commented Nov 18, 2012 at 3:38

It helps a lot to know and understand the abstraction presented by the hardware, and a little of the general idea about how that illusion is created -- but trying to truly understand how modern hardware really works is a tremendous amount of work from which you're likely to see only minimal return.

If you'll pardon a minor diversion: this reminds me of something I noted a few years ago. Decades ago (up through the late 1970's or so), most people thought computers were one step short of magical -- hardly affected by the laws of physics, capable of all manner of things that made little real sense, and so on. At the time, I spent a fair amount of time trying (mostly unsuccessfully) to convince people that no, they weren't magic. They were really fairly ordinary machines that did a limited number of things very quickly and dependably, but were otherwise extremely mundane.

Nowadays, most people's view of computers have changed. They're now fairly ordinary -- to the point that quite a few very ordinary people have a practical grasp of them. Just for example, a while back while I was having supper, I saw/heard a waiter and waitress on their break discussing what she should get in her new computer. The advice he was giving was entirely reasonable and realistic.

My view of computers has changed too though. I've gone to Hot Chips, and before that the Microprocessor Forum going back to the mid-1990's or so. I probably know more about microprocessor hardware than at least 99% of programmers -- and knowing what I do, I'll say this: they're not ordinary anymore. They do almost break the laws of physics. I've done a lot of low level testing and I can say this for sure: getting past the illusion created by the CPU and into the level of showing how the hardware really works is often incredibly difficult. I wish I could post a picture of one of our setups with a computer buried under cables from no fewer than 4 logic analyzers just to properly measure one aspect of how caching works on a modern CPU (not to mention some truly fastidious programming to ensure that what we measured was exactly what the CPU was doing, and nothing else).

  • Jerry - thanks for your comments. Being an EE, I'm more comfortable with the transistor level than some of the higher abstraction levels. I'm really just wondering what I need to know to be a good C++ programmer.
    – bev
    Commented Oct 18, 2011 at 0:38
  • That picture sounds interesting. Why can't you post it? Commented Oct 18, 2011 at 0:39
  • @Bev: You don't really need to know anything at the transistor level to be a good programmer. Those abstractions are there for a reason, and you can almost always consider anything at an abstraction level below that of machine code/assembly to be completely irrelevant and just assume it works. Commented Oct 18, 2011 at 0:40
  • @MasonWheeler: I took it where I used to work, but since I don't work there any more, getting access to it would probably be a bit more difficult (probably not impossible -- I quit on good terms, but even so...) Commented Oct 18, 2011 at 2:23

Twenty years ago it was important, but not so much now - there are a lot more abstraction layers between software and modern hardware.

It is useful to know things like needing multiple threads to take advantage of multiple cores or that using more memory than exists on the system is a bad thing, but beyond that you don't really need it unless it is your job to write those abstraction layers.

The rest of your question suggests that you may be more concerned with the compiler than the hardware, which is a bit different. You may run into cases where it is important, but these tend to be either trivial (infinite recursion doesn't work very well) or the kind of edge cases where you can feel good about solving it but will likely never run into the same problem again.

  • Yes, you're right, I'm more concerned with the compiler. Also, thx for your suggestion about multiple threads, multiple cores, etc. It's just gone into my toLearn notes file.
    – bev
    Commented Oct 18, 2011 at 0:07
  • @bev multithreading is easy to learn, just dont do it unless you really have to and even then dont do it. more trouble than its worth in my experience.
    – Skeith
    Commented Oct 18, 2011 at 9:34
  • @Skeith - thanks for the tip. I'll keep it in mind.
    – bev
    Commented Oct 18, 2011 at 23:12

Different languages work at different levels of abstraction from the hardware. C and C++ are very low-level. Scripting languages, on the other hand, require you to know less about the underlying detail.

However, I would still say that in all cases, the more you know, the better of a programmer you'll be. Part of programming is being able to juggle multiple levels of abstraction at the same time.

If you're programming in C++, you do need to have a pretty good understanding of how a modern CPU works, at least at the level of abstraction that the compiler works at. (There are things going on inside the CPU that are transparent to the compiler, too).

  • Scott - by " a pretty good understanding of how a modern CPU works.." do you mean how digital logic works (e.g. how karnaugh maps, truth tables, AND/OR/NOR/XOR gates work)? or do you mean what resources the compiler directly uses (i.e. registers)?
    – bev
    Commented Oct 18, 2011 at 0:13
  • Knowing more is good. The real trick, though, is knowing what kind of "more" will give the biggest bang for your buck. Knowing instruction timings, for example, won't be much use when it's near impossible to predict what instructions your compiler will use. Learning how to use a profiler will probably give a much a much better cost/benefit ratio.
    – user8709
    Commented Oct 18, 2011 at 1:55
  • 1
    @bev - No, I don't think you need to get down to the gate level. If you just knew the basic architecture (memory, bus, CPU), how it loads an instruction, executes it, stores the result, etc., you're probably OK. You also need to understand how the compiler lays out a program's memory space including how it uses the stack and the heap. Commented Oct 18, 2011 at 11:17
  • @ScottWhitlock - Thanks - this is just the sort of specific recommendations that I was looking for.
    – bev
    Commented Oct 18, 2011 at 23:16

It is necessary to have an idea of what kind of hardware you are targeting, and how that hardware works if you want to write code which has high performance.

An example for this is coding a Multi-core processor. You need to know Parallel Programming to take maximum advantage of the hardware. To learn parallel programming you need to know the architecture of the machine and basic concepts of how a multi-core processor works.

Having a basic knowledge of the hardware will help you choose the right hardware for your application. For example, you want to create a Machine Learning project: choosing the right hardware will save power and money (some hardware designs will be more efficient for Machine Learning).

You cannot code firmware or device drivers without knowing how the hardware works.

In conclusion, you need to have knowledge of hardware if you want to code for High Performance applications or if you want to code firmware.

Note: In all these cases, you usually won't need to know how the hardware is implemented, you just need to know the characteristics and performance of the particular hardware you are coding on.


It depends on the domain. Are you writing an operating system kernel? Then yes, you need a pretty deep understanding of the underlying hardware level. Are you writing a web application in React? You probably don't need to know much about hardware (after all, you don't even know what what processor or platform your code will run on!).

Rule of thumb: A good programmer should understand one level below the abstraction level you work on. For example, to be able to work with Java you should of course know how the langauage is used, the API's of the libraries you are using and so on. But in order to be really competent, you should have an understanding of what happens at the byte code and memory management level, have looked into the source code of the standard library and so on.

Of course knowing many layers down is great, but the returns are diminishing. If you develop in Python it is great to know C, but it will not (IMHO) help you significantly to also know assembler. But if you are developing in C helps your understanding a lot to have a good understanding of what happens at the assembler level.


I'd like to add a point about the overal design of higher level languages like C.

In general I think it's safe to say that such languages can be viewed as implementing an abstract machine, and indeed that's how Dennis Ritchie himself has described how C works and how the particular design of C's abstract machine has made it a more portable language. As such having some understanding of computer architecture and machine-level functioning, can be extremely helpful in also understanding a language's abstract machine.

DMR's paper Portability of C Programs and the UNIX System is the first I remember to discuss the (abstract) machine model for C.

I think DMR's paper about the history and development of C is also extremely useful in showing how real hardware affects language design, and is perhaps also an exemplar of early programming language design: The Development of the C Language

  • As you're new here you seem to think this is a forum, it most certainly isn't. Your answers to a question shouldn't be a point you add, that should be relegated to comments, and answer should attempt to be a comprehensive answer directly to the question. That said you are making a good point and this is valuable on topic information, perhaps if you could put a few lines in that answer the question directly along with this explanation would be great. Cool info you're sharing here. Welcome to Programmers! Commented Nov 18, 2012 at 6:03
  • comments are not versioned, and not permanent, and so are useless for adding to a set of answers. Most posters are also prone to ignoring use of comments to update their answers, and most answers are not tagged as "community wiki" answers and so cannot be edited by others in such a way as to maintain some attribution to the subsequent contributor(s). Besides, this particular question has started a true discussion, and like it or not that's the way some of these things go. Trying to force every contribution into one mold is a major failing of the stackexchange concept. Commented Nov 18, 2012 at 21:48
  • and, btw, I did actually answer, albiet implicitly, the one true question from the OP: one should have enough of an understanding of hardware to be able to model the abstract machine at the core of a language's design. Commented Nov 18, 2012 at 21:51
  • I just wanted to warn you that your links are dead Commented Aug 17, 2020 at 14:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.