First, that a pointer contains a memory address is an implementation detail. That's not how it's defined in the standard. The standard works very, very hard to be vague with regards to the hardware so that C can run on the weirdest, most exotic architectures around. For example, you can't assume two pointers of different types have the same representation (except for char* and void*), or that pointers to objects have the same representation as pointers to functions.
To address your actual question:
- Memory addresses point to places in memory. Memory is external to the CPU.*
- A process sees memory as a continuous sequence of addressable units (usually 8-bit bytes). So, yes, you can compare memory addresses. As a side note, unless you're running on bare metal, the fact that a process has a continuous segment of memory to work with is just an illusion the OS provides. In reality the memory a process sees is mapped to different ranges of hardware memory and potentially disk space. In other words, memory address 1 for a process doesn't necessarily map to hardware address 1. You don't really need to concern yourself with this unless you want to know how OSs work.
In short, just think of memory as an array of bytes and a memory address as an index into that array.
*Yes, I know CPUs have caches.