It's sort of difficult to work out what you already know and what you need to know, but I'll have a go anyway. Sorry if I patronise you.
The Intel Instruction Set Architecture is extremely large and complicated, for various historical reasons. Fortunately, you only actually need to support a very small subset of the instruction set to be able to target a general purpose compiler at it. That said it may be easier to target a different architecture, at least at first, because it is still a considerable undertaking.
There exist virtual architectures specifically designed to be good for compilers to target, such as Common Intermediate Language, the Java VM, or LLVM. The first two run on a virtual machine with well defined object formats, and the third can be translated into machine code for many platforms using different back ends.
Targeting a RISC processor such as ARM or MIPS would be much easier than x86 too, as they have smaller, better designed and more uniform instruction sets. Because of their ubiquity in mobile and embedded computing, emulators exist for these processors which would let you test the machine code your assembler outputs. Another idea rather than jumping in at the deep end would be to target an old, obsolete architecture, as these tend to be smaller and well documented. Again, emulators exist to let you test your code.
Or if you just want a hard thinking challenge rather than sheer volume of work challenge, you could try making a compiler target something esoteric like the One Instruction Set Computer...
Basic assembler operation
Since you've already got high level languages working, I'm going to assume you can parse and so on. An assembler usually runs in two passes—first it translates most of the code, leaving blanks for the symbols it doesn't know yet. After the first pass, it should know all the symbols, and it runs over the object file again and fills in the addresses of the symbols.
Typically one object file is generated for each assembler file. Any symbols defined in other files are therefore listed in a symbol table in the object header, and the linker (often a separate program) uses this information to stitch the objects together into an executable afterwards.
You asked about sections. In x86 (and other architectures) the memory is segmented to allow the amount of addressable memory to be much bigger than the actual amount of physical memory. A program has a number of segments assigned to it by the operating system. The stack gets its own segment, so that if you push more onto the stack than you've got space for, you don't start overwriting stuff. Strings and other constants defined in your assembler file typically get put into the
.rodata segment. The
.bss segment contains enough space for the uninitialised data in your application. The actual code goes in the
.text segment, and the operating system and hardware usually protects against writing in this segment to prevent various kinds of exploits. The operating system is in charge of loading these segments out the object file and putting them in different places in virtual and physical memory.
If a program is sitting about in the background, then operating system may choose to remove some of the (lesser used) pages from physical memory and write them to disk, in order to make more space for other programs. A memory access to those addresses then causes the processor to interrupt, and the operating system loads up the page from disk again and lets the program continue unaware that anything happened.
The Dragon Book is often considered 'the bible' on compiler development. osdev.org has some useful information on ELF format. Since you're interested in this sort of stuff, and since an appreciation of operating system structures is required once you start getting into things like object file formats, Operating Systems: Design and Implementation is like the bible on operating systems.
Oops, got a bit carried away...