A header normally contains declarations to tell a compiler about the functions (and classes, etc.) that exist in a library.
Most C and C++ compilers include a standard library that they'll link with by default. To link with other libraries, you typically have to tell the compiler (linker, really, at least in most cases) to link with them.
As to why there are so many headers: mostly because there are lots of declarations, many of which are only minimally related to each other. Since a header (normally) contains source code, each time you compile a file that includes a header, you may also be re-compiling the entirety of that header. This can slow down compilation considerably (and compilation speed is often a problem with C and C++ anyway).
Libraries themselves typically come in two forms: one is a "static library". This is basically just a bunch of object files collected together into a single file, often with a directory of some sort appended to make it easier to find which functions are contained in which files. The linker basically just looks through the directory to find files that will satisfy currently-unresolved external references in your code, and links them in like it would any other object file.
The second is a dynamic library (e.g., DLL on Windows, .so on Linux). This is more like an executable--a set of object files that have been pre-linked into a loadable form. When you link against one, the linker inserts some sort of reference to the dynamic library into your executable. When you execute your program, the OS' loader maps the dynamic library to your process' address space, and "fixes up" the references to the functions in the dynamic library, so they refer to the correct addresses in the library.