LLVM is lower level than typical VMs like JVM and CLR. For example, while it has hooks for a garbage collector, it does not provide a garbage collector itself.
Likewise, the JVM has a built-in JIT compiler (except in really ancient versions). LLVM has some JIT compilers for LLVM IR, but it's still up to the developer to hook things together and actually JIT the code.
When the JVM encounters an unresolved external, it goes out and finds the right class to satisfy it, and knows how to look for .class files directly in the file system and in .jar files1. LLVM's JIT compilers have hooks where you can decide how things like that are handled. As you'd expect, some people have written some default versions, so it can do things roughly on the same order as the JVM can -- but you're also free to ignore those and do things differently if you choose.
Simply put, if you're developing a compiler (or something on that order) it has a lot of tools to make your life easy. Instead of worrying much about optimization, you can do roughly the simplest translation you can manage from your source code to LLVM IR, and then use the LLVM libraries to manage optimizing, JITing, linking, etc. Nonetheless, they are libraries -- it provides some really useful functions so you don't have to deal with all the details, but they're still functions and you're still writing code to invoke them. It's not a finished product, just useful tools for building products relatively quickly and easily.
1 Technically, not all of this is built into the JVM proper. It specifies what's usually called the primordial class loader as part of the JVM proper, and then there are user class loaders specified in java.util.ClassLoader that handle other things. Some class loaders are included by default, and if you want to badly enough, you can supplement those by defining your own.