It all comes down to the cost of fast memory.
When you consider how expensive is the RAM, it starts making sense to use it for a small amount of data you need right now, and keep everything else you don't need that much on a cheaper medium. Often, there are even more levels than that:
- RAM for a small subset of data you need to read as fast as possible,
- Local SSD for a larger subset of data you would like to be able to read fast enough,
- Local hard disks, directly attached storage or NAS for the data you may need to read in a reasonable time,
- Tapes for data you are expected to read only in exceptional circumstances.
This applies equally well to home PCs, data closets, even cloud storage. Compare the price of a AWS server with a hundred gigabytes of RAM with a hundred gigabytes of S3 storage and a hundred gigabytes of Glacier storage. If you want S3 reliability but at a speed of a local RAM, you should expect the price to be accordingly high.
The good thing is that there are specific patterns which are designed to work with this topology (i.e. small but hugely fast non-persistent data medium combined with increasingly large, persistent, slower mediums). Cache is one of them: instead of having to juggle with data, trying to guess which one should be put in RAM, you simply delegate this job to a caching solution, picking the right caching approach. And when it comes to caching approaches, you have a large choice.
Aside the cases handled by proper caching, there are situations where you need high speed access to perform a task (an example which comes to mind is the restoration of a SVN repository from a backup, which is extremely dependent on the speed of storage medium). In most of those situations, persistent aspect is welcome, but not required: for instance, if the server restarts while I was recovering a SVN repository, I can always do the operation again (or do it in parallel on multiple servers if it's worth the money).
Finally, there are situations (scientific analysis, statistical data processing) where it would be great to have huge amounts of very fast data mediums. Persistent aspect is usually irrelevant in those cases (with map reduce, you just redo the job of a machine which terminated unexpectedly), and huge costs of any fast medium usually force to fallback to ordinary SAN solutions.