Is Moore's law (empirically) the same for both memory capacity and processing speed?

Question

Has memory and processing power increased over time around basically in the same rate, or can we say that one or the other has increased more? Has cost decreased in the same rate? And how about the near future?

I am aware this question is not directly related to programming, but I thought it would be better to ask it here for some reasons:

1. I work as a programmer, not a computer science theorist (so I probably would not find it very helpful the kind of question I would receive from the Theoretical Computer Science site, for instance).
2. I prefer answers based upon a programmers real experience.
3. And I am worried about real programming-related issues, specially the functional x imperative debate.

About the 3rd item: currently we've being seeing a lot of arguments about how the increase of using functional programming techniques is related to the now common multi-core machines, and how the free lunch is over [pdf].

But it seems to me that the functional paradigm is much more memory consuming (but this is an opinion, not a fact), and I have not seem many arguments about this future increase in memory needs.

Answer 1

It's important to note that Moore's Law does not talk about processor speeds. It talks about transistor density.

In fact, we hit the wall on clock rates quite a long time ago, and clocks have generally been decreasing since the P4. There have been additional performance gains with stuff like OOO execution and other mechanisms to exploit ILP, but these days the extra density is going towards more cache and more cores, rather than straightforwardly boosting the speed of each core.

Memory, on the other hand, is much more straightforward to implement - a higher transistor density basically means you can pack more memory cells into the same package, without having to do any complex redesign.

Basically, memory scales exceptionally well with increasing transistor density. Processors, not so much.

Answer 2

Not strictly an answer, but ...

Your question implies that you are wondering whether functional programming will become unavoidable due to limitations in moore's law. Perhaps that's besides the point. The main driver behind new technologies in programming is not performance, or we would all be writing assembly. The main driver is complexity. New programming techniques and technologies become popular because they make it easier to build bigger systems.

In other words, if functional programming takes over, it won't be because it offers better performance, it will be because it lets you build bigger apps faster. And that has nothing to do with moore's law.

Answer 3

So Moore's Law specifically states that the density of transistors will double about every 18 months. What this translates into effectively is faster processors and more RAM. Let's look at it from a few comparisons.

1996 A top of the line desktop had about a 300 MHz Processor, 32 MBs of RAM and a 2GB hard drive. Computers were just becoming powerful enough that the MPEG audio format (MP3) could be decompressed and played back in real time. And people were concerned about the performance of a Virtual Machine based language such as Java.

2011 My phone has a 1GHz Processor 512 MB RAM, 16GB of storage and can play MP3 and practically every other format.

In 15 years, my phone is more powerful than my computer was, not only is Java used for application development, but so is another interpreted platform (.NET). So yes given time, Moore's law will make any thing that is resource constrained today a viable technology.

Also, I've never heard of functional languages as being more memory intensive than OO.

Answer 4

A huge problem today is that memory latency has hardly improved at all. This article estimates that over the last 20 years memory has improved

• size - 128x
• Bandwith - 20x
• Latency - 1.3x

Processors combat this with larger and more advanced caching systems and pre-fetching. And as a result it becomes more important to keep memory access predictable and avoid random access.

Most of the benefit of functional programming is related to making it easier to parallelize computation, and this is clearly an increasing concern since processor frequency has stopped scaling.

I'm not aware of any significant difference regarding memory usage of functional programming. I would think that this is much more related to language design and compiler efficiency than programming paradigm. If performance is important, low level access to the hardware is useful, and functional languages typically does not provide this.

It seem to me that functional techniques has become much more popular, in large part due to increased productivity and easier parallelization. Pure functional languages has remained somewhat uncommon.

Answer 5

Moore's Law was originally about the number of circuit elements that would fit on a chip. The comparison to increased processor speed came later.

Faster and smaller elements benefit both the CPU and memory, so one would expect a roughly parallel increase in CPU and memory capability. Simply as a matter of qualitative impression, it seems to me that they've been increasing more or less together, along with hard disk size. (My first computer with a hard disk was a Mac SE, with an 8 MHz 68000 processor (16/32 bits), 1M of memory, and a 20M disk drive. Everything's increased in my current machines by factors between one and ten thousand.)

Answer 6

As others have stated Moore's law was really about transitor density. Since smaller geometries mean faster switching and data travel times, we saw speed improve along with density and the two were conflated. Several years ago a total power dissapation barrier was reached, faster clockrates are more power hungry, and without exotic cooling mechanisms we couldn't continue to advance them. But transistor budgets have gone on exponentially increasing, that means greater parallelism both within a core, and by replication of more cores has been pursued instead. Also caches become bigger with time, and this helps reduce the CPUspeed to memory speed mismatch penalty somewhat.

Functional programming, or any other technique which trades off programmer complexity for (somewhat) reduced resource usage efficiency is a consequence of the advances in HW capability. The sweetspot in the tradeoff between source code complexity and raw performance changes when machine capabilty increases. Also the capability to afford/contemplate more complex computations, leads to demand for more complex applications, which has an impact on the sweetspot of the tradeoff as well. If we only had kilohertz clocks and small memories, we would probably consider assembly to be the highest level language we would want to consider.