Just went over some slides and noticed that the L1 cache (at least on Intel CPUs) distinguishes between data and instruction cache, I would like to know why this is..


There are actually several reasons.

First and probably foremost, the data that's stored in the instruction cache is generally somewhat different than what's stored in the data cache -- along with the instructions themselves, there are annotations for things like where the next instruction starts, to help out the decoders. Some processors (E.g., Netburst, some SPARCs) use a "trace cache", which stores the result of decoding an instruction rather than storing the original instruction in its encoded form.

Second, it simplifies circuitry a bit -- the data cache has to deal with reads and writes, but the instruction cache only deals with reads. (This is part of why self-modifying code is so expensive -- instead of directly overwriting the data in the instruction cache, the write goes through the data cache to the L2 cache, and then the line in the instruction cache is invalidated and re-loaded from L2).

Third, it increases bandwidth: most modern processors can read data from the instruction cache and the data cache simultaneously. Most also have queues at the "entrance" to the cache, so they can actually do two reads and one write in any given cycle.

Fourth, it can save power. While you need to maintain power to the memory cells themselves to maintain their contents, some processors can/do power down some of the associated circuitry (decoders and such) when they're not being used. With separate caches, they can power up these circuits separately for instructions and data, increasing the chances of a circuit remaining un-powered during any given cycle (I'm not sure any x86 processors do this -- AFAIK, it's more of an ARM thing).

| improve this answer | |
  • 3
    It is also important to mention that code and data may exhibit different access patterns; for instance, the instructions for summing all elements in an array exhibit temporal locality (same instructions are used often (if you do it by a loop)) and the data in the array exhibit spacial locality (the following data is used next). – gablin Feb 6 '11 at 19:52
  • 1
    @gablin: while true, those differences in patterns would often favor a unified cache. In a tight loop like you mention, most of the instruction cache is sitting idle. A unified cache would basically double the size of the data cache for the duration of the loop. – Jerry Coffin Feb 6 '11 at 20:06
  • Not really, because there's more code after that small loop and that's also likely to be working with the array. That characterizes an awful lot of code (e.g., string handling). In fact, the first caches in CPUs were unified caches – they sat between the CPU's principal memory interface and the external bus, which was a simple place to put them – but we now use a partitioned cache because it is faster in practice. – Donal Fellows Feb 6 '11 at 22:23
  • @Donal Fellows: Yes, really. I'm well aware of how early caching was done, and why they changed to a split cache. – Jerry Coffin Feb 6 '11 at 23:02

Just like real estate, cache use is driven by three things: location, location, location. The whole point of having a cache is that most programs exhibit patterns of location: if they access byte 1111111, then the next byte they will access is probably 1111110 or 1111112, and not so much byte 9999999. However, most programs will exhibit very different patterns of location for their instructions and their data. This means that is would be unlikely for the instructions and data to be able to share the cache efficiently. Because instructions and data aren't necessarily near each other in memory. A data access would bump instruction from the cache, and loading instructions would bump data from the cache.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.