Just went over some slides and noticed that the L1 cache (at least on Intel CPUs) distinguishes between data and instruction cache, I would like to know why this is..
-
1en.wikipedia.org/wiki/Harvard_architecture– MchlFeb 6, 2011 at 17:06
Add a comment
|
3 Answers
There are actually several reasons.
First and probably foremost, the data that's stored in the instruction cache is generally somewhat different than what's stored in the data cache -- along with the instructions themselves, there are annotations for things like where the next instruction starts, to help out the decoders. Some processors (E.g., Netburst, some SPARCs) use a "trace cache", which stores the result of decoding an instruction rather than storing the original instruction in its encoded form.
Second, it simplifies circuitry a bit -- the data cache has to deal with reads and writes, but the instruction cache only deals with reads. (This is part of why self-modifying code is so expensive -- instead of directly overwriting the data in the instruction cache, the write goes through the data cache to the L2 cache, and then the line in the instruction cache is invalidated and re-loaded from L2).
Third, it increases bandwidth: most modern processors can read data from the instruction cache and the data cache simultaneously. Most also have queues at the "entrance" to the cache, so they can actually do two reads and one write in any given cycle.
Fourth, it can save power. While you need to maintain power to the memory cells themselves to maintain their contents, some processors can/do power down some of the associated circuitry (decoders and such) when they're not being used. With separate caches, they can power up these circuits separately for instructions and data, increasing the chances of a circuit remaining un-powered during any given cycle (I'm not sure any x86 processors do this -- AFAIK, it's more of an ARM thing).
answered Feb 6, 2011 at 17:16
-
4It is also important to mention that code and data may exhibit different access patterns; for instance, the instructions for summing all elements in an array exhibit temporal locality (same instructions are used often (if you do it by a loop)) and the data in the array exhibit spacial locality (the following data is used next).– gablinFeb 6, 2011 at 19:52
-
2@gablin: while true, those differences in patterns would often favor a unified cache. In a tight loop like you mention, most of the instruction cache is sitting idle. A unified cache would basically double the size of the data cache for the duration of the loop. Feb 6, 2011 at 20:06
-
Not really, because there's more code after that small loop and that's also likely to be working with the array. That characterizes an awful lot of code (e.g., string handling). In fact, the first caches in CPUs were unified caches – they sat between the CPU's principal memory interface and the external bus, which was a simple place to put them – but we now use a partitioned cache because it is faster in practice. Feb 6, 2011 at 22:23
-
@Donal Fellows: Yes, really. I'm well aware of how early caching was done, and why they changed to a split cache. Feb 6, 2011 at 23:02
Just like real estate, cache use is driven by three things: location, location, location. The whole point of having a cache is that most programs exhibit patterns of location: if they access byte 1111111, then the next byte they will access is probably 1111110 or 1111112, and not so much byte 9999999. However, most programs will exhibit very different patterns of location for their instructions and their data. This means that is would be unlikely for the instructions and data to be able to share the cache efficiently. Because instructions and data aren't necessarily near each other in memory. A data access would bump instruction from the cache, and loading instructions would bump data from the cache.
answered Feb 6, 2011 at 18:55
You need to implement massive amounts of bandwidth between L1 cache and CPU. Obviously, whatever bandwidth you have, you’d love twice as much. But doubling the bandwidth is expensive - much more than twice as expensive as having two caches.
So to increase the cache bandwidth, you just have one L1 data cache and one L1 CPU cache. It’s cheaper than one cache with double the bandwidth. And they need to deliver data to different places: The data cache communicates mostly with registers, the instruction cache with the instruction decoder. So both get put closer to the place where they need to be, saving money and time.
As mentioned elsewhere the instruction cache doesn’t need to process writes. It also doesn’t need to support prediction. And finally, you can optimise their sizes independently.
All in all, two separate caches give you much better value for money.
answered Apr 2, 2022 at 18:32
