From Instruction pipelining section in Tanenbaum's Structured Computer Organization:

Pipelining allows a trade-off between latency (how long it takes to execute an instruction), and processor bandwidth (how many MIPS the CPU has).

With a cycle time of T nsec, and n stages in the pipeline, the latency is nT nsec because each instruction passes through n stages, each of which takes T nsec.

Since one instruction completes every clock cycle and there are 10^9/T clock cycles/second, the number of instructions executed per second is 10^9/T. To get the number of MIPS, we have to divide the instruction execution rate by 1 million to get (10^9/T)/10^6 = 1000/T MIPS.

I think the smaller latency the better, and the bigger bandwidth the better. Since the two are related as reciprocals, they always move simultaneously in either both good or both bad directions. Then how do the two form tradeoff? Thanks.


The short answer is they aren't related as reciprocals.

Assume you have a very boring CPU with one instruction that takes 1 ns to complete. Therefore your latency is 1 ns, and your bandwidth is 1 instruction per ns.

Say you add a pipeline stage to split that instruction into fetch and execute phases. Each of those stages might be able to complete in 0.5 ns, so now you can increase your clock speed, but you can't double it, because you've added the overhead of the pipeline logic. Also, stages all have to use the same clock, even if one is faster than another. So maybe you can do a 0.6 ns clock. Your latency is now increased to 1.2 ns, but your bandwidth has also increased to 1.7 instructions per ns.

The relationship isn't linear, especially when you are already close to the limits of your semiconductor technology, but latency and bandwidth both increase when pipeline stages are added, up to a certain point.

| improve this answer | |
  • Thanks! (1) in pipeline, a stage must take a clock cycle? Must a cpu finish only one stage in a clock cycle? (2) without pipeline, does an instruction also take a clock cycle, and a cpu must finish no more than one instruction in a clock cycle? – Tim Feb 12 '15 at 4:18
  • 2
    The clock is when all the data gets shifted to the next stage, so if a stage gets done early, there's no way for the next stage to start early, and if a stage gets done late, the next stage won't get its data at all. So the clock has to be set longer than the slowest stage. The same principle applies for no pipeline, except the next "stage" is memory or a register or something. – Karl Bielefeldt Feb 12 '15 at 4:44
  • Thanks. Karl. Without pipeline, what do you mean by "The same principle applies for no pipeline, except the next "stage" is memory or a register or something"? Does an instruction always take exactly one clock cycle, regardless if it finishes earlier? (In which topic can I find the answer in a textbook on computer architectures?) – Tim Feb 12 '15 at 12:20
  • The whole reason you have a clock is to mark the time when one part of the chip is ready to hand off its data to the next part, and the next part is ready to accept. That might be more of a digital design topic than a computer architecture one. It's been a while since school for me. That means an instruction can potentially take more than one clock cycle, but it always passes off its data on a clock boundary. If it finishes early, it has no way to know if the next part of the chip is ready to accept it. – Karl Bielefeldt Feb 12 '15 at 13:11
  • 1
    There is another source of overhead: the cycle time of a pipeline is the one of the slowest stage. While designers try to equalize the stages, that is not always possible and thus some stages have some unused slack. – AProgrammer Feb 12 '15 at 15:33

They aren't reciprocals. Those equations don't give a complete relationship between latency, MIPS and pipeline stages; in that excerpt they're only being used to define what he means by "latency" and "MIPS" so that his claim about the tradeoff is more precise.

The tradeoff Tanenbaum's referring to is that when you split the CPU's pipeline into more stages, you can execute more instructions in parallel (more bandwidth), but each instruction takes longer to finish because of the added complexity (more latency).

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.