49

It's commonly stated that Intel's Itanium 64-bit processor architecture failed because the revolutionary EPIC instruction set was very difficult to write a good compiler for, which meant a lack of good developer tools for IA64, which meant a lack of developers creating programs for the architecture, and so no one wanted to use hardware without much software for it, and so the platform failed, and all for the want of a horseshoe nail good compilers.

But why was the compiler stuff such a difficult technical problem? It seems to me that if the explicit parallelism in EPIC was difficult for compiler vendors to implement... why put that burden on them in the first place? It's not like a good, well-understood solution to this problem didn't already exist: put that burden on Intel instead and give the compiler-writers a simpler target.

Itanium came out in 1997. By this point, the UCSD P-Code bytecode system was nearly 20 years old, the Z-machine just slightly younger, and the JVM was the hot new rising star in the world of programming languages. Is there any reason why Intel didn't specify a "simple Itanium bytecode" language, and provide a tool that converts this bytecode into optimized EPIC code, leveraging their expertise as the folks who designed the system in the first place?

  • 5
    Really-low-level IRs (that are actually specified beyond being internal to one compiler, and intended to be compiled onto specific hardware rather than interpreted portably) are a more recent invention AFAIK. That's not to say they didn't exist at all, but I think the idea was not at all obvious or well-known for quite a while. I mean, most people still associate "bytecode" with "interpreter". – user7043 Apr 16 '15 at 20:38
  • 4
    Assuming this doesn't merely resolve to "what were they thinking," it's a pretty good question. – Robert Harvey Apr 16 '15 at 21:01
  • The P-system was dog slow compared with what native machine code could do. For future processor architectures the strategy you describe might be good now that the JVM has demonstrated that a JIT can achieve general-purpose code performance that's competitive with native code, but I don't think that was clear when IA64 was being developed. Burdening a new supposedly-faster architecture with a slow VM would probably not make buyers very happy. – supercat Apr 16 '15 at 22:19
  • @supercat: I'm not talking about a hypothetical VM, but about a hypothetical IR that would be compiled the rest of the way by an Intel code generator. – Mason Wheeler Apr 16 '15 at 22:22
  • 3
    I remember discussing this specific question in my graduate Computer Architecture class years ago. There were specific reasons why Intel did what they did, unfortunately I cannot dig up any definitive resources to provide an answer. – user22815 Apr 16 '15 at 23:44
33

As I recall at the time, the issue was not just the particulars of IA64, it was the competition with AMD's x86-64 instruction set. By making their architecture backwards compatible with the x86 instruction set, AMD was able to leverage the existing tools and developer skill sets. AMD's move was so successful that Intel (and Via) were essentially forced to adopt the x86-64 architecture.

The big barrier at the time was 4 GB RAM on desktop PCs (more realistically ~3.4GB usable on Windows). x86-64 smashed that barrier and opened up higher power computing to everyone. Had AMD never come up with x86-64, I'm sure Intel would have been happy to have everyone who wanted to jump to 4GB+ RAM pay a hefty premium for years for that privilege. Demonstrating how slowly markets move, it has taken years for applications to catch up to 64-bit, multi-threaded programming, and even now 4GB RAM is standard on low-end PCs.

In short, Intel tried to make a revolutionary leap with the IA64 architecture, and AMD made an evolutionary step with x86-64. In an established market, evolutionary steps that allow knowledge workers to leverage existing skills will win over revolutionary steps that require everyone to learn new skills. Regardless of the qualitative differences between the architectures, IA64 could not overcome the momentum of its own x86 platform once AMD added the x86-64 extensions.

I don't buy the explanation that IA64 was too difficult to program for. It was only difficult relative to the alternatives. @delnan's point about low-level IR is smack on, I just don't think it would have made a difference.

As to why Intel didn't try to shoulder that burden themselves, who knows? They were the market power at the time. AMD was something of a threat but Intel was the king of the hill. Maybe they thought that IA64 would be so much better than anything else that they could move the entire market. Maybe they were trying to make a premium tier and leave AMD, VIA, etc. in the second tier fighting over low-margin commodity hardware - a strategy that both Intel and Apple have employed quite successfully.

Was Itanium a deliberate attempt to make a premium platform and pull the rug out from under AMD, VIA, etc.? Of course, that's how business works.

  • 4
    All very interesting, but you mostly explain why Itanium failed, whereas the question was about Intel's strategy in pushing Itanium. There is a hint in "Intel would have been happy to have everyone [...]" but it's not clear to me if you're implying whether this was a deliberate decision by Intel (and if so, what you have to support this assertion). – user7043 Apr 16 '15 at 21:36
  • 2
    Great points. As a former compiler writer, it's true that being able to take an existing compiler back and tweak it for performance is better than writing one all over again. Back then (and maybe now... not sure) writing a compiler back-end was something a team of 4 or 5 devs could do in a year. That's a tough nut to crack when nobody has adopted the hardware. We chose at the time instead to build PowerPC back ends to support the flavors of Unix boxes that were being built on it. – Chris Steele Apr 16 '15 at 21:45
  • @delnan, good point, I have added commentary to address the other questions. – Robert Munn Apr 16 '15 at 22:43
  • 2
    More succinctly, Intel vastly underestimated the inertia from those wearing the yoke of backward compatibility. AMD beat Intel at its own game by taking the same evolutionary step from the x86 family that the x86 family did from the 8086/8088 family. – Blrfl Apr 17 '15 at 11:28
  • 1
    Erm. 80x86 has supported 36-bit physical addressing (or a limit of "not quite 64 GiB of RAM") since the introduction of PAE and PSE36 in about 1995. The problem was very few versions of Windows supported PAE due to device driver incompatibilities (but some did). – Brendan Apr 26 '15 at 11:50
31

The Wikipedia article on EPIC has already outlined the many perils common to VLIW and EPIC.

If anyone does not catch the sense of fatalism from that article, let me highlight this:

Load responses from a memory hierarchy which includes CPU caches and DRAM do not have a deterministic delay.

In other words, any hardware design that fails to cope with (*) the non-deterministic latency from memory access will just become a spectacular failure.

(*) By "cope with", it is necessary to achieve reasonably good execution performance (in other words, "cost-competitive"), which necessitates not letting the CPU fall idle for tens to hundreds of cycles ever so often.

Note that the coping strategy employed by EPIC (mentioned in the Wikipedia article linked above) does not actually solve the issue. It merely says that the burden of indicating data dependency now falls on the compiler. That's fine; the compiler already has that information, so it is straightforward for the compiler to comply. The problem is that the CPU is still going to idle for tens to hundreds of cycles over a memory access. In other words, it externalizes a secondary responsibility, while still failing to cope with the primary responsibility.

The question can be rephrased as: "Given a hardware platform that is destined to be a failure, why (1) didn't (2) couldn't the compiler writers make a heroic effort to redeem it?"

I hope my rephrasing will make the answer to that question obvious.


There is a second aspect of the failure which is also fatal.

The coping strategies (mentioned in the same article) assumes that software-based prefetching can be used to recover at least part of the performance loss due to non-deterministic latency from memory access.

In reality, prefetching is only profitable if you are performing streaming operations (reading memory in a sequential, or highly predictable manner).

(That said, if your code makes frequent access to some localized memory areas, caching will help.)

However, most general-purpose software must make plenty of random memory accesses. If we consider the following steps:

  • Calculate the address, and then
  • Read the value, and then
  • Use it in some calculations

For most general-purpose software, these three must be executed in quick succession. In other words, it is not always possible (within the confines of software logic) to calculate the address up front, or to find enough work to do to fill up the stalls between these three steps.

To help explain why it is not always possible to find enough work to fill up the stalls, here is how one could visualize it.

  • Let's say, to effectively hide the stalls, we need to fill up 100 instructions which do not depend on memory (so will not suffer from additional latency).
  • Now, as a programmer, please load up any software of your choice into a disassembler. Choose a random function for analysis.
  • Can you identify anywhere a sequence of 100 instructions (*) which are exclusively free of memory accesses?

(*) If we could ever make NOP do useful work ...


Modern CPUs try to cope with the same using dynamic information - by concurrently tracking the progress of each instruction as they circulate through the pipelines. As I mentioned above, part of that dynamic information is due to non-deterministic memory latency, therefore it cannot be predicted to any degree of accuracy by compilers. In general, there is simply not enough information available at the compile-time to make decisions that could possibly fill up those stalls.


In response to the answer by AProgrammer

It is not that "compiler ... extracting parallelism is hard".

Reordering of memory and arithmetic instructions by modern compilers is the evidence that it has no problem identifying operations that are independently and thus concurrently executable.

The main problem is that non-deterministic memory latency means that whatever "instruction pairing" one has encoded for the VLIW/EPIC processor will end up being stalled by memory access.

Optimizing instructions that do not stall (register-only, arithmetic) will not help with the performance issues caused by instructions that are very likely to stall (memory access).

It is an example of failure to apply the 80-20 rule of optimization: Optimizing things that are already fast will not meaningfully improve overall performance, unless the slower things are also being optimized.


In response to answer by Basile Starynkevitch

It is not "... (whatever) is hard", it is that EPIC is unsuitable for any platform that has to cope with high dynamism in latency.

For example, if a processor has all of the following:

  • No direct memory access;
    • Any memory access (read or write) has to be scheduled by DMA transfer;
  • Every instruction has the same execution latency;
  • In-order execution;
  • Wide / vectorized execution units;

Then VLIW/EPIC will be a good fit.

Where does one find such processors? DSP. And this is where VLIW has flourished.


In hindsight, the failure of Itanium (and the continued pouring of R&D effort into a failure, despite obvious evidence) is an example of organizational failure, and deserves to be studied in depth.

Granted, the vendor's other ventures, such as hyperthreading, SIMD, etc., appears to be highly successful. It is possible that the investment in Itanium may have had an enriching effect on the skills of its engineers, which may have enabled them to create the next generation of successful technology.

7

TL;DR: 1/ there are other aspects in the failure of Itanium than the compiler issues and they may very well be enough to explain it; 2/ a byte code would not have solved the compiler issues.

It's commonly stated that Intel's Itanium 64-bit processor architecture failed because the revolutionary EPIC instruction set was very difficult to write a good compiler for

Well, they were also late (planned for 98, first shipment in 2001) and when they finally delivered the hardware, I'm not even sure that it delivered what was promised for the earlier date (IIRC, they at least dropped part of the x86 emulation which was initially planned), so I'm not sure that even if the compilation problems has been solved (and AFAIK, it has not yet), they would have succeeded. The compiler aspect was not the only aspect which was overly ambitious.

Is there any reason why Intel didn't specify a "simple Itanium bytecode" language, and provide a tool that converts this bytecode into optimized EPIC code, leveraging their expertise as the folks who designed the system in the first place?

I'm not sure where you place the tool.

If it is in the processor, you have just another micro-architecture and there is no reason not to use x86 as public ISA (at least for Intel, the incompatibility has an higher cost than whatever could bring a cleaner public ISA).

If it is externally, starting from a byte-code make it even harder than starting from an higher level language. The issue with EPIC is that it can use only the parallelism that a compiler can find, and extracting that parallelism is hard. Knowing the language rules give you more possibilities than if you are constrained by something already scheduled. My (admitted unreliable and from someone who followed that from far) recollection is that what HP(*) and Intel failed to achieve on the compiler front is the language level extraction of parallelism, not the low level which would have been present in a byte code.

You are perhaps underestimating the cost at which current processor achieve their performance. OOO is more effective than the other possibilities, but it is surely not efficient. EPIC wanted to use the area budget used by the implementation of OOO to provide more raw computing, hoping that compilers would be able to make use of it. As written above, not only we are still unable -- as AFAIK, even in theory -- to write compilers which have that ability, but the Itanium got enough other hard-to-implement features that it was late and its raw power was not even competitive (excepted perhaps in some niche markets with lots of FP computation) with the other high end processor when it got out of fab.


(*) You also seem to underestimate HP role in EPIC.

  • I updated my answer in response to one of your claim. In my opinion, failure to cope with memory latency is the sole cause of death of EPIC architecture. Compilers have decent success at extracting instruction-level parallelism, as are modern CPU hardware. – rwong Apr 17 '15 at 11:43
  • 1
    @rwong, I made a TLDR of what I consider my main points. BTW, for me variable latency -- between models, data dependent for some instructions in some model, memory access is obviously a major category here -- is one aspect of the difficulty of parallelism extraction. CPU hardware has the advantage of dynamic scheduling, and I don't think there is an example of statically scheduled processor which is competitive on pure performance for single thread with OOO. I don't think even the Mill team make that claim (their merit factor include power). – AProgrammer Apr 17 '15 at 12:31
6

A few things.

IPF was in-order, for one. This meant you couldn't rely on reorder to save you in the event of a cache miss or other long-running event. As a result, you ended up needing to rely on speculative features - namely, speculative loads (loads that were allowed to fail - useful if you didn't know if you'd need a load result) and advanced loads (loads that could be re-run, using recovery code, if a hazard occurred.) Getting these right was hard, advanced loads especially! There were also branch and cache prefetch hints that could really only be used intelligently by an assembly programmer or using profile-guided optimization, not generally with a traditional compiler.

Other machines at the time - namely UltraSPARC - were in-order, but IPF had other considerations too. One was encoding space. Itanium instructions were, by nature, not especially dense - a 128-bit bundle contained three operations and a 5-bit template field, which described the operations in the bundle, and whether they could all issue together. This made for an effective 42.6 bit operation size - compare to 32 bits for most of the commercial RISCs' operations at the time. (This was before Thumb2, et al - RISC still meant fixed-length rigidity.) Even worse, you didn't always have enough ILP to fit the template you were using - so you'd have to NOP-pad to fill out the template or the bundle. This, combined with the existing relative low density, meant that getting a decent i-cache hit rate was a) really important, and b) hard - especially since I2 only had a 16KB L1I (although it was quite fast.)

While i've always felt that the argument of "the compiler was the one and only problem" was overblown - there were legitimate microarchitectural issues that really did I2 no favors for general-purpose code - it was not especially fun to generate code for compared to the narrower, higher-clocked OoO machines of the day. When you could really properly fill it, which often involved either PGO or hand-coding, it did great - but a lot of the time, performance from compilers was really just uninspiring. IPF didn't make it easy to generate great code, and it was unforgiving when code wasn't great.

4

But why was the compiler stuff such a difficult technical problem? It seems to me that if the explicit parallelism in EPIC was difficult for compiler vendors to implement... why put that burden on them in the first place? It's not like a good, well-understood solution to this problem didn't already exist: put that burden on Intel instead and give the compiler-writers a simpler target.

What you describes is a bit what Transmeta tried to do with their code morphing software (which was dynamically translating x86 "bytecode" into Transmeta internal machine code).

As to why did Intel failed to make a good enough compiler for IA64... I guess is that they did not have enough compiler expertise in house (even if of course they did have some very good compiler experts inside, but probably not enough to make a critical mass). I guess that their management underestimated the efforts needed to make a compiler.

AFAIK, Intel EPIC failed because compilation for EPIC is really hard, and also because when compiler technology slowly and gradually improved, other competitors where also able to improve their compiler (e.g. for AMD64), sharing some compiler know-how.

BTW, I wished that AMD64 would have been some more RISCy instruction set. It could have been some POWERPC64 (but it probably wasn't because of patent issues, because of Microsoft demands at that time, etc...). The x86-64 instruction set architecture is really not a "very good" architecture for compiler writer (but it is somehow "good enough").

Also the IA64 architecture has builtin some strong limitations, e.g. the 3 instructions/word have been good as long as the processor had 3 functional units to process them, but once Intel went to newer IA64 chips they added more functional units, and the instruction-level parallelism was once again hard to achieve.

Perhaps RISC-V (which is an open source ISA) will gradually succeed enough to make it competitive to other processors.

  • Intel spend billions on R&D, I have a hard time believing they would have a difficult time developing a good compiler for a new hardware platform. – user22815 Apr 17 '15 at 11:30
  • 1
    Money is not everything: see the mythical man month, no silver bullet and consider also that time to market is very significant. – Basile Starynkevitch Apr 17 '15 at 11:32
  • 3
    They employ many talented engineers and computer scientists. Their non-VLIW compilers are top-notch, regularly pumping out code much faster than other compilers. Intel are probably the one company that have more compiler expertise in-house than any other company. Intel succeed at everything else they do: why was Itanium an albatross? – user22815 Apr 17 '15 at 11:43
  • 1
    It probably was a bit less true in 1997. And as several explained, EPIC compilation is really hard. – Basile Starynkevitch Apr 17 '15 at 11:44
3

As Robert Munn pointed out -- it was the lack of backward compatibility that killed the Itanium ( and many other "new" technologies).

While writing a new compiler might have been hard you only need a few of them. A C compiler which produces optimized code is a must -- otherwise you will not have a useable Operating System. You need a C++ compiler, Java and given that the main user base would be Windows some sort of Visual Basic. So this was not really a problem. There was a decent operating system (NT) and a good C compiler available.

What would seem like a trivial effort for a company offering a software product -- recompile and retest your C code base (and at that time most would have been written in pure C!) was not that simple; converting a large set of C programs which assumed a 32 bit integer and assumed 32 bit addressing to a native 64 bit architecture was full of pitfalls. Had IA64 become a dominant chip (or even a popular one!) most software companies would have bitten the bullet and made the effort.

So fast chip with a reasonable OS but a very limited set of software available, therefore not many people bought it, therefore not many software companies provided products for it.

3

What killed Itanium was shipment delays that opened the door for AMD64 to step in before software vendors commited to migrate to IA64 for 64 bit apps.

Leaving optimization to the compiler was a good idea. A lot of stuff can be done static that otherwise is inefficient in hardware. The compilers became quite good at it, especially when using PGO profiling (I worked at HP and HP's compiler tended to outperform Intel's). PGO was a hard sell however, it's a difficult process for production code.

IPF was meant to be backwards compatible, but once AMD64 launched it became moot, the battle was lost and I believe the X86 hardware in the CPU was just stripped to retarget as a server CPU. Itanium as an architecture was not bad, the 3 instruction per word was not an issue. What was an issue is the hyper-threading implementation by swapping stacks during memory IO was too slow (to empty and reload the pipeline) until Montecito etc. which prevented it from competing vs out-of-order PowerPC CPUs. The compilers had to patch up late-to-detect flaws of CPU implementations, and some of the performance edge was lost to hard to predict mistakes.

The architecture allowed Itanium to be relatively simple while providing tools for the compiler to eek out performance from it. If the platform had lived, the CPUs would have become more complex, and eventually become threaded, out of order etc. like x86. However the first gens focussed transistor count on other performance schemes since the compiler handled a lot of the hard stuff.

The IPF platform bet on the compiler and tools, and it was the first archiecture to expose an extremely complete and powerful Performance Monitoring Unit (PMU) design, that was later ported back to Intel x86. So powerful tool developers still don't use it to its full ability to profile code.

If you look at ISA successes, it's often not the technical side that rolls the dice. It's its place in time and market forces. Look at SGI Mips, DEC Alpha... Itanium was just supported by the loosers, SGI & HP servers, companies with managements that piled on strategic business mistakes. Microsoft was never full-in and embraced AMD64 to not be boxed-in with only Intel as a player, and Intel didn't play right with AMD to give them a way to live in the ecosystem, as they intended to snuff AMD.

If you look at where we are today, X86's complex hardware has lead it to an evolution dead end so far. We're stuck at 3+GHz, and dumping cores with not enough use for it. Itanium's simpler design would have pushed more stuff on the compiler (room for growth), allowing to build thinner,faster pipelines. At same generation and fab technology, it would have been running faster and capped all the same but a bit higher, with maybe other doors to open to push Moore's law.

Well at least the above is my beliefs :)

1

Memory is getting vague... Itanium had some great ideas that would need great compiler support. The problem was it wasn't one feature, it was many. Each one wasn't a big deal, all together were.

For example, there was a looping feature where one iteration of the loop would operate on registers from different iterations. x86 handles the same problem through massive out-of-order capability.

At that time Java and JVMs were in fashion. What IBM said was that with PowerPC, you could compile bytecode quickly and the CPU would make it fast. Not on Itanium.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.