153

You'd often see that JavaScript is actually being transported over the web with all the useless stuff that doesn't need to be there -- Comments, particularly those containing licenses, indentations ('\t', '\n'), etc. Given enough time, it could end up wasting terabytes of data worldwide! Would a JavaScript bytecode cause another bigger problem, or has nobody thought of this yet?

  • 110
    You don't have to compile to bytecode to get rid of comments and whitespace, there are plenty of uglification/minification tools, but before WebAssembly there was no consistent low-level representation to ship, your users are running different browsers on different operating systems – jonrsharpe Dec 9 '19 at 12:04
  • 30
    @jonrsharpe, that could have been an answer. – Bart van Ingen Schenau Dec 9 '19 at 12:29
  • 11
    So you can view source, if the author intends. Not seen much in practice any more, due to minification etc, but one of the most brilliant features of the web. – davnicwil Dec 9 '19 at 22:32
  • 14
    Source script (yes, even with comments) transported over a compressed channel (like gzip, brotli or deflate) tend to be much smaller than bytecode or bytecode over a compressed channel (yes, it's counter-intuitive). Basically HTTP negates benefits of data being binary. Observe the popularity of JSON and XML over binary packet formats like ASN.1, protobuf and bencoding – slebetman Dec 10 '19 at 8:17
  • 10
    All content sent over network is usually gzipped. That means all repetitive things like whitespace and variable names is reduced in size drastically. – Tomáš Zato - Reinstate Monica Dec 10 '19 at 12:14
380

Why is JavaScript not compiled to bytecode before sending over the network?

Background: I was on the ECMAScript technical committee in the late 1990s and one of the implementers of Microsoft's JScript engine.

Let me begin by saying what I always say when faced with a "why not?" question: language designers are not required to give good reasons why they did not spend hundreds of millions of other people's dollars on a feature that someone happens to like. Rather, the person pitching the feature is required to give good reasons why that's the best way to spend that time, effort and money. You've made an argument with no numbers attached to it that bytecode would be a cost savings in terms of bandwidth. I would encourage you to work up some actual numbers, and compare that to the costs of creating yet another language; those costs are significant. Remember in your analysis that "implementation" is one of the smallest costs. Also in your analysis include who saves the money vs who spends the money, and you will find that the people spending the money are not the ones saving it; incentives matter.

That said, this is one of the more reasonable "why not?" questions because it is a feature we considered and rejected for reasons.

We considered such a scheme, both within Microsoft and at the TC level; since JScript was already implemented as compiling to a well-designed, principled bytecode language, it would have been straightforward for us to propose it as a standard and we considered doing so.

We decided not to, for a variety of reasons including:

  • Holy goodness it was hard enough to standardize JavaScript. Everyone and their dog would have an opinion about what the ideal characteristics of a bytecode language were, and it would be multiple years of bikeshedding. No one really wanted to go there.
  • It was an expensive solution with no associated costly problem. There's no reason to suppose that a bytecode language would be more efficient in either size or speed. JavaScript already minifies reasonably well and is highly compressible.
  • It would have created an enormous amount of work for browser providers, who were already vexed by the expense of producing an efficient, compliant JS implementation.
  • Creating a secure JS implementation that resists attacks by bad actors is hard enough; should we double the surface area available to attack? Probably not.
  • Standards are an impediment to innovation. If we discovered that a small change to our bytecode language would make a big difference in some previously-unforeseen or previously-unimportant user scenario, we were free to make that change. If it was a standard, we would not be free to create that user benefit.

But that analysis presupposes that the reason to do the feature at all is performance. Interestingly enough, the customer requests that motivated considering this feature back in the 1990s were not primarily about performance.

Why not? The 1990s was a very different time for JS than today; scripts were mostly tiny. The notion that there would someday be frameworks with hundreds of thousands of lines was not even close to being on our radar. Downloading and parsing JS was a tiny fraction of the time spent downloading and parsing HTML.

Nor was the motivation the extension to other languages, though that was of interest to Microsoft as we had VBScript running in the browser as well, which used a very similar bytecode language. (Being developed by the same team and compiled out of the same sources and all.)

Rather, the primary customer scenario for motivating bytecode in the browser was to make the code harder to read, understand, decompile, reverse-engineer and tamper with. That a bytecode language is hardly any additional work to understand for any attacker with reasonable resources was major points against doing this work; we did not want to create a false sense of security.

Basically there were lots of expenses and precious few benefits, so it did not get done. Something must have changed between 1998 and 2015 that made WebAssembly have a reasonable price-to-benefit; what those factors are, I do not know. You'd have to ask an expert on WebAssembly.

  • 56
    WASM's goal is not necessarily to be a target for Javascript compilation, but rather to be a better compiler target than Javascript is (for a variety of languages). – Robert Harvey Dec 9 '19 at 20:32
  • 21
    Happily, this question is answered in the WASM Goals and WASM faq. For context, the industry seems to have collectively given up on Java and Flash, security-wise; ASM can help fill that void. Newer standards like service workers gets us on the way to web applications as an alternative to mobile applications, but less so as a means to stuff a AAA game into a web page (one of the specific WASM Use Cases). – Brian Dec 9 '19 at 21:13
  • 42
    @Brian: Ironically, about half of all sites using WASM are using it for malicious purposes. – Robert Harvey Dec 9 '19 at 21:16
  • 24
    I think the second paragraph (we are not required ... hundreds of millions of dollars ...) ruins the experience of this otherwise good answer. It appears to be unnecessarily hostile - especially because such a feature would not cost that much to implement. Hundreds of millions of dollars is the equivalent of thousands of myselfs for a whole year - how could you even finance a whole compiler then? – phresnel Dec 12 '19 at 8:16
  • 9
    @phresnel: If your question is "what are the economics of implementing languages with industry-wide implications in the corporate world?" that is a complex question that you should do some research on if it interests you, and then ask a more focused question. Consider for example the total cost to Microsoft of C#: a large team has worked on it for 20 years; include in your estimates the costs of design, implementation, testing, documentation, marketing, user education, and so on, and then compare that to offsetting cost savings or revenues. – Eric Lippert Dec 12 '19 at 14:42
114

View Source

"View Source" was in the beginning, and still is to some extent, considered to be an important feature of the web. It is how generations of web developers learned web development, and the relevant standards bodies (ECMA TC39, W3C, WHATWG) still take it very seriously.

Minification

ECMAScript files are typically "minified" before being deployed. This includes removal of all comments, all whitespace, and renaming of all identifiers to be as short as possible, plus some higher-level optimizations such as removal of dead code.

Compression

Support for compression exists in HTTP since HTTP/1.0 (early 1996). ECMAScript is text, and text compresses really well. In fact, ECMAScript is text with lots of redundancies (lots of appearances of ;, {, }, (, ), ,, ., function, var, if, for, and so on), and compression algorithms thrive on redundancy. So, the amount of data that is transferred is much smaller than you make it out to be. As an experiment, try compressing an ECMAScript source file with one of the typical compression algorithms used on the web (e.g gzip or deflate), and compare that to the size of the compiled bytecode of the same file.

Bytecode format

Which brings us to the next problem: there is no standardized bytecode format for ECMAscript. In fact, some implementations may not even use bytecode at all! For example, for the first couple of years, V8 compiled ECMAScript straight to native machine code, with no bytecode step in between. Chakra, SquirrelFish Extreme, and SpiderMonkey all use bytecode, but they use different bytecode. dyn.js, TruffleJS, Nashorn, and Rhine don't use ECMAScript-specific bytecode, they compile to JVML bytecode. Likewise, IronJS compiles to CLI CIL bytecode.

Now, you might say: why not define a standardized bytecode format for ECMAScript? The problems with this are two-fold:

  1. A bytecode format constrains the design of the execution engine. For example, look at JVMs: JVMs are much more similar to each other than ECMAScript engines. Personally, I believe the "performance race" of the late 2000s / early 2010s would not have been possible without the wide range of experimentation that the lack of a standardized bytecode format afforded.

  2. Not only is it hard to get all ECMAScript engine vendors to agree on a common standardized bytecode format, but consider this: it doesn't make sense to add a bytecode format for only ECMAScript to the browser. If you do a common bytecode format, it would be nice if it supported ActionScript, VBScript, Python, Ruby, Perl, Lua, PHP, etc. as well. But now you have the same problem as in #1, except exponentially increased: not only do all ECMAScript engine vendors need to agree on a common bytecode format, you also have to get the PHP, Perl, Ruby, Python, Lua, etc. communities to agree as well!

Caching

Well-known widely-used libraries are hosted at canonical URIs, where they can be referenced from multiple sites. Therefore, they only need to be downloaded once and can be cached client-side.

CDN

Many libraries use CDNs, so they are actually served from a location close to the user.

Wasm / asm.js

WebAssembly (Wasm) is a compact binary instruction format that is currently being standardized by the W3C and already being shipped in Firefox, Chrome, Safari, and Edge. It is, however, not designed as bytecode format for ECMAScript, rather it is designed as a low-level portable machine code and compilation target for languages like C, C++, and Rust.

Before Wasm, there was already asm.js, which had similar goals, but it was designed as a syntactic and semantic subset of ECMAScript, so you could run it unmodified in a non asm.js-aware engine, and it would work, just much slower.

  • 8
    WebAssembly reached recommendation status as of December 5, 2019. – Rob Dec 10 '19 at 13:00
  • 1
    This explains it best with main and important points first – tgkprog Dec 11 '19 at 5:02
  • 1
    @tgkprog Well, the most important point is still "It's a specific feature, and you need to have a good reason to include any feature. There was no reason to standardise JS bytecode, rather than JS language, and many reasons why it would be bad idea." :) Features need to be useful enough to not only return on their investment (which this wouldn't), but also be better than alternative features you could implement with similar effort. As Jörg notes, even today, WebAssembly doesn't care about being a target for Javascript applications - there's little point in doing that. – Luaan Dec 11 '19 at 8:13
  • 1
    @MichaelMrozek: The fact that different portions of the ECMAScript community (standards writers and library writers) have different reasons does not invalidate either reason. – Jörg W Mittag Dec 11 '19 at 21:41
  • 3
    @MichaelMrozek: As long as the people on the standards committee are idealists that believe in "View Source", they have no incentive to standardize a bytecode format. As long as the people in the community can work around this using minification, they have no incentive to replace the people on the standards committee. Again: two different groups of people can have different motivations, this does not invalidate either motivation. – Jörg W Mittag Dec 18 '19 at 9:51
18

JavaScript was invented by Netscape. The design goal was to make web pages interactive through an embedded scripting language. It was not initially intended for complex application - complex stuff was supposed to be written as Java applets or plug-ins, and JavaScript was positioned as simple "glue" code which could connect HTML elements with Java applets and other scriptable plug-ins.

JavaScript was designed to be embedded directly in HTML, like <input type="button" onclick="alert('hello world')">. These days it is frowned upon to embed JavaScript in HTML, but in those days this was the standard way to hook up event handlers. Given this use case, JavaScript basically had to be text-based.

There were also facilities for generating HTML in JavaScript, like:

<script>
  document.write("<input type=\"button\" onclick=\"alert('hello world')\">"
</script>

Again, this basically requires JavaScript to be textual format to be useful.

Furthermore, a textual format is much easier for casual developers, since you don't need a development environment to compile it into bytecode. You just type the text and reload the browser to see it run. At the time when JavaScript was introduced, dedicated HTML editors barely existed. People wrote web pages in notepad. There were no such thing as build pipelines for web pages.

The downsides you mention wasn't really a consideration at the time. Since it was designed for small scripts, the overhead of a textual format was completely insignificant.

  • 3
    Eh? Who claims Javascript was invented by Microsoft? Why would anyone want you to think that? Microsoft was part of the standardisation process (which produced the first version after IE 2 was released, with support for Javascript), but who cares? It's a bit silly thing to put on top of an otherwise good answer. – Luaan Dec 11 '19 at 8:33
  • 1
    @Luaan: OK I removed that sentence. It was in response to another answer which suggested this was some decision by MS. It was not. They had no choice that to copy JavaScript exactly as it was designed by Netscape. – JacquesB Dec 11 '19 at 11:02
  • 1
    @wvxvw: Obviously it would be technically possible to embed base64-encoded bytecode in HTML - but it would be an immensely complicated solution compared to just embedding the source code. Just imagine the hassle of debugging. – JacquesB Dec 11 '19 at 11:03
  • 1
    @JacquesB: As one of the people making those choices I can assure you that we did have the choice to make changes to JS, and we occasionally made choices contrary to poor implementation decisions in the Netscape version. Once the standardization process started up, we worked closely with our Netscape counterparts as well as interested parties from other companies to ensure that the language evolved in a consistent and reasonable manner that created value for users without undue burden to implementers. – Eric Lippert Dec 12 '19 at 1:22
  • 2
    @wvxvw Base64 increases the size of the encoded text by as much as one third. So, I'm not sure how that helps when you'd prefer lower sizes of your web pages. – VLAZ Dec 12 '19 at 8:45
1

Data use is probably not actually a problem.

To respond to the assumption in the body (since the wonderful response by Eric Lippert seems to have the actual questions quite well covered):

Whether you're talking about data caps or bandwidth, my Google-Fu has been unable to unearth any research that suggests that Javascript is actually "wasting terabytes of data" (whatever that means).

As for the rest of your questions, in many things, it is less useful to ask "what problems will this cause?" than to first ask "what benefits will this create?".

  • Given the question subject I think it would be safe to assume that “wasting terabytes of data” refers to network usage. You could also factor in storage too I suppose when you consider the millions of devices that host JavaScript. – Caltor Dec 26 '19 at 9:34

Not the answer you're looking for? Browse other questions tagged or ask your own question.