114

Apple launched its new programming language Swift at WWDC14. In the presentation, they made some performance comparisons between Objective-C and Python. The following is a picture of one of their slides, of a comparison of those three languages performing some complex object sort:

enter image description here

There was an even more incredible graph about a performance comparison using the RC4 encryption algorithm.

Obviously this is a marketing talk, and they didn't go into detail on how this was implemented in each. I leaves me wondering though:

  1. How can a new programming language be so much faster?
  2. Are the Objective-C results caused by a bad compiler or is there something less efficient in Objective-C than Swift?
  3. How would you explain a 40% performance increase? I understand that garbage collection/automated reference control might produce some additional overhead, but this much?

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • 12
    @MathewFoscarini Obj-C goes to assembler, but it's got an expensive object message dispatch mechanism. For most GUI work that doesn't matter, but for sorting it matters a lot. – Donal Fellows Jun 2 '14 at 22:12
  • 17
    The comparison to Python is the real head-scratcher here. – asthasr Jun 2 '14 at 22:20
  • 9
    @syrion marketing, and the language appears to borrow from python's syntax (much like golang). They are trying to say "hey, python developers, you can write something not too foreign on the mac and have it be SO much faster, even faster than Objective C that you never really got the hang of" – user40980 Jun 2 '14 at 22:47
  • 4
    @MichaelT I get that, but it's still odd. Anyone who knows anything about languages will realize that Python, as an interpreted language, simply isn't going to be in the same ballpark as Objective-C or other compiled languages (for most tasks). Using it as a benchmark is odd. – asthasr Jun 2 '14 at 22:50
  • 7
    They probably mean the time it takes to write the code... – Lukas Eder Jun 4 '14 at 13:38
61

First, (IMO) comparing with Python is nearly meaningless. Only comparison with Objective-C is meaningful.

  • How can a new programming language be so much faster?

Objective-C is a slow language. (Only C part is fast, but that's because it's C) It has never been extremely fast. It was just fast enough for their (Apple's) purpose, and faster then their older versions. And it was slow because...

  • Do the Objective-C results from a bad compiler or is there something less efficient in Objective-C than Swift?

Objective-C guaranteed every method to be dynamically dispatched. No static dispatch at all. That made it impossible to optimize an Objective-C program further. Well, maybe JIT technology can be some help, but AFAIK, Apple really hate unpredictable performance characteristics and object lifetime. I don't think they had adopted any JIT stuff. Swift doesn't have such dynamic dispatch guarantee unless you put some special attribute for Objective-C compatibility.

  • How would you explain a 40% performance increase? I understand that garbage collection/automated reference control might produce some additional overhead, but this much?

GC or RC doesn't matter here. Swift is also employing RC primarily. No GC is there, and also will not unless there's some huge architectural leap on GC technology. (IMO, it's forever) I believe Swift has a lot more room for static optimization. Especially low level encryption algorithms, as they usually rely on huge numeric calculations, and this is a huge win for statically dispatch languages.

Actually I was surprised because 40% seems too small. I expected far more. Anyway, this is the initial release, and I think optimization was not the primary concern. Swift is not even feature-complete! They will make it better.

Update

Some keep bugging me to argue that the GC technology is superior. Though stuff below can be arguable, and just my very biased opinion, but I think I have to say to avoid this unnecessary argument.

I know what conservative/tracing/generational/incremental/parallel/realtime GCs are and how they are different. I think most of readers also already know that. I also agree that GC is very nice in some field, and also shows high throughput in some cases.

Anyway, I suspect the claim of GC throughput is always better than RC. Most of overhead of RC comes from ref-counting operation and locking to protect ref-count number variable. And RC implementation usually provide a way to avoid counting operations. In Objective-C, there's __unsafe_unretained and in Swift, (though it's still somewhat unclear to me) unowned stuffs. If the ref-counting operation cost is not acceptable, you can try to opt-out them selectively by using the mechanics. Theoretically, we can simulate almost unique-ownership scenario by using non-retaining references very aggressively to avoid RC overhead. Also I expect the compiler can eliminate some obvious unnecessary RC operations automatically.

Unlike RC system, AFAIK, partial opt-out of reference-types is not an option on GC system.

I know there're many released graphics and games which are using GC based system, and also know most of them are suffering by lack of determinism. Not only for performance characteristic, but also object lifetime management. Unity is mostly written in C++, but the tiny C# part causes all the weird performance issues. HTML hybrid apps and still suffering by unpredictable spikes on any system. Used widely doesn't mean that's superior. It just means that's easy and popular to people who don't have many options.

Update 2

Again to avoid unnecessary argument or discussion, I add some more details.

@Asik provided an interesting opinion about GC spikes. That's we can regard value-type-everywhere approach as a way to opt-out GC stuff. This is pretty attractive, and even doable on some systems (purely functional approach for example). I agree that this is nice in theory. But in practice it has several issues. The biggest problem is partial application of this trick does not provide true spike-free characteristics.

Because latency issue is always all or nothing problem. If you have one frame spike for 10 seconds (= 600frames), then the whole system is obviously failing. This is not about how better or worse. It's just pass or fail. (or less then 0.0001%) Then where is the source of GC spike? That's bad distribution of GC load. And that's because the GC is fundamentally indeterministic. If you make any garbage, then it will activate the GC, and spike will happen eventually. Of course, in the ideal world where GC load will be always ideal, this won't happen, but I am living in real world rather than imaginary ideal world.

Then if you want to avoid spike, you have to remove all the ref-types from the whole system. But it's hard, insane, and even impossible due to unremovable part such as .NET core system and library. Just using non-GC system is far easier.

Unlike GC, RC is fundamentally deterministic, and you don't have to use this insane optimization (purely-value-type-only) just only to avoid spike. What you have to do is tracking down and optimizing the part which causes the spike. In RC systems, spike is local algorithm issue, but in GC systems, spikes are always global system issue.

I think my answer is gone too much off-topic, and mostly just repetition of existing discussions. If you really want to claim some superiority/inferiority/alternative or anything else of GC/RC stuffs, there're plenty of existing discussions in this site and StackOverflow, and you can continue to fight at there.

  • 4
    Garbage collection, especially generational, is usually significantly faster than reference counting. – Jan Hudec Jun 5 '14 at 16:33
  • 8
    @JanHudec Your significantly faster is simply meaningless in realtime graphics field. That's why I mention huge leap is required on GC. Generational GC is not even close to be spike-free both in theoretically, and practically. – Eonil Jun 5 '14 at 21:37
  • 5
    Faster and spike free are completely orthogonal categories. Generational garbage collectors are faster. They are not spike free. – Jan Hudec Jun 5 '14 at 21:46
  • 4
    What you're talking is throughput. Faster has been always a vague term, and can mean anything by the context. If you want to argue about meaning of terms, you should use more precise term rather than faster especially considering current context - realtime graphics. – Eonil Jun 5 '14 at 22:32
  • 10
    @JanHudec: On mobile devices, or any device with constrained resources, GC is not significantly faster and in fact is a major part of the problem. – Mason Wheeler Jun 14 '14 at 13:18
71

Being 3.9 times faster than python, the language that consistently loses most benchmarks by considerable margin (ok, it's on par with Perl, Ruby and PHP; but it loses to anything statically typed), is nothing one should be boasting about.

The benchmarks game shows C++ programs that are more than order of magnitude faster than python programs in most cases. It is not much better when comparing with Java, C# (on Mono), OCaml, Haskell and even Clojure, which is not statically typed.

So the real question is how is Objective-C only 2.8 times faster than python. Apparently they carefully chose benchmark where the slow fully dynamic dispatch of ObjC hurts a lot. Any statically typed language should be able to do better. In complex object sort there are many method calls for comparing the objects and the actual comparison was probably not very complex itself. So if Swift takes at least some advantage of the type information, it can easily do better on the method calls and there is not enough other operations that ObjC could be better in.

Of course, as the benchmarks game clearly demonstrates, the relative performance on different tasks varies wildly, so one benchmark is not really representative. If they had benchmark where it had bigger advantage, they would have shown us that one instead, so on other tasks it probably isn't better or not so much.

  • 13
    I don't quite understand the point of this answer. Are you answering "how is swift faster" by saying "the benchmarks are flawed"? Is that the point you're making? I don't see how that answers what was being asked. – Bryan Oakley Jun 3 '14 at 11:57
  • 15
    @BryanOakley: I don't think the benchmarks are flawed, but the possibility that they picked the one benchmark where Swift was faster certainly has to be considered. – Jan Hudec Jun 3 '14 at 12:18
  • 23
    It's possible that the answer to "How is Swift faster?" might be "It's not, actually", @BryanOakley; that is the gist that I get from Jan's answer. "Lies, damn lies, and statistics", after all. – Josh Caswell Jun 3 '14 at 19:51
  • 4
    A while back we benchmarked Codename One running on iOS and our Java implementation was much faster than Objective-C codenameone.com/blog/… Jan is correct, dynamic dispatch is really slow, if they improved them even a little bit then some benchmarks will show a huge improvement. If they improve ARC even by a fraction (thanks to better code analysis) they can get rid of a ridiculous amount of complexity. The more restricted the language the more the compiler can do to optimize (see Java) and Swift adds restrictions. – Shai Almog Jun 4 '14 at 5:34
  • 7
    Jan's answer is a perfect answer to the Q1 and probably Q2. When I saw the benchmarks on a marketing event as the keynote, I thought: "Wow, only 1,3x in the selected best-case. What will we the average result? 0.3x?" – Amin Negm-Awad Jun 9 '14 at 8:30
4

Objective-C dynamically dispatches every method call.

Hypothesis: The benchmark uses static typing to let the Swift compiler hoist the compare method lookup out of the sort loop. This requires a narrow type restriction that only allows Complex objects in the array, not subclasses of Complex.

(In Objective-C you could hoist the method lookup manually if you really want to, by calling on the language runtime support to look up the method pointer. You'd better be sure that all instances in the array are of the very same class.)

Hypothesis: Swift optimizes reference-counting calls out of the loop.

Hypothesis: The Swift benchmark uses a Complex struct in place of an Objective-C object, so sort comparisons don't need dynamic method dispatches (since it can't be subclassed) or reference-counting work (since it's a value type).

(In Objective-C you can fall back to C/C++ for performance as long as it doesn't involve Objective-C objects, e.g. sort a C array of structs.)

3

Honestly, unless they release the source to the tests they are using I wouldn't trust anything Apple has to say on the subject. Remember, this is the company which switched from PPC to Intel based on power concerns when 6 months earlier they were saying that Intel sucked and actually torched the Intel bunny in a commercial. I would like to see hard irrefutable proof that Swift is faster than ObjC in more categories than just sorting.

Additionally you have to question any stats which are released at WWDC since they have the smell of marketing all over them.

All of that being said I haven't run any tests between swift and ObjC myself, but from what I know swift has it's own LLVM IR extensions and it is possible that more optimization is being done at compile time than in ObjC.

Full Disclosure: I am writing an open source Swift compiler located at https://ind.ie/phoenix/

If anyone would like to help make sure that Swift isn't just available on Apple hardware, let me know and I would be glad to include you.

0

I've struggled myself through the Swift tutorial, and it seems to me that Swift is more down to earth (makes me think of Visual Basic) with less 'object-ification' than Objective-C. Had they taken into account C or C++, I presume the latter would have won, since they are even more compile-time only.

In this case, I presume Objective-C is the victim of it's object oriented pureness (and overhead).

  • 13
    Doing a "complex object sort" benchmark would be a bit difficult in a language such as C that doesn't have a native implementation of objects. Given that the audience for this talk was likely 100% Objective-C programmers, comparing to a language like C++ doesn't make much sense either. The point of swift isn't "hey, this is the greatest language ever!" but rather "hey, this is faster than the language you're using now for OSX/iOS development". – Bryan Oakley Jun 3 '14 at 12:00
  • 10
    C has a perfectly fine qsort which will allow complex object sorting; it just uses a callback which understands the objects at hand. I suspect C++ is missing because std::sort would embarrass Swift. (Since it's a template, a C++ compiler can heavily optimize it, up to loop unrolling.) – MSalters Jun 4 '14 at 10:46
  • @MSalters: I fully agree with you. Both C and C++ have capabilities to outrun Swift. Would it be possible to get the executed test. I'm more than willing to execute the same benchmark with Swift, Objective-C, C++ and C. – Painted Black Jun 4 '14 at 21:07
  • @BryanOakley also, "This language requires less square brackets!" – Nick Bedford Jun 20 '14 at 2:57
  • 1
    This answer doesn't hold water at all and is horribly misleading. OO is not really slow, in fact the fastest systems you'll find around are going to be C++, Java and C# and the style (heavily OO or not) of programming will have very little to do with the resultant speeds unless you have really bad code. – Bill K Nov 14 '14 at 2:43

protected by maple_shaft Jun 3 '14 at 19:56

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.