35

An external consultant to our team advised us to rewrite our SaaS offering (essentially a CRUD API) in .NET because this is more "scalable" than using Node.js (or Ruby on Rails, Flask, etc.). By that they seem to mean that a backend API written in .NET will handle the growing performance requirements much better than a backend API written in a scripting language like JavaScript, Ruby or Python, once the startup scales to tens of millions of users (an ambitious dream).

To me this seems wrong. The performance of a CRUD API should be completely dominated by the choice of architecture and hardware instead of the programming language.

Am I right that switching programming languages will have little impact on the scalability of a CRUD API?

9
  • 22
    Did the consultant mean scalable in the sense of request per minute, or did they mean in terms of lines of code? Anyone can knock up a quick API in any language, but once it reaches a level of complexity the choice of programming language becomes more important (for example, typescript was created to be better for larger applications than vanila JS)
    – DaveShaw
    May 10 at 19:42
  • 2
    @DaveShaw "lines of code" actually sounds like the opposite of what a switch from JS to e.g. C# would do - you would end up with more lines of code! The increase in scalability would be more regarding maintenance; those "more lines" are basically more rules, leaving less room for bugs. Which is especially helpful for those modules nobody has touched in years, and the people who did don't even work here anymore.
    – R. Schmitz
    May 11 at 17:20
  • 3
    What comes to mind are the people calling Java "verbose" because a hello world has public static void - because they don't know that every single word there is a feature of the language.
    – R. Schmitz
    May 11 at 17:21
  • 3
    The idea that more code == less room for bugs doesn't necessarily hold up in practice. Bugs result from human oversight, and in a more verbose language, there's a lot more room to overlook things. No type system in the world is sufficiently expressive to rule out every possible bug, either. If a lot of the code is just instructions for the type system, then code that actually specifies features is spread thinner and harder to comprehend in full. If you need really correct code, I don't think there's any substitute for rigorous testing, which you can do in any language. May 11 at 23:40
  • 13
    As a proud fan of a little-used language, I regret to have to say this, but recoding a non-trivial working production program just to change the codebase's programming language is almost always an incredibly dumb idea. Even if that new language is my favorite language. At best after all that effort you are going to end up with something that does the same thing, but with new bugs.
    – T.E.D.
    May 12 at 0:42

10 Answers 10

42

There's a good video (warning, contains really a lot of swearing, but worth it) that will give you a good basis for arguing against this consultant... Solving Imaginary Scaling Issues (at Scale), by Laurie Voss at DinosaurJS conference.

The crux of the matter is that the consultant hasn't identified any actual, specific problem, so there's simply no way to know if the solution will fix any problem.

(Rehashing some territory from other answers here...) Scalability when used in this vague context means many things, horizontal scaling, vertical scaling, team scaling, etc. But they aren't the same, and when you actually need to fix something then you need to identify the real, underlying problem as specifically as possible.

If your general problem (scalability) was that your leg hurts, then one solution (new language) might be a cast, another solution (api change) might be antibiotics, etc., but you really have to know what the underlying cause is, e.g. broken bone, infection,..., (cpu usage, network times) to know which cure will actually do anything for you. And there are costs to using unnecessary cures, even in the best case they are simply expensive to implement, in the more common case they actually cause other problems when inappropriately used.

Your consultant should be providing you with real data about your system showing what the underlying problem is and why this will fix it. If you were currently experiencing the problem in the wild then I would want some metrics such as memory usage from prod, metrics from the ticketing system showing that devs are spending a long time struggling with language issues, whatever. Since this doesn't actually appear to be live now, I would still want some study with simulated load in non prod or specific areas of the code that will become difficult to work on or whatever. About the only time this random opinion nonsense is sort of OK is if the proposed solution is very low/no cost, like if the system doesn't even exist yet, but then I'd be more interested in opinions of the people who may actually maintain it one day, as opposed to a consultant who will be waving goodbye and leaving all the work with other people.

(I am on my fifth or sixth re-watch of this video, my management is notoriously buzzword happy and more influenced by outside opinions vs on the ground devs who could easily point out several problems which are low hanging fruit for fixing, so I have seen many of these ideas being proposed with actual seriousness.)

13
  • 17
    It's a macro level version of "optimizing without having profiled the code".
    – Dave
    May 11 at 15:20
  • 2
    Thank you, @user3067860, for pointing me at that talk. Not because it's useful to me, but because it's one of the most fun and enjoyable programming talks I've watched. May 11 at 18:58
  • 2
    Oh yeah, this is the answer. So what if C# can execute a use case 200ms faster than whatever you are using. If you spend 2 seconds sending SQL across the network, c# isn't going to help. It will just be 2000 - 200 ms of time. Ask for specific metrics about parts that are slow, or ask for time to profile what you have. And ask specifically what they mean by "scaling". Also all what problems they ran into before that caused them to suggest C#. May 11 at 20:17
  • 1
    @Moo "Better in most cases"...but no clue if it's going to be better in this specific case...is exactly the point. Minifying your UI is pretty much always better, but if you're minifying your O(n^3) ToDo list application then it's still going to be incredibly slow and also now you've spent time on something that didn't move you any closer to fixing the actual problem. If you are switching languages, that could be a lot of time spent without being sure it will help anything. softwareengineering.stackexchange.com/questions/80084/… May 12 at 19:11
  • 1
    @PieterB Sure, but you don't know which case you're in until you actually...figure out which case you're in. Otherwise you're just throwing darts blindfolded and hoping you're facing towards the dartboard. And rewriting your application is a really, really expensive dart to throw randomly. May 16 at 15:17
51

At a global level, you're wrong - language does matter, or at the very least you will spend more $$$ on compute if you write it in a less computationally efficient language.

While I'm not at liberty to go into details, I work for Disney Streaming and it's well known we're a Scala shop. If our highest scale services were written in (say) Node.js rather than a JVM language, we would be spending significantly more each month on the containers/servers we need to run our services. Our services are often compute bound, so there does just come a point at which raw language efficiency does matter.

That all said, you don't have as many users and as much concurrency as we do. If you had to run twice as many servers, would that have a significant impact on your business? If not, you're probably right to keep working in the languages your team is used to.

8
  • You're right, I think the way I phrased the question is bad. For video streaming and many other computationally intensive tasks, the choice of language must be crucial. In our case it's essentially a CRUD app, and I think then it doesn't matter. I should have made that clear.
    – Vincent
    May 10 at 11:58
  • 11
    The services I'm talking about here are not the actual video streaming, that's all served from CDNs (not a secret in any way, before anyone gets interested). At large enough scale, even SSL termination is expensive. May 10 at 12:01
  • 4
    @Vincent: video streaming is not computationally expensive. It's just bandwidth and disk expensive. Video encoding is computationally expensive, but nobody writes their own codecs anyway, and most off the shelf ones are already written in fast languages and have an API you can call from any scripting language.
    – Lie Ryan
    May 11 at 3:18
  • 12
    Did you actually try node.js? I don't think you can claim that JS is faster than JVM without actually measuring it, any more. It used to be an obvious truth (so did C beating JVM).
    – user253751
    May 11 at 17:39
  • 1
    "Our services are often compute bound" — well, that's important.
    – Džuris
    May 13 at 7:58
20

You should remove this consultant from touching any of your system. A consultant suggesting a complete rewrite of an application in a different language are going to cause a lot of damage to the system and to your company.

Consultants often make these kind of expensive, vague suggestions; the real reason they are suggesting to rewrite your application in their preferred pet language is to lock you in into their service, not anything for your own benefit. Find a different consultant that fits into your system, not a consultant that wants to retrofit your system into their world.

Most web applications won't scale better when rewritten in a different language, because the majority of web applications spend their time in database or API calls, which are already written in fast languages. You'll get better miles for your effort by optimising the uses of those database calls than by rewriting the same logic in a different language. And even when there's a good reason for rewriting a slow calculation in a different language, you usually only want to rewrite a very small part of the performance sensitive component in a faster language, while keeping the bulk of your application in easier to use scripting language.

When it comes to scalability, there are many different metrics.

In the vast majority of cases, the speed of the language matters none to scalability, designing good system architecture is where you need to design the system to scale.

  • making your application stateless, so your application can be scaled out to a cluster rather than a single machine

  • rearchitecting parts of the system that can run as edge service (close to the user) or as asynchronous tasks (not tied to request-response cycle)

  • modularity so multiple developers can work concurrently on the code base

  • writing tests so that anyone can confidently deploy their changes and not fear breaking anything

  • API designs that suits the use case

  • database design, as well as using multiple databases/data store that's appropriate for your use case

  • etc

These are all scalability improvements that will have much better return on investment than rewriting an application in a different language.

And in most cases when you want to rewrite a performance sensitive parts in "faster" language, the language you should be using isn't going to be .NET, but usually a systems programming language like C/C++ or Rust, which you can then FFI call from your existing application. .NET is too much still a high level framework that won't actually gives you the scalability or performance benefit of a rewrite.

1
  • 7
    Thank you for being the one to callout the TL;DR: the consultant is a f@#$ing idiot. That being said, OP should probably refute this from a business rather than technical perspective, like "a rewrite is going to cost the company an estimated $X, what's the ROI?" May 11 at 17:39
15

With a simple CRUD api, your bottlenecks are probably the network and the database. So the code you use for mediating and translating between the two layers is probably not that performance-critical.

But is it really? How much business logic do you have in the application? How computationally expensive is that business logic? These were rhetorical questions, because this is for you to determine through analyzing what you actually want to do, prototyping it and doing benchmarks.

5
  • 2
    Agree with the overall sentiment, but I'll push back on the implication that simple CRUD apps won't benefit from one choice of language/platform over another if they're IO bound. A lot of "scripting" languages (Ruby, Python) are typically scaled by increasing the number of processes (as a workaround for their GILs), which has a higher memory footprint, and limits how many of them you can fit on one server, and thus how many concurrent IO-bound things a single server can wait on.
    – Alexander
    May 10 at 17:01
  • .Net performance increased considerably when they introduced their async/await model over a decade ago - this is because its less overhead to manage a thread thats stalled because of IO wait time than it is to start a new thread to handle a new request or wait for an existing thread in the pool to become completely available. So if you have a lot of IO, its best to let the thread be used for other things while that IO completes, else you potentially block new requests...
    – Moo
    May 10 at 22:03
  • 1
    @Alexander: processes does not necessarily have higher memory footprint. Not in Linux/Unix-based machines anyway if the program uses prefork processing model and/or careful use of shared library so that most of the process memory are shared between worker processes.
    – Lie Ryan
    May 11 at 10:08
  • @LieRyan I thought so too, but I remember some cases where things that you thought would be perfect for being in an immutable __data segment of a dylib that could be shared, aren't. Turns out the way a lot of implementations lay out their class hierarchies and method tables is more mutable than you expect (JS is especially guilty of this), which precludes the ability to share these libs in memory. IDK if forking at just the right time can save you though, I've never played around with that
    – Alexander
    May 11 at 13:12
  • @LieRyan UNIX forking strategy performs badly with GC languages/VMs and especially JIT.
    – Dan M.
    May 13 at 14:47
6

By that they seem to mean that a backend API written in .NET will handle the growing performance requirements much better than a backend API written in a scripting language like JavaScript, Ruby or Python, once the startup scales to tens of millions of users (an ambitious dream).

Are you sure about that?

"Scalable" could have multiple meanings here. Not just in the computational runtime performance, but also for growing out your engineering organization and being able to add functionality in parallel without too much friction. Statically typed languages tend to fare better there, since the type checking becomes more valuable as the codebase grows and it becomes impossible for every/any engineer to know the entire system in detail.

Because your infrastructure costs for a CRUD api are going to be a tiny fraction of your labor costs until the product is very mature and widely used. The consultant might be more worried about that than how linearly your server to request trends are.

1
  • 2
    Typescript exists. It's even good. The static typing available for Python isn't as solid but it's there and fairly mature at this point (I have no idea what static analysis options are available for Ruby). While none of those are ever going to be as fast as the JVM or CLR the person-scalability is more debatable and I don't see how a rewrite could possibly be justifiable from a business perspective unless the AWS bill is already massive. May 11 at 17:43
3

The scalability of a CRUD application is mainly linked to the completion of elementary transactions.

Filling in a couple of variables and calling some functions is only a fraction of the time needed to lock a record in the database, write an operation in the journal (in case a rollback is needed), update the records on the disk, update the indexes (i.e inserting a new value and balancing the btrees).

So, with Node, most of the time will be spent in the database part. If you switch to C++, Go, or C, this part of the job will not change drastically, even if the speed of the preparation for the database calls would be faster.

Another scalability issue is the ability to increase the throughput by reducing the locks. For example, if your application reads a table to find the next free ID, all the processes that concurrently insert a new value would be sequentialized waiting for the locks to be freed. You could even write your code in optimised assembler: the scalability would be 0. So don’t diddle code, find better algorithms.

Last but not least, scalability will also depend on the possibility to split the database across several processing nodes (distributed database) instead of relying on a single instance. This is very challenging, and again, the preparatory steps in native languages are insignificant compared to the DB coordination effort.

1

I don't have the rep to comment yet, and the others here have addressed the critical points:

  1. you need to be presented with data demonstrating that you do have a compute bottleneck,
  2. the amortised cost of rewriting is lower than the cost of just adding more compute/memory (and managing that additional hardware), and
  3. that the incentives of the external consultant align with your business, not with generating more work from you, which will be proved by 1 and 2 above.

However, there does appear to be a 10x+ performance boost available by switching from Express (NodeJS) to actix(Rust) or one of the .net frameworks:

https://www.techempower.com/benchmarks/#section=data-r20&hw=cl&test=composite

1

The language matters for several reasons. The first ones that come to mind are:

  • The language may bring with itself paradigms and special implementations of the common design patterns that will eventually affect the design.
  • The APIs and the available libraries will affect the architecture and the design even more.
  • On a minor note, you can add that there is still some difference in the performance of the different runtime environments.
4
  • 2
    You forgot to mention: if the language is one that's well-understood by your existing engineers, you don't spend time and money training and/or recruiting. May 11 at 16:39
  • @TobySpeight I beg to disagree. Of course the developers skills have an impact on the result. But here the question is about the language and with the tools we have today the choice of the language will have on influence over the choice of the technology stack and the design. Making architectural decisions based on the skills of the team means that often you can't use the right tool for the problem. I don't know whether in this specific case they decided to make a backend in Node.JS because they were skilled on it, but it is not a great choice for a backend.
    – FluidCode
    May 14 at 14:20
  • Oh, sure - don't go down the "every problem is a nail" route and use a hammer because it's all you know. However, don't discount the skills available when choosing between languages that are already suited to the problem. Obviously, don't avoid languages just because they are unknown - but do factor in the necessary learning time in the decision (both formal training and project rework as the language paradigms and idioms become better known). May 15 at 13:00
  • 1
    @TobySpeight "every problem is a nail" we are going a bit OT, but anyway, in this case I prefer an older say, one coming from the ancient Greeks: You recognise the good craftsman by the tools they use.
    – FluidCode
    May 16 at 11:38
0

The consultant probably uses "scalable" as a buzzword devoid of meaning.

Scalable means (simply put) that a system can cope with significantly increased demand without the cost per user going up or the performance per user suffering.

Programming language choice will not affect scalability, since the difference in performance between languages is a constant factor. Let's say language A is 50x faster on average than language B. This factor will remain constant when the number of users increases. So it doesn't affect scalability.

Scalability problems are typically due to architectural bottlenecks and bad algorithms - both of which are issues independent of programming language.

-2

Programming language / framework does matter.

Even within .NET. ASP.NET Core vs. .NET Framework can see a 7 times speed increase for tasks such as serialisation and deserialization and simple string validation such as StringStartsWith. See: .NET Core vs .NET Framework: Testing Performance

Taking into account other languages, their specific overheads, compiled vs scripted, etc. all add up.

Key information from the above site:

.NET Core featured in all my tests much faster than the full .NET – sometimes 7 or even up to 13 times faster. Choosing the right CPU architecture can dramatically change the behaviour of your application, so the results gathered from one architecture can be invalid on the other and vice versa. For example – if your .NET Core application performs much SHA256 hashing computations, then the best option would be to use AMD Zen-based servers instead of Intel-based ones. So to sum up – if performance is a significant factor in your

5
  • 1
    images on that blog-post are broken for me
    – PeterT
    May 12 at 7:45
  • 2
    It sound like something coming straight from Microsoft marketing department.
    – FluidCode
    May 16 at 12:06
  • Yes, serialisation and deserialization in .NET can easily be slowed down by a factor of 30-40 if reflection is invoked (the default). It doesn't scale by default. Real story. May 20 at 15:41
  • This answer misses the point entirely. Of course some languages/frameworks are faster than others for select tasks. The question is: Does it matter for the scalability of the system as a whole?
    – JacquesB
    May 25 at 20:18
  • The question mentions performance several times and uses it interchangeably with scalability if changing languages improves performance by 7 times it by default scales to handles 7 times the request. So not sure how that misses the point entirely ?
    – Daveo
    May 26 at 0:26

Not the answer you're looking for? Browse other questions tagged or ask your own question.