51
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
At a global level, you're wrong - language does matter, or at the very least you will spend more $$$ on compute if you write it in a less computationally efficient language.
While I'm not at liberty ...
- 22.9k
42
votes
Accepted
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
There's a good video (warning, contains really a lot of swearing, but worth it) that will give you a good basis for arguing against this consultant... Solving Imaginary Scaling Issues (at Scale), by ...
- 773
27
votes
Accepted
How does a company like Amazon avoid bottlenecks accessing the database layer?
However, if multiple users are being served by separate servers and both try to add the same item to their cart, for which there is only one remaining, there must be some "source of truth" for the ...
- 51.1k
20
votes
Accepted
How can I manage trash users?
Some services remove accounts that have not seen any activity in a certain amount of time, say, a year.
Others don't bother, on the ground that keeping a user record in their system is a trivial ...
- 9,783
20
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
You should remove this consultant from touching any of your system. A consultant suggesting a complete rewrite of an application in a different language are going to cause a lot of damage to the ...
- 12.3k
19
votes
How can I manage trash users?
Weirdly, Facebook or Google have so many users that this isn't much of a problem for them.
Whoever picked a really desirable username (e.g. "Frank") probably already did so back in 2008. The many, ...
- 108k
15
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
With a simple CRUD api, your bottlenecks are probably the network and the database. So the code you use for mediating and translating between the two layers is probably not that performance-critical.
...
- 23.2k
11
votes
Saving high-frequency events to a connection-limit constrained database
My guess is that you need to explore more carefully an approach that you have rejected
Enqueue the events on our server
My suggestion would be to start reading through the various articles published ...
- 32.2k
10
votes
Accepted
How can I scale a database that requires that writes are consistent?
The basic tradeoff of distributed system is that if you increase the number of write replicas that is needed to be updated for commitment, you improve reliability, but increase latency. On the flip ...
- 12.3k
9
votes
How can I manage trash users?
Separate login credentials from display names
Allow users to log in with their email address or existing account from a site which provides such a service (e.g. Google or Facebook). If you really ...
- 1,692
9
votes
Accepted
Saving high-frequency events to a connection-limit constrained database
Input stream
It is not clear if your 1000 events/second represent peaks or if it's a continuous load:
if it's a peak, you could use a message queue as buffer to spread the load on the DB server over ...
- 74.8k
8
votes
Accepted
switching implementations dynamically according to execution time estimates
The standard approach for, say, sorting a list is to use quicksort for lists greater than a threshold k and insertion sort otherwise. So partitioned lists using quicksort of a certain dimension are ...
- 22.7k
7
votes
How does a company like Amazon avoid bottlenecks accessing the database layer?
I have seen the 'Last Item In Stock' problem solved in the following way:
Update all the stock levels daily and flag products as high, low, on order or out of stock categories according to threshold ...
- 71k
7
votes
Should data transformation be on the front or on the back end in this scenario?
The decision to do the transformation on the front-end or the backend should depend much more on the kind of transformation that you want to do rather than the number of items involved in the ...
- 72.5k
7
votes
Developing your application as microservices or as a monolithic app - which is cheaper when deploying and scaling?
There's no "normally". Whichever of two options is cheaper to deploy, operate and scale depends on many factors. Often the deployment of a monolith is cheaper, but for example if you need changes in ...
- 14.7k
7
votes
Accepted
How can adding redunancy adversely affect performance
Scenario A: You have a system that solves problem X. Scenario B: You have a system that solves problem X and has to ensure that it is always synchronized with the redundant backup.
It is pretty clear ...
- 102k
7
votes
Designing a microservice architecture
To meet this requirement, I need several microservices.
Not really. You don't 'need' microservices at all. There may be good reason to decompose your architecture into smaller loosely coupled ...
- 25.2k
6
votes
How does a company like Amazon avoid bottlenecks accessing the database layer?
A combination of
hashing
sharding
replication
distribution
high fail-over
key-value stores
There's no magic, just more and more complex situations. Just like DNS, it is made to scale.
The 'single ...
- 13.1k
6
votes
How to adapt the architecture of a website and have suitable hosting along with it to meet increasing users?
Just a quick note on terminology:
Scaling up refers to improving the hardware.
Scaling out refers to adding more hardware in a load balanced environment.
In the web world, we tend to scale out the ...
- 26.1k
6
votes
Accepted
What time series database can support high cardinality?
Performing complex queries in real time on large time series data is not scalable. It puts an unreasonable load on your store, and will not perform well regardless of the database used. The only time ...
- 9,137
6
votes
Saving high-frequency events to a connection-limit constrained database
If I understand correctly the current flow is:
Receive and event (I assume through HTTP?)
Request a connection from the pool.
Insert the event to the DB
Release the connection to the pool.
If so I ...
- 25.2k
6
votes
Should data transformation be on the front or on the back end in this scenario?
Where to transform data?
Should data transformation be on the front or on the back end?
The question is slightly nonsensical, or at least glossing over certain necessary consideration. Conversion ...
- 45.6k
6
votes
Accepted
Sharding rule updating on DB scaling
Starting from V11,
Postgres supports Declarative Partitioning where you can divide the table into partitions with built-in Hash partitioning strategy (modulus or round-robin).
It also allows you to ...
- 3,142
6
votes
Am I right that switching programming languages will have little impact on the scalability of a CRUD API?
By that they seem to mean that a backend API written in .NET will handle the growing performance requirements much better than a backend API written in a scripting language like JavaScript, Ruby or ...
- 109k
5
votes
Accepted
similarity measure between millions of documents
I think you are taking the wrong approach by comparing every document with every other document. You should define an N-dimensional space to contain your documents then use a nearest neighbour ...
- 879
5
votes
Shopping Cart Concurrency Question
The problem is of course the database access on the 'add to basket' request.
Customers will add stuff to their basket all the time, they aren't always going to be buying that item. Plus at times of ...
- 71k
5
votes
Saving high-frequency events to a connection-limit constrained database
Assumptions
I'm going to assume that the load you describe is constant, as that is the more difficult scenario to solve for.
I'm also going to assume you have some way of running triggered, long-...
- 51
5
votes
Should we share a database between two 'Microservices'?
The reason why it's generally preferable to separate out writes and reads is because the problems that arise from them are different, and depending on whether your application is read or write heavy ...
- 489
5
votes
Accepted
How do atomic updates work at scale?
First things first, based on this question it appears you do not at all understand the concept of atomicity. I’d recommend doing a bit more work to understand that before moving on to distributed ...
- 109k
5
votes
Performance implications of a game project monorepo on *modern* Git
Git just does not scale to huge projects. Repos should not generally grow beyond 5GB including the entire history if you want a good user experience (or if you want to use commercial hosting services)....
- 133k
Only top scored, non community-wiki answers of a minimum length are eligible