How to deal with massively concurrent events?

Question

I got the following interview question:

Say the company has this event, where we will create a button on a website and only the first ten clicks will be accepted and win a prize. Assume that there a millions of people around the world participating in this event. How would you implement it?

My answer:

Just save every click in the database, sort them by create time ASC, then get the first ten.

My answer was challenged with an additional question:

What if there are multiple server nodes handling this process? How would you ensure the integrity of the data? That what the database would come up with is really the first ten clicks?

That's where I got lost. I just answered let these nodes operate on a single database, or check the logs across nodes the compare them, blah blah blah. But it seems it's not the answer that was looked for.

Why doesn’t my answer address the issue and how could this scenario be solved.

PS: I was applying for a team that handles the finance/payment system of our company so I think this is something related to what the team does.

Answer 1

They are trying to get at your knowledge of distributed systems. One of the tenets of distributed systems is unreliable clocks. You cannot count on clocks being in sync on different nodes, and you cannot reliably measure how out of sync they are.

There are algorithms to gain consensus on an order of events. One common way to agree on an order in practice is to push the events to Kafka. However, that's just an agreement on an order, there's really no way to ensure the agreed order is the actual real life order. In particular with Kafka, buffering for efficiency adversely impacts fairness.

I think most interviewers would be looking for you to acknowledge the impossibility of being perfect, lay out some reasons for that, then try to design something as fair as practical.