0

I have a group of 100 clients around the globe, that need to communicate between then self, fast. Each message that each of the clients sends needs to be broadcasted to everyone. The messages will be between 1 to 16 MB. Each client is expected to send one message every half minute.

I was thinking of a tree structure and then using p2p connections, but then it slows down the system.

Another idea that I had is using email protocol(this is where the cheap part come into the picture). There are many email providers that already have a good distribution of servers and can help achieving this task. So it might be cheaper using those solutions.

What will be a good solution to my problem? A cheap fast and reliable way to communicate with 100 clients.

I guess that my biggest concern is that I don't want to build a system and later find out that it takes minutes to deliver a message. Performance is my most important parameter and I need a fest solution.

  • scale is not a consideration at the moment
  • Never ask for "the best way" at any SE site, if you don't want to get subjective and opinion based answers. – πάντα ῥεῖ Mar 2 '18 at 19:14
  • 2
    You already have a scaling problem. Using a worst case estimate, these 32MB per minute add up to 45 GB of traffic each client is sending during a day (in addition to receiving all the other messages). Times 100 clients, your network has a monthly traffic of 131TB. For comparison, on Amazon AWS these 130TB outgoing traffic per month would set you back roughly $10k. Unless I've miscalculated and am off by an order of magnitude. – amon Mar 2 '18 at 19:28
  • 1
    "people" or "software clients" ?? No way would humans be able to keep up with such rate of messages of such size. – Euphoric Mar 2 '18 at 19:34
  • 1
    What is the nature of the data you are transmitting? Have you given any thought to finding ways of thinning this data? – Robert Harvey Mar 2 '18 at 19:39
  • 1
    As for your email idea. I'm sure all email providers that have bandwidth necessary for you have really good ways to detect this kind of "malevolent" behavior. So they would immediately flag you are spammer and either stop you outright or demand explanation. And they won't like your explanation of using their free service as data distribution platform. – Euphoric Mar 2 '18 at 20:11
3

Its the classic triangle problem. you want:

cheap fast and reliable

But you have to sacrifice at least one.

reliability: You could sneakily use other peoples bandwidth by leveraging free tier services such as email distribution lists or dropbox/google drive/amazon free accounts

But you would soon hit the various controls to stop people abusing the services and suffer an outage until you had developed a new trick.

speed: You can use the clients connection via a p2p network, but unless they all have lots of bandwidth, speed will suffer.

cost: You can pay for the bandwidth on a centralised service and have both speed and reliability. But it will be expensive

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.