1

I'm trying to determine how best to accomplish the following scenario using the Azure Service Bus. I have millions of devices (desktop software, mobile phone apps, etc) that need to have messages sent to them based on subscriptions. This in and of itself is pretty easy. However, the Service Bus has limits on the number of subscriptions per topic, filters per subscription, etc. (listed below). So, that being said, I was wondering if someone has solved this problem before or if there are best practices or whatever.

Assumptions

  • 20,000 customers
  • each with 1-* appliances
  • each appliance with one customerId, applianceId
  • each appliance manages 1-* devices
  • each device with a deviceId

Requirements

  • discover appliances by customerId (filter by customerId)

    • When a customer logs in we need to "ping" all appliances belonging to that customer.
  • commands to specific appliance (filter by applianceId)

    • Once we know all of the appliances used by the customer, we can execute commands against specific appliances.
  • commands to a specific device (filter by deviceId)

    • We can also provide commands against a specific appliance for a specific device

Now, none if this is very difficult. I'm really just bumping up against some of the Service Bus quotas as seen here: http://msdn.microsoft.com/en-us/library/azure/ee732538.aspx

  • 2000 filters/topic
  • 2000 subscriptions/topic
  • 10,000 total topics and queus per namespace

So my first inclination was to have a single SB Namespace with a single a single Topic and a single Subscription. Each appliance could then simply add or utilize the following SqlFilters:

  1. Appliance by CustomerId - this would allow for messaging with all appliances that belong to a specific customer.
  2. Appliance by applianceId - this would allow for messaging with a specific appliance.
  3. Appliance by deviceId - this would allow for messaging with a specific appliance for a specific device.

Using the assumptions stated above that would be 20k filters for #1, a filter per appliance (lets assume it's a million) for #2, and potentially millions of filters for #3.

Obviously that wouldn't work. I could even go so far as to disregard #3 which would then still leave me with over a million filters. This is the specific quota that I'm bumping up against.

The most important two things here are that I can send messages to all appliances owned by a customer and the second is that I can send messages to a single appliance. Appliances and devices communicating back is not an issue.

Queues may figure into this equation but simply replacing topics with queues in certain scenarios will only relieve part of the problem.

Anyhow, let me know if anyone has any ideas!

  • Consider that Azure Service Bus may not be the best choice given the scale/design of this application. Amazon SNS, for example, has much higher default limits (see aws.amazon.com/sns/faqs/#limits-restrictions) but may be unsuitable for other reasons. – Steven A. Lowe Sep 27 '14 at 18:23
  • What quotas are you bumping up against? – Peter Ritchie Sep 27 '14 at 19:23
  • @PeterRitchie I updated the question to better explain the specific quota issue that I'm running into. – devlife Sep 27 '14 at 19:35
  • Maybe use notification hubs to get each device to send a message? I would imagine that notifications to a particular device are rare and not a lot of messages back from devices would be needed at the same time. – Peter Ritchie Sep 27 '14 at 19:55
  • Not necessarily. The two important aspects are that I can reach all devices by customer and to a lesser degree be able to reach a specific device. I would prefer not to send messages to each appliance owned by a customer if I only wanted to send a single message to a specific device. Appliances and devices sending messages back aren't the concern here as they can simply all utilize a single Topic/Subscription. – devlife Sep 27 '14 at 20:10

protected by gnat May 4 '15 at 20:27

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Browse other questions tagged or ask your own question.