1

I'm aggregating a lot of data from different vendors and have to provide an unified data set to a set of clients. The clients will be identified with a specific identifier upon fetching the data from CDN, however, i'd like to distribute all of the costs associated with running this CDN to the clients. In other words, there's going to be a single CDN endpoint, which is going to be accessed by multiple clients.

What i'd like to understand if and in which way could I measure the amount of requests associated with each client. I tried searching for "monitor CDN usage", but all of the information was mostly about fault tolerance and debugging, but there wasn't any info if any of the CDN providers would offer some sort of monitoring services built for the purpose that i'm after.

Thanks :)

6
  • Doesn't the CDN provide metrics/analytics or a traffic report? Maybe you can download that and calculate some numbers based on the path in the URLs being accessed. Commented Jun 10 at 12:01
  • learn.microsoft.com/en-us/azure/cdn/cdn-advanced-http-reports : check by browser
    – Ewan
    Commented Jun 10 at 12:15
  • ahh wait, do you mean by customer!
    – Ewan
    Commented Jun 10 at 12:16
  • which CDN are you using, most seem to support ksome kind of token auth and once thats enabled I would assume you will be able to generate reports grouped by the auth id?
    – Ewan
    Commented Jun 10 at 12:21
  • The service is new and I can pick any CDN solutions, i'm just unfamiliar with the possibilites and setup options. I've heard feedback that filtering based on query parameters on CDN level to provide output is not possible, though it seemed like a strange thing.
    – Banana
    Commented 2 days ago

2 Answers 2

1

If you have a limited number of clients, one thing you could look at is requiring them to download the data over a separate VPN which you provide them with one VPN account per customer. Many VPN services provide usage monitoring which you can hook into. Ideally, you would also want to configure the VPN service to either be a partial VPN or otherwise limit the VPN traffic to only talk with traffic going to the CDN in order to avoid your users accidentally leaving the VPN on after downloading their data.

However, if you don't want to use a dedicated VPN company and want to use a CDN from a cloud provider like Azure, Google Cloud, AWS, etc., you would probably be better off routing things through something like AWS Lambda/Azure Functions and tracking data usage that way or even better by doing the monitoring using metrics from the CDN metrics if those are available.

The reason why a separate VPN might be better than proxying the CDN data with a server is because you don't need to keep dedicated servers around for the VPN.

One more thing you could do which would be the most complex, easiest for customers to exploit, but potentially the least expensive of all these options (short of tracking usage via the CDN itself) would be providing your own custom download client/SDK for this data. It would first authenticate with a middleman server and issue a request for some/all of the data. The server would then create some sort of auth token to the CDN with as short of expiry time as possible on behalf of the client and then send that token along with the real URL of the CDN. The client would then query the CDN using that token, potentially renewing the token by talking with the middleman again if necessary.

The weird redirect client option is almost certainly the cheapest for large datasets besides doing everything on the CDN directly since there would be almost zero cloud data egress costs.

New contributor
Dyllon Gagnier is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
1
  • The CDN will be accessed by sites which have thousands of users, asking them to download a VPN or do any type of work other than loading a simple website is out of question for the business. I think tracking the function usage within the cloud service is going to be the most optimal option. We are updating the CDN cache with 10 minute intervals and AWS lambda function will be used to convert an inbound HTTP response (with fresh data) into a document which can be served by the CDN. We'll track the amount of those conversions on AWS side.
    – Banana
    Commented 2 days ago
0

OK I've just been looking through Azure CDN and I think you are right, this isn't easily solvable.

While the CDN does offer token auth, so you could secure the CDN calls and force the client to pass a token with an id, it looks like this is not exposed in any reporting. So you can't split the usage by AuthID like you would in app insights or similar reports.

I checked some other CDNs and couldn't find anything, which makes me wonder if this is a deliberate thing to prevent reselling.

You can however setup multiple CNames on a single CDN, so I think my recommendation (without having actually tested this) would be to use the customer name in the address for your website/api https://supercompany.myapp.com and also in the CDN https://supercompany.myappcdn.com This does seem to be exposed in the reports, so you would be able to split the data.

Alternatively, I would be tempted to just look at traffic to my site, split that by customer via the AuthID to get the proportional usage and then just bill the same proportion of the CDN cost. This might not be 100% accurate, but it would be a fair split.

Not the answer you're looking for? Browse other questions tagged or ask your own question.