Implementing per-user storage usage quota in SaaS products

Question

In a SaaS solution, that allows users to maintain their data on the server, if I have to implement various pricing schemes, where each scheme offers different storage quota, say, Basic subscription gets you 2GB storage, whereas Premium gets you 10GB storage - what kind of architecture one has to use?

1. Do I need to maintain against user profile the quota being used?
2. Will this work only if data is maintained in file system? What happens if data is stored in relational or NoSQL database - how one keeps track of user-level usage of storage in such cases?

Answer 1

1. Add a size field to each thing that is stored. This field could be approximate, but it should be updated to reflect the new size whenever the item is modified.

2. Add a space_used field on the User. Whenever something with a size is added or modified, run a query for all the items the user owns, sum the size fields, and update the space_used field on the user. If you aren't using a relational database, you could try little efficiency tricks like just adding or subtracting the size change of the item they just modified, but this will fail if there is any possible way that a user could modify 2 items at once.

3. Either on the User, or on the Company (depending on who is paying for the licenses) you need to store the space_quota to compare against.

The result:

You'll have to check User.space_used < space_quota before allowing the user to add any items or increase the size of any items, so space_used needs to be pre-calculated to keep the query fast. You need Item.size to later answer the question, "How did we calculate User.space_used because it looks wrong?"

Using such a system, it should be easy to give the user warning when they reach 80% or 95% of their quota.

Answer 2

The academic approach would be to have a discrete, as brief as possible, but complete layer between application and persistence medium (whether it be filesystem, database, or a combination thereof) which counts the size of everything that is added, modified, or removed from persistence and keeps updating the total in the profile.

However, at some point in time you will inevitably run into the problem of having to recalculate everything, so since you will most probably have to write the recalculate-everything code, you might wish (at least in the beginning) to reuse that code and skip the academic solution. So, each time there is an insertion, update or removal, you set a "dirty" flag in the profile, and then you have a separate batch process which looks for "dirty" profiles and recalculates the amount of space occupied, by querying the filesystem, the database, etc.

Of course, this means that quota changes will not be visible immediately, and most importantly, once you have enough customers this approach will be too expensive to be viable, but once you have enough customers you will presumably be rich enough to be able to afford a complete rewrite.

_{Or, go with the academic solution.}