Azure BLOB grained access

Question

I have a standalone WPF desktop application which gathers data from Digital-to-Analog converter boards, computes, analyzes this data and then it creates binary files (proprietary format) with the result. There is a pace of one record (basically a dictionary with the resulted data) added each second in the final storage file.

Further, this real-time data must be available to a web application (which displays real-time graphs) hosted in Azure cloud. So, maybe obviously, an Azure Blob storage will be used where final storage files will be stored.

The ideal scenario is to update, as soon as possible, the web application with the new record computed on a machine where the standalone app is running. This means that the standalone app must upload a single record of data at once and not an entire file each second. The web app must be able to read the new record and not only the entire file. So the blobs which are hosted in the Azure blob storage must be read similar with local streams, using APIs like Stream.Seek(int) and Stream.ReadBytes(int).

Are there any possibilities to accomplish this using a grained control over a blob hosted in an Azure storage account?

If not, what other scenarios can be useful? An ASP Web Api 2 application exists for other purposes so maybe it can be leveraged as an intermediate between the standalone app and the web app. Maybe something as a queue?

Later edit

There are two kind of usage of the computed data. First one is described above, computed data must be viewed real-time on a web application. The second one emphasize the need of binary files and blobs: files are downloaded and edited offline. So is mandatory to have files generated and stored in a blob storage.

Answer 1

Why do you want to use blobs instead of record-based storage?

The typical pattern for a data collection and analytics solution is

Collect raw data -> Transform data -> Store data -> Analyze -> Display

Azure has a set of services specifically designed to handle these steps very efficiently (like thousands of transactions per second):

Event Hub -> Data Factory -> DocumentDB or SQL DB -> HDInsights -> PowerBI or Web App

For lower volume applications you can use the general-purpose services which would probably be simpler to configure:

Web API or Storage Queue -> Web Job -> DocumentDB / SQL DB / Table Storage -> Web App

Answer 2

Yes you can do this by using appendBlobs which allows you to append bytes to an already existing blob. However I don't think that's what you want, append blobs are limited to 50 000 appends and after that they won't work anymore. So in the case of using an appendblob you need to make sure that you blobs are short lived. I suggest you use the suggestions of BenV if you want to do this the "azure way".

Other suggestions is to use a virtual machine and implement your own "blobStorage" service on it. (it sounds complex, it really isn't). I recently tested this with a similar problem and got very good results. A virtual machine in azure can have a blobStorage as a backend but it's optimized so that you can append to it in a more efficient way.