We host a dashboard where users can view different reports, each powered by a templated SQL query. The users have an option to "download" those reports as CSV.
Currently the above use case is handled in the following steps,
- User views a report and she clicks on download.
- The browser makes an API call, and on the server we save the output of the same SQL query in a csv file. (Using some trivial logics)
- We save the file on a distributed gluster FS. This fs is mounted on all the prod instances.
- And we return a download link after the above operations as the response of the API call.
I think the above approach is too synchronous and gluster is becoming an overhead to maintain.
We generally have lots of download requests, but each file is around ~2-3KB.
Does the above approach is the most optimal way to solve this problem? I also wanted to get some ideas on how Google Analytics provides this same kind of feature. Is there a way I can remove gluster while still not overloading the client to generate a CSV?
GlusterFS is a distributed file system which is mounted on our prod servers. We have configured it to replicate, so if any of the instance writes on the mounted path, the changes are available on all the instances on the same path.
We are facing issues with this approach of mounting and maintaining this distributed file system. We face issues like gluster gets unmounted on one of the many prod instances and if the load balancer directs requests over to that instance, it raises errors.