3

I currently have several web applications that offer document upload and storage. I am now trying to create a central document management service that these application can leverage to store and retrieve these documents. Currently the documents are stored on disk. The file name is a Guid and its real file name and datatype are stored in a database.

Am I following the right approach whereby I have a database table that stores the documents information (Name, ContentType, CreatedDate, CreatedBy, SystemId etc), and then store the document in a folder on the local file system of the server ?

When I started storing documents, they were originally blobs in the database. This I found an unfavourable approach. My database was suddenly MASSIVE and impossible to backup and restore. Be separating the documents into the local file system the database is now far simpler and my backup strategy easier. (Backup database, File Diff backup from folder)

Is there a more sensible way to do this I am missing? Is there some open source project that I can leverage to improve the design of my new service that I am now making from scratch ?

Improve this question
4
  • Storing the file on disk and the metadata in the db is very sensible. But other than that its not clear what you are after here. If there's a specific design problem you are encountering, you need to describe it before anyone can suggest a way to fix it. Just asking for program recommendations is off topic here.
    – GrandmasterB
    Commented Aug 6, 2013 at 5:55
  • 2
    Related: programmers.stackexchange.com/questions/152824/… programmers.stackexchange.com/questions/150669/… programmers.stackexchange.com/questions/119488/…
    – rwong
    Commented Aug 6, 2013 at 5:56
  • 2
    In particular see this answer by Thomas which lists the pros and cons of storing large binary BLOB/FILESTREAM into the database. Namely: ACID, ease of backup, size limitation (usually 2GB), portability (not supported by all SQL or requiring a rewrite for cloud computing), etc.
    – rwong
    Commented Aug 6, 2013 at 6:01
  • I was really looking for some ideas or direction. I think FILESTREAM sounds like a great approach considering I'm using MS SQL Server.
    – WebDude
    Commented Aug 6, 2013 at 6:24

2 Answers 2

Reset to default
1

This is probably the best approach.

My only caveat would be that you need to think about backup and error recovery of your document files.

Storing two copies on separate disks would give you roughly the same recoverability as a blob in the database.

Improve this answer
1
  • Thanks James. I think from the comments on my question FILESTREAM seems like a great approach. Regarding backing up my files on disk, I run xcopy scripts to diff the files to backup servers and offsite locations.
    – WebDude
    Commented Aug 6, 2013 at 6:26
0

Depending on the size of your enterprise, as the demands of your content management system increase, you may need multiple instances and so your architecture should be designed in such a way as to target not only a document id, but the content management instance id for which it resides.

The service users should not have to worry about this routing detail, however. Architect a solution where you have some sort of rules engine that figures out the best instance to use to store the document. It could be based on some sort of statistics gathering or monitoring process (perhaps some other scheme altogether) so that the most efficient routing path is used.

The point is that this "routing system" is decoupled from both the content management system instance AND the clients who use them, allowing you to maintain your routing scheme without too much disruption to either the users or the content management system instances.

As the document management process runs over an extended period of time, you will have a better feel about what rules to apply for routing, possibly changing your rules engine.

Your guid would be thus created by your routing system, not the content management system (for example docid + ":" + instanceid). The routing system database that stores metadata should have sufficient information that can be used to support future changes to rules engine (to make it work even more efficiently)

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.