0

For a web application we are planning (some kind of product information system). We have to manage all the assets (pictures) of the products.

There are system (Wikis or DAM System) that save the assets in the DB itself or only a reference.

What are the criterias to decide wheter the files itslef it should be stored on the FS or DB?

The system will manage thumbnails and meta data of the assets.

For a PDF production it is possible that we are going to need them on the FS in other cases it is probably better we have them in the DB.

(We are working in a windows server environment with mongoDB and C#.)

When would it be a disadvantage to have it on the file system (in the DB only meta data)?

Or in which situation clearly wins the DB only approach?

Update

The linked questions brings in some good points.

  • the concern about backup issues with larger DBs: mongoDB's GridFS is good for larger data. We could have a separated asset DB with another backup interval to solve the problem of the longer backup process.

  • Serialization and and deserialization to the FS: Could cost a lot of time. In case we need access to them from third party tools (e.g. for PDF production).

Improve this question
1
  • 3
    This question has been here in different forms many times. I'll refer you to my question because there are some great answers there and there's also another similar linked question there. Check it out.
    – Alternatex
    Jan 9, 2015 at 8:46

1 Answer 1

Reset to default
2

It is mostly an opinion-based question (and it is surely operating system and file system specific), however:

you probably want to avoid having a single directory with many thousands of files (it is file system and operating system specific, but some OS & FS may be unhappy), so prefer dir01/subdir02/file0345.txt to justfile01020345.txt

you probably want to avoid having a big lot of very small files (e.g. less than 2Kbytes).

So it might be worthwhile to use a database for "small" contents (e.g. less than half or a few megabytes) and use file for "large" contents (several megabytes each). BTW, you could store in your database some metadata about the large enough file (e.g. at least store something related to the pathname).

You surely need some backup strategy, and perhaps some cleanup strategy too.

Read also about blob-s

Improve this answer
9
  • The point regarding sub directories is a good one. That is why MediaWiki is creating some sub directories based on an algorithm. I think a lot of small files lesser than 2 KByte is not my use case. I think pictures actually in print quality and their thumbnails will take more in almost every case. Having a mixed approach based on the file size seems to be very complicated.
    – Matthias
    Jan 9, 2015 at 9:10
  • Why is that complicated to have a mixed approach? You just need to define a threshold (typically 4Mbytes) above which data goes into its own file!
    – Basile Starynkevitch
    Jan 9, 2015 at 9:12
  • The mixed approach could eventually cause different ways of fetching the data e.g. when needed on the FS. When I am using the FS, I want to have the advantage of having 100% transparent access by the folder structure to all assets. I think it does not make it complicated in general, but in my scenario.
    – Matthias
    Jan 9, 2015 at 9:15
  • 1
    Then, put everything in the database, but limit the blob size to some sensible limit (eg 32 or 64 Mbytes); your DBMS might have some limitations already. BTW transmitting a dozen of megabytes on the web takes some significant time...
    – Basile Starynkevitch
    Jan 9, 2015 at 9:16
  • 1
    Segregating small and large content is certainly not the difficult point of your software. All the rest is much more difficult
    – Basile Starynkevitch
    Jan 9, 2015 at 9:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.