I'm self-taught and I don't have a CS degree. The more I've been learning about data structure, the more I wonder, in this day and age, how are we still saddled with the filesystem, with directories and files, as the basic data storage structure on the OS?

I understand the simplicity of it, but it seems nowadays that there could be more options available natively. As far as I'm aware, the only project to improve the basic functionality of the filesystem was ReiserFS, where you could tell what line of a file was changed by whom, and when.

For instance, if I could have native tagging for files, where I could tag images, diagrams, word-processing documents, an entire code repository, all as belonging to a single project, that would really be helpful to me. Since I'm stuck in the filesystem paradigm, I know that I could put all those into a single folder/directory, but what if they already exist in disparate directories, and they need to stay there? I know there are programs out there that can do this, but why aren't they on the filesystem?

Something that would be nice to have is some kind of relational feature in the filesystem, like you get with RDBMSes. I understand that that was supposed to be part of Vista/7, but that fell off the feature list too.

Sure, any program can store a binary file and have any data structure it wants in it, by why couldn't the OS offer more complex ways of storing data, beyond the simple heirarchy of the filesystem?

  • 2
    The core of it should be simple. The optional bloat you mention should go on top of a simple core. Alternatively, wait two decades and someone will reinvent the notion of a file system.
    – Job
    Mar 16, 2011 at 20:42
  • 3
    "what if they already exist in disparate directories, and they need to stay there?" Sometimes you can use hard-links to solve this problem... Mar 16, 2011 at 20:43
  • 1
    Also, some interesting reading on the topic: c2.com/cgi/wiki?FileSystemAlternatives Mar 16, 2011 at 20:48
  • 3
    Not really a solution in Windows 7 but the new Libraries can give you some of the functionality you seem interested in: lifehacker.com/#!5464350/…
    – DKnight
    Mar 16, 2011 at 21:19
  • 1
    If I want to put a file into two different folders at once, I put a shortcut to that file in one. The disadvantage is that if you relocate that folder/file, the shortcut will be invalid. Mar 18, 2011 at 3:49

6 Answers 6


Start with this: http://en.wikipedia.org/wiki/Unix_File_System

Read this: http://www.unix.org/what_is_unix/history_timeline.html

Then read this: http://www.amazon.com/UNIX-Filesystems-Evolution-Design-Implementation/dp/0471164836

There's a simple answer to "why couldn't the OS offer more complex ways of storing data, beyond the simple heirarchy of the filesystem?"

Because it's too much for the OS to do.

That's what libraries and application packages are for.

Oracle, for example, will sell you a file-system-like set of features that you manage with the Oracle toolset.

Python uses the DBM library to create very sophisticated on-disk storage structures.

CouchDB and Mongo (and others) are very sophisticated storage structures that offer some database-like features.

The point is that the OS should do the minimum and everything is an add-on.

  • 4
    Quite agree. Actually, a lot of what OP was asking for is present in the eaither dead or dying WinFS project: en.wikipedia.org/wiki/WinFS. As much as the geek in says, 'Neat!' the experienced user and software engineer in me says, "Trying way too hard!" Mar 16, 2011 at 21:03
  • 6
    "The point is that the OS should do the minimum and everything is an add-on." Quite a bold statement in an age where some operating systems contain a built-in windowing system, file indexing service, media player, remote desktop, firewall or Netris.
    – biziclop
    Mar 16, 2011 at 21:23
  • 1
    @biziclop: Agreed. Windows has diverged from the Linux viewpoint. Nothing surprising there.
    – S.Lott
    Mar 16, 2011 at 21:26
  • 1
    @S.Lott Don't get me wrong, I agree with your approach, but Windows is saddled with so much useless rubbish anyway, one extra feature won't make a difference. :)
    – biziclop
    Mar 16, 2011 at 21:39
  • 5
    That is the Unix philosophy. It is not necessarily right. It does (and a C complier) make Unix easy to port to hardware. It also makes it simple enough for people to clone Unix into the flavours of -ix likes we find today. If a feature is useful, and all programs need it, like say, spell checked input fields, then there is value in having the runtime environment provide it. We don't need 400 independent versions of a ribbon bar. Mar 16, 2011 at 23:25

The short answer is : Everyday people understand the file system. It reminds them of a file Cabinet. Think about web pages and even Fat apps, why do you think Tabs are so popular? People can identify with them, and understand them quickly.

Imaging trying to teach Grandma to search a DB for File based on property tags.. With the file system, Grandma knows the file is simply where she Put it.

Even with WinFS I don't think MS was going to get rid of the file system look and feel.

  • 11
    I have to disagree with this. Most people who aren't forced to navigate the filesystem don't do it. They open a word processor and click their recent document, or search in Windows 7's start menu, etc. And lots of people lose track of where they put their files. It would be a lot easier for Grandma to search "cookie recipes" or "grandson photos" or whatever than to maintain a folder hierarchy. Mar 16, 2011 at 20:55
  • 18
    This might come as a shock for you: everyday people don't understand the file system. They haven't got the faintest of ideas. And I don't mean a Unix style FS with its mount points, symlinks and hardlinks, but a bog standard directory structure with files in it.
    – biziclop
    Mar 16, 2011 at 21:26
  • 2
    @Morons, my grandmother never knows where she puts things. Gmail already has shifted my desired paradigm to a tagging system, especially with filters to automatically tag things. I think the filesystem paradigm was implemented largely due to the simplicity of programming tree-structures. It also makes addressing easier from a programming perspective. How would you specify a document's location in a tag-based system? Not saying it can't be done, but the details need to be ironed out.
    – zzzzBov
    Mar 16, 2011 at 21:26
  • 3
    Do you purchase your file cabinets full of thousands of folders and documents necessary to the operation of the cabinet itself, which you must navigate through and around but be careful not to touch? Does your file cabinet seem to open to a different location every time you pull out the drawer? Etc. etc. I agree with Matthew and biziclop - "Everyday" people don't get it.
    – Nicole
    Mar 16, 2011 at 21:51
  • 2
    I have a CS degree. But I don't know into which folders any Windows does put what files into. Especially Desktop, StartMenu, QuickLaunch, and all the other user/system specific default folders. (That M$-Help system does not help in explaining to me how to press a button.) I need to install CygWin to be able to search for my own files, because newer M$ search features do not find simple existing files anymore like on win2k. Disabling misfeatures like hide-system-files, hide-file-extensions don't solve most problems anymore. I gave up Windows, when I was forced to work on the (brand new) winXP.
    – comonad
    Mar 17, 2011 at 3:54

There's a little truth in every answer here but I don't think it's the whole truth.

What you list are mostly features that are sorely missed every day by users and developers alike.

People don't understand the tree-based file system any more than they would understand a DAG-based one.

And there's absolutely no excuse for the pathetic appendages of file names called extensions. They are not only completely unsuitable for their purpose (identifying the file type) but also an endless source of nuisance for users.

The reason we're still using them is a mixture of a "that'll do" attitude and the real need to maintain compatibility with older code. A new approach to storing files would mean a radical change in basic file I/O API, rendering most existing code useless. Either that or you have to tiptoe around them, maintaining the legacy API. Remember PROGRA~1.

I think for the reasons above, although the future could hold more specialised file systems for special applications, but while the desktop and laptop PC architectures of present survive, we're stuck with the largely tree-based file system with its lack of metadata and its horrible little extensions.

Now I'm going to switch sides.

Because it's all around us, we never really appreciate how mind-bogglingly powerful the tree metaphor is. On my hard drive I've got several hundred thousand files. If I have to find one, it rarely takes more than a minute, even if I know very little about the file. Now imagine the same task without any structure, just a flat list of names, scrolling endlessly.

Yet all the operations are straightforward, there's no spooky action at a distance, nothing that would make me go wtf.

Actually, I did implement a document store with rich metadata and a DAG-based hierarchy once. (It wasn't even a free-form DAG, it was strictly a two-level metastructure and the documents, which could be children of either a level 1 or a level 2 collection. So it's really simple.)

Obviously, the requirement that document names should be unique within a collection had to stay.

And then the problems started to flow. What if you open a collection and change the name of the document to something that clashes in a different collection the document also belongs to? We displayed an error message but the users were completely baffled. (These are the very same users who had asked for this requirement.)

They tried to delete a document, but all that did was remove it from the collection. So it still showed up in search results. We tried it the other way around too, but then they were complaining that they deleted a document from collection A and it magically disappeared from collection B. So we needed both an "unlink" and a hard delete operation.

Eventually we conceded defeat, fortunately still in time.

The extra search facets the metadata made possible worked an absolute treat though.

  • Rememebr CP/M on a 5 MB hard drive? Hundreds and hundreds of files scrolling past. AWFUL! Mar 17, 2011 at 3:01
  • @quickly_now Ah, the good old CP/M. :)
    – biziclop
    Mar 17, 2011 at 10:41
  • 1
    I think the part about deleting vs unlinking touches on one of the biggest factors here: location is a very pervasive metaphor, because it's something we use all the time in the real world. However hard you try to tell users that a view "finds" their documents rather than "contains" them, they will re-invent that metaphor, and think of the document as being "in" that location. The idea of "your document is in exactly two places, and unlinking from both will erase it from the disk" doesn't match anything in every day experience. So metadata always comes on top of location, not instead.
    – IMSoP
    Nov 3, 2021 at 18:29

To be honest, I barely touch metadata on my files on the Mac. I think in the last 5 years of using OSX (which supports comments and so forth), I've used metadata on maybe 2 files. Not saying it's a bad idea.

I'm just not sure how the overhead of tagging is pragmatic for me.

I think the nicest filesystem feature that I know of would be a filesystem level versioning system... that works cross partitions. It was done on VAXen in the 70s and early 80s, not sure why it didn't catch on with Unix and NTFS/Windows.

  • Modern versions of NTFS/Windows do offer versioning. It's not exactly in-your-face, but it does exist. Can't say how it compares to VMS though.
    – Shog9
    Mar 17, 2011 at 4:45

I've worked with non-hierarchical file systems on older minis like HP3000 and Encore/Gould. You didn't have directories; you had a group and an account, and files were named as "group . account . file", like "users.jbode.myfile1", "dev.jbode.main", etc.

Now, these are old systems, where individual disk space quotas were in the single megabytes, so it's not like you needed too many levels to organize your stuff, but from a user's and programmer's perspective hierarchical systems are much nicer.


I don't see where (at least some) current file systems really need to do much [Edit: anything, to be honest] to support tags. When you get down to it, supporting tags means little more than some extra data associated with a file, but isn't written into the stream of bytes for that file.

NTFS (to choose one example that's in wide use) can do that just fine: as far as NTFS cares, a file isn't necessarily a single stream of bytes. On NTFS you can associate an arbitrary number of streams of data with a single file name. Each file has a (possibly empty) "primary stream" that doesn't have a name. It can also, however, have an arbitrary number of other streams, each of which must have a name. Using this, it would be truly trivial to add a stream named (just for example) "tags" to an existing file, and (obviously enough) write your tags to that stream.

After that comes the somewhat more difficult part: getting your tools to make use of the tags you put there. Ideally, you'd probably want to index them for fast searching, so you'd be able to do things like create a "virtual directory" of all the files with a specific tag.

At least from my perspective, the file system already has what's needed though -- it's supposed to store and retrieve the data, and it can do that perfectly well right now. Making use of that data is the job of other tools. Those tools don't currently exist, but the file system infrastructure to support them does.

If I'm allowed to be cynical for a moment, I'd say it was inevitable that this feature of NTFS would remain almost completely ignored and unknown. After all, it's simple to use and doesn't require any special API or anything else. You can use it quite nicely in completely portable C, C++, or anything else that will let you specify an arbitrary file name. Here's a quick bit of code to demonstrate creating a file with an AFS:

#include <fstream>

int main() {
    std::ofstream out("test.txt");
    std::ofstream tag("test.txt:tags");

    out << "This is the output file";
    tag << "tag1 tag2";

    return 0;

And, here's some code to read and display the tags:

#include <fstream>
#include <iterator>
#include <iostream>
#include <string>

int main() { 
    std::ifstream tags("test.txt:tags");

          std::ostream_iterator<std::string>(std::cout, " "));
    return 0;

All very simple and easy. Note that although I've only written a trivial bit of data there, you can treat an AFS just like any other file -- all the usual "stuff" works just like with anything else. In a normal directory display, all that will show up is the primary stream (e.g., the size shown for the file will be the size of the primary stream), but if you want to see it, dir can show information about alternate streams as well with the /R flag. For example, a listing for the file created above looks like this:

03/16/2011  08:22 PM                23 test.txt
                                     9 test.txt:tags:$DATA
               1 File(s)             23 bytes
  • 1
    DIR might be able to show it, but backup of a file with alternate streams is horribly difficult, especially to some other system. For example most NAS drives today use Linux, and the file systems there don't handle alternate streams at all. Copy the file over... and all the alt stuff just disappears. Mar 17, 2011 at 3:04
  • Yes, I have noticed that most NAS systems are rather...challenged (and this isn't the only way either). For actual backup and restore kinds of things, it doesn't cause a problem though (at least if the software in question is competently written at all): BackupRead will serialize all the streams, and BackupWrite will reconstitute the file (with alternate streams) from the serialized format. Mar 17, 2011 at 4:14
  • Depends if you want the backed-up files to be directly readable on the NAS. If you do (and avoid the need for special restore programs) then you are stuck with plain-ole files. Mar 17, 2011 at 8:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.