7

I have been in long pursuit of an XML-based query-able data store, and despite continued searches and evaluations, I have yet to find a solution that meets the my needs, which include:

  1. Data is wholly contained within XML nodes, in flat text files.
  2. There is a "native" - or at least unobtrusive - method with which to perform Create/Read/Update/Delete (CRUD) operations onto the "schema". I would consider access via http, XHR, javascript, PHP, BASH, or PERL to be unobtrusive, dependent on the complexity of the set of dependencies.
  3. Server-side file-system reads and writes.
  4. A client-side interface element, accessible in any browser without a plug-in.

Some extra, preferred (but optional) requirements include:

  1. Respond to simple SQL, or similarly syntax queries.
  2. Serve the data on a bare bones https server, with no "extra stuff", either via XMLHTTPRequest, HTTP proper, or JSON.

A few thoughts:

What I'm looking for may be possible via some Java server implementations, but for the sake of this question, please do not suggest that - unless it meets ALL the requirements. Java, especially on the client-side is not really an option, nor is it appealing from a development viewpoint.*

I know walking the filesystem is a stretch, and I've heard it's possible with XPATH or XSLT, but as far as I know, that's not ready for primetime, nor even yet a recommendation. However the ability to recursively traverse the filesystem is needed for such a system to be of useful facility.

At this point, I have basically implemented what I described via, of all things, CGI and Bash, but there has to be an easier way. Thoughts?

8
  • 5
    -1 because half your question is a rant against what you see as the bad name XML has gotten. It adds no value to the question, so I suggest you just remove it. Jul 24, 2011 at 22:14
  • 4
    @Dean You have enough reputation to edit. That's the whole point of allowing other users to edit questions - to make them viable to the community. As long as the author's original intent is not disturbed, I have no problems editing other people's postings. If more people took this approach, we would have far fewer closed questions, I think.
    – Thomas Owens
    Jul 24, 2011 at 22:27
  • My rant gives some background to the elusive nature of the answer, and a context as to why, although counterintuitive, that a basic functionality such as what i described is not readily apparent within a platform as mature, as well-researched, and with as high expectations attached to it...
    – alex gray
    Jul 24, 2011 at 22:39
  • 1
    @Alex Rants are not acceptable here. See the FAQ (programmers.stackexchange.com/faq#dontask). If you have a question, ask it with only the background necessary to answer the question, minus any ranting or raving.
    – Thomas Owens
    Jul 24, 2011 at 22:55
  • 1
    Actually, I'm having a hard time understanding what Alex is actually trying to do. @Alex are you looking for an out-of-the-box system that stores this data for you? Or are you implementing some sort of web application that you want to use a framework for? If anyone can clarify this question I would greatly appreciate it. Jul 25, 2011 at 2:31

3 Answers 3

3

There are native XML-Databases out there, eXist might the most famous of them.

They're easily queryable and modifiable using the standardized xQuery language. You can also use all the other XML-Features like xPath or xslt with them. Full-Text search is easily integrated via Apache Lucene.

The downside for your requirements is, that it is Java-based. But you can just use it on the server-side and access it with whatever language you like, as it's standard method for retrieving data is basically a REST-Service. Some languages also have already libraries to get access to the eXist-Datastore, like PHP and PERL.

If that sounds good to you, take a look at the eXist feature sheet. I'm pretty sure it statifies your requirements.

2

Storing the data in xml files seems like a really terrible storage mechanism from an engineering perspective and that's probably the main reason there is no standard solution to what you're asking for. Most people will store data in some kind of relational or non-relational data store and perform the translation from data to xml on the fly if necessary. Databases are designed for fast CRUD operations and a lot of research has gone into making them so. Reinventing all that and bolting it onto a file system is a little silly.

8
  • I don't really like the word "NoSQL" here because NoSQL could mean anything - including XML. There are specific types of data appropriate for mid-to-high-scale applications: Hashtables/key-value pairs, serialized objects (MongoDB uses BSON, CouchDB uses something similar), and tabular data stores (Hadoop/HBase). There are also other types for more specialized applications, e.g. CDF, HDF, etc. None of them use XML on disk though.
    – Aaronaught
    Jul 25, 2011 at 3:23
  • 1
    @Aaronaught: I changed the wording but my point was that disks and how data is arranged on disks is not in general optimized for database type access.
    – user7146
    Jul 25, 2011 at 5:35
  • 1
    I'd also like to state that the choice of the data storage engine highly depends on the data as well. Many datastructures do not fit the relational model at all, translation are cumbersome and hard to maintain. Especially if you have to deal with hierarchical data which are to contain other arbitrary data structures on any level. Using a relational database is horrific in such scenarios.
    – Falcon
    Jul 25, 2011 at 8:18
  • 1
    @Falcon: true, but I don't think XML is the answer, either. At least, not as the on-disk format. In my opinion, the fact that they exist doesn't make them a good idea... ;) Jul 25, 2011 at 8:46
  • 1
    @Falcon: XML really doesn't make sense there. A document database such as MongoDB or CouchDB would give you all the same benefits as XML flat files but without most of the headaches.
    – Aaronaught
    Jul 25, 2011 at 13:32
2

Have you had a look at and/or evaluated these XML databases?

a data persistence software system that allows data to be stored in XML format. These data can then be queried, exported and serialized into the desired format. XML databases are usually associated with document-oriented databases...

one reason for the use of XML in databases: the increasingly common use of XML for data transport, which has meant that "data is extracted from databases and put into XML documents and vice-versa". It may prove more efficient (in terms of conversion costs) and easier to store the data in XML format...

1
  • +1: Would you mind if I included that link in my answer as well?
    – Falcon
    Jul 25, 2011 at 8:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.