I'll be receiving tens of thousands of XML files, which will continue to grow with new files over time. I need to be able to query these files as a whole (like SQL querying all the records in a database). What technologies and/or approaches should I be looking at for this problem?
I feel like I should be looking into XQuery. I'm not that familiar with it, though.
Example queries I might run:
- Return all the product code, product name, product description from all XML files in /folder 1/ where the supplier id = x and store id = y
- Return all the product code and product price from all XML files in /folder 2/ where the supplier id = x and store id = y
- Maybe join the results of the two queries above, joining on product code
- As there may be multiple results per product code for the two queries above, only get the most recent values based on some tag / attribute in the XML
It seems like this article suggests that XQuery can be used: http://www.ibm.com/developerworks/xml/library/x-xquerymaven/index.html?ca=. As I am implementing a LAMP solution, I think there might be XQuery libraries available to PHP. Does this mean that I don't need an XML database to query loads of XML files?