1

Is there such thing?

I designed a while back a rudimentary form based app for my users.

We receive from our suppliers hardware manufacturing data in XML files: file name is made of eleven fields separated by tildes, with each field having its own meaning.

R&D guys wanted to be able to search each field of the file names so I used regex() with decent results.

Problem is that we have now in the upwards of 2.5 million files. And my app can't hack it anymore.

I looked at Apache Lucene & Solr. Though it seemed like the best solution to my problem, the fields in the filenames are not peers to the file content. Big no-no with Solr.

What is the best way to implement a PHP app with indexing and search capability with such large number of files?

Do I have to buy Zend and use Zend_Search? Is it the only way?

Thanks for your input.

Improve this question
3
  • 1
    I'm assuming something like MySQL's FULLTEXT search is not an option?
    – user11586
    Commented Feb 27, 2011 at 15:28
  • Thanks for your reply. I am a newbie and did not know a darn thing about MySQL's FULLTEXT capability. This is actually what I am working on at the moment.
    – Chris
    Commented Feb 28, 2011 at 18:32
  • Does this not belong on stackoverflow?
    – Chris
    Commented Mar 21, 2011 at 21:59

2 Answers 2

Reset to default
2

When you receive xml files from suppliers, put the information in a database. Thats what they are designed for. You'll have essentially instantaneous searches. If the fields are consistent, a regular old SQL database will do the trick. If each is different, perhaps look at a NoSQL solution.

Improve this answer
1

Well, there's always Xapian, or more specifically Omega. You can integrate it through cgi, and then be able to have lightning-fast indexing and searching of different web server documents.

In this case, you might want to have a look at their guide for how to add support for XML files.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.