I am looking to write a Bayes filter that will act as an indicator of topic for a number of topics with a variable number of sources. Given a really big number of RSS feeds and here really big might only be a few hundred only few percent (let's say 2%) are interesting to a given group.
What I want to write (in PHP) is a filter I can train to say this is Topic X and this is not. Then I want selected users to be able to further train the filter. But I want to use only the base training for picking topics to mark as interesting for Topic X and then refine that collection based on the training the user has provided to give the most interesting results for that user by reassessing the probabilities including the user supplied training on the short list per user. Or maybe scoring the short list and picking the "best" items to show first.
So over time I am going to gain a fairly large collection of data (all of the headlines and blurbs from hundreds of feeds) and I am going to have flagged the feed items "yes" and "no" on an unspecified number of topics and have had users have likewise done so.
The headline and blurb will be stored for easy display later but I don't want to hold on to too many items that are "no" (up to 98% I imagine). Given I go with, say, this implementation from GitHub that I found earlier today https://github.com/benwaine/BayesPHP (or can be talked into rolling my own) what is the most efficient way to go about storing the data needed to carry out an assessment? Am I asking for trouble if simply try and store any news headlines etc flagged "yes" but store some sort of summary for the base "no" assessments and then just work it out on the fly each time storing the resultant "yes".
The server I have to work with is already doing a fair amount and is not going to spare me much horse power for this.
So balancing processing efficiency and storage needs what can you recommend?