I'm working on a modular video processing pipeline. It's currently presented as a tree of modules. Each module has a set of "results" and can dynamically request data results from other modules. Each request and result are marked with frame ID. User requests one or more "metrics", i.e. end results, for a set of frames (each frame, or selected frames).
For example, the top of the tree is the video reader. Then, there is a module of scene edge detection, which requires frames i-1 and i from video reader to generate results for frame i. There are modules for various information about video frames: histograms, edge maps, occlusion maps, disparity maps, etc. Many results are shared between modules. Some modules require data from a single frame, some—from a fixed number of consecutive frames; some, given request for a frame, start "exploring" data from neighboring frames in both directions—nobody knows in advance, how much data they will need.
Data requests are handled by a centralized service, which owns instantiations of all modules and knows, what module produces what data. It has the right to cache data for further usage by other modules or output it (in the case of metrics).
It all works well in theory, but in practice, I met some problems. How long should the cache hold the data portions? If I don't cache at all, then the same frame will be read multiple times. Some "exploring" metric (see above) may need it later. If I cache everything, then in the worst case the whole video (or its large portion) will be loaded into memory—which is unacceptable.
This cache is the problem my question is about. I have full control over the sources. Modules can tell the caching service any information they want about themselves.
The perfect solution would never have to generate the same data twice and would keep memory usage at a minimum (get rid of data as soon as possible). Well, I don't expect to hear this perfect solution, but some patterns, which would help me come as close to it as possible.
