1

I'm developing a backend service that is supposed to process items in a pipeline-fashion. Each stage is essentially a Function<IN, OUT>. So the current stage's input is the previous stage's output. Moreover, each stage needs to write it's result to the Record object and each stage can get the previous results (not only the previous one)

The problem with this solution is that we break somewhat crucial rule - Immutability. This Record object is being built one step at a time but it comes with the risk that the state (=the model) is shared between all stages.

So I thought about three options:

  1. Accept the limit that a stage can't access the Record but rather only get as input the previous result
  2. Limit the setters to package-level access modifier a stage can only get data but not to set (and hopefully the data itself is immutable too)
  3. Accept the risk of mutability

So what do you think is the right design for my problem?

2

Pipes and Filters

The first point is that IN is the model. A given filter can only access this data.

As far as the filter is concerned, anything it is given is perfectly mutable, and it can change it as much as it wants, even write that data to its own output.

No two filters have to share a model. If they do then there are implied dependencies that filter 1 results are are available to filter 12, but nothing in the architecture will enforce this. A single filter could generate all of the models data and make it appear as if 11 filters had been run before. Each filter is independent and expects a model with data in a specific state.

Misconfigure the pipeline and you will simply get garbage. There is no way to prevent his. But you could if you desire add validators that can be run during say debug mode to verify that data being passed through is infact correct.

The immutability comes in via the Pipe. The pipes job is to ensure that the data provided to it for serving to the next filter has no hold over owners. If that means it must duplicate the data then so be it.

| improve this answer | |
  • So your claim is that each IN must be cloned? Isn't a better idea would be to make the model immutable? – yaseco Aug 1 at 13:02
  • 1
    @yaseco My claim is that it is that it is the Pipes responsibility to ensure that data received as the out from one filter is delivered faithfully without retroactive alteration as the input to the next filter. Its not the filters job. Now if the pipe is aware that a given model is immutable, or it has captured the output by value, or some other platform/language specific technique then such a claim is trivial to enforce. If it does not have that special knowledge, then it must fall back on good old copying. To do less, means its not pipe and filter architecture, a close cousin perhaps. – Kain0_0 Aug 3 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.