Pipeline design pattern and immutability

Question

I'm developing a backend service that is supposed to process items in a pipeline-fashion. Each stage is essentially a Function<IN, OUT>. So the current stage's input is the previous stage's output. Moreover, each stage needs to write it's result to the Record object and each stage can get the previous results (not only the previous one)

The problem with this solution is that we break somewhat crucial rule - Immutability. This Record object is being built one step at a time but it comes with the risk that the state (=the model) is shared between all stages.

So I thought about three options:

1. Accept the limit that a stage can't access the Record but rather only get as input the previous result
2. Limit the setters to package-level access modifier a stage can only get data but not to set (and hopefully the data itself is immutable too)
3. Accept the risk of mutability

So what do you think is the right design for my problem?

Answer 1

Pipes and Filters

The first point is that IN is the model. A given filter can only access this data.

As far as the filter is concerned, anything it is given is perfectly mutable, and it can change it as much as it wants, even write that data to its own output.

No two filters have to share a model. If they do then there are implied dependencies that filter 1 results are are available to filter 12, but nothing in the architecture will enforce this. A single filter could generate all of the models data and make it appear as if 11 filters had been run before. Each filter is independent and expects a model with data in a specific state.

Misconfigure the pipeline and you will simply get garbage. There is no way to prevent his. But you could if you desire add validators that can be run during say debug mode to verify that data being passed through is infact correct.

The immutability comes in via the Pipe. The pipes job is to ensure that the data provided to it for serving to the next filter has no hold over owners. If that means it must duplicate the data then so be it.