2

I've been struggling with refactoring my code to use a generalized design pattern for a theme I see being repeated. The biggest struggle is that each instance of this repeated code uses different types. I have a tendency to keep code generic, but I haven't been able to find a way to support this with generics, even though the structure of the code and its types are known at compile time.

My C# console application operates against different logical parts of an HTTP API. To provide a contrived example, suppose this remote API provides access to databases for movies and books. And my application is interested in updating these databases. I have a pipeline for updating movies and another for updating books. Both share the following steps:

  1. Load a cache of metadata, if needed. Not everything has a cache so this step may be optional / no-op.
  2. Read information from a configuration file provided by users which tells the application what we're updating.
  3. Determine if this pipeline needs to be processed, exit if not.
  4. Invoke a REST API to obtain information from the DB
  5. Pass the cache, config info, and service information to an object to calculate transactions that will take place (i.e. what we will update and how)
  6. If only a preview was requested, invoke an option to render a preview and do not process further steps.
  7. Invoke a REST API to update the database based on transactions calculated.
  8. Save any updated metadata to the cache. Not everything has a cache so this step may be optional / no-op.

The steps above represent what I feel is close to a pipeline design. In general, outputs from previous steps feed into the next steps as inputs. There's a bit of cross-sharing here and there, but that's generally the flow.

Some steps may be optional depending on what we're processing. For example, for processing books, we might have a metadata cache. But for movies, we do not have a cache.

In this contrived example, I have a BooksPipeline and MoviesPipeline classes that each implement the steps above. In reality, though, these steps are currently repeated for 6-7 different logical categories. There's definitely a theme here and perhaps some opportunity for code re-use and interfaces to help enforce the pattern.

What I am trying to figure out is how I can implement a GenericPipeline class of some kind that lets me pass in units of work, such as CacheStep, ConfigurationStep, ObtainApiDataStep, TransactionStep, PersistApiStep, etc. and each of these will be implemented differently for Books vs Movies.

Assuming I'm on the right track here, I'm stuck on how to do this in a way that doesn't involve throwing away types (i.e. doing tons of downcasting or boxing/unboxing of object). It seems like I'm forced to resort to runtime solutions for this even though at compile time I seem to know all the types and steps that will take place.

What would be a good blueprint to move forward with a generic and reusable implementation for this pattern?

1
  • Are you asking if it's possible to do this with generics, or if it would be cost/effort effective to do so? If the latter, that is a question that needs significantly more consideration than you've added (or, in my expectation, can possibly add). I would be thinking of things like the volatility of individual types' implementations, your developers' ability to handle a significantly more abstracted codebase, the added design constraints and future maintenance complexities, ... I'm not saying you shouldn't do it, but I'm unconvinced that you're not just chasing an ideal dream here.
    – Flater
    Oct 6, 2023 at 22:30

2 Answers 2

1

Read information from a configuration file provided by users which tells the application what we're updating.

This tells me you don’t know which pipeline you need until this config file tells you.

Which is fine so long as you don’t mind generic code reading it.

Once it’s read you can invoke whichever pipeline it calls for. This could be done with everything from a conditional structure (if, switch), to a hashmap of strings:objects, to a full blown class loader.

Something, somewhere, has to know concretely what you’re doing. And only this will know exactly what type you’ll get out of it.

Which is why the generic code can’t know. All it knows is the stuff common to every pipe.

7
  • What I mean is, a configuration file is capable of specifying movie or book information. If that information isn't there, then early on in the "pipeline" that handles the respective logic for it, it bails out early because there's nothing to do. But code-wise, the work we do to update books or movies is fixed at compile time. I hope that clarifies. Oct 6, 2023 at 20:01
  • Actually no. Every pipe will need to handle problems gracefully. What does that have to do with the question? Oct 6, 2023 at 22:32
  • It has more to do with your answer. You stated "This tells me you don't know which pipeline you need...". But that's not true, because every pipeline runs. The pipelines introspect the configuration and decide to exit early or not if needed. But the steps they take are "fixed". Oct 7, 2023 at 18:46
  • @void.pointer so rather than run the appropriate pipe you run them all? Oct 7, 2023 at 22:32
  • That's correct. For no other reason than I just decided to let each pipeline determine when to early-out or not. Keeps that logic with the pipeline itself. Oct 8, 2023 at 23:50
1

It's not always cut n dry when you should use DRY.

As your context grows to larger code fragments, you start to run into more exceptions - block of code A is the same as B except for special cases X,Y and Z. Sometimes you can find a clean way to incorporate the exceptions in your overall design, sometimes it's just easier to have some code repetition.

Since you already have a working code base I would start with a bottom up iterative refactoring. Specifically take each of the 8 steps you outlined above and try to generify each of them individually - don't worry too much about consistency between the steps. Since you are working in smaller chunks it should be easier to standardize each (with a minimal number of special cases).

Once thats done, shift your context to the main workflow, it is likely there are groups which can be standardized/generified easily for example 3 consecutive steps that make a good candidate for pulling out to a new method

Finally pull back to the overall process, realistically one of two things will happen:

  • It will be the case that there are significant similarities between different domains (books vs movies) and you can fairly easily genericize it.
  • The flows look totally different because all the similarities have already been pulled out by the earlier steps - if this is the case STOP .
1
  • The 8 steps for each of books and movies are not identical in any way. It's really only the workflow above that which pulls the 8 steps together that starts to show some similarity. Oct 6, 2023 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.