3

Please bear with me as I find this tricky to articulate exactly what I am after (probably not a good sign..).

I have several services set up which can be combined in a particular order to create workflows. For example, imagine I have the following services:

  • 1) HTTP -> File Store (takes a file over HTTP and stores it)
  • 2) FTP -> File Store
  • 3) File Store-> FTP (takes a file over HTTP and pushes to FTP)
  • 4) File Store-> Transformation -> File Store(applies a transformation to a file in File Store).

I might want to combine services to create a particular workflow, for example.

1) -> 4) -> 3)

Or

2) -> 4) -> 3)

And it might be the case that I perform exactly the same operation e.g. in the workflows above, step 4 could be applied on the same item twice, if the same item is sent over HTTP and FTP. I might also want to trigger subworkflows for example:

4) -> 3)

Would be a valid workflow. Is there a design pattern/known approach to handling these 'cascading' workflows? I would like to avoid duplicate operations where possible but also design workflows in a separate, contained manner. The only thing that comes to mind for me is some sort of composition based approach where workflows are composed of several sub-workflows.

I hope this makes sense!

  • Actor Model: Akka, NServiceBus. – Den Dec 11 '14 at 10:51
  • Hi Den, can you go into more detail about a particular design pattern? – Alex Dec 11 '14 at 10:54
  • 1
    It's not a design pattern, it's a paradigm. Represent coarse reusable functionality as actors, decouple them by message-based communication, combine as needed. Also I am not sure what is a "workflow" for you - chain of actions or just one action. – Den Dec 11 '14 at 11:00
  • @Alex, Perhaps something like a state monad is what you're after? – blz Dec 12 '14 at 18:29
1

Not sure really whether this will help but in F# you could create a computation expression something like the following,

 let workFlow = 
      workflow {
         let! pageContents = ("read url", readHttp "http://www.google.com")
         do! ("write file", writeFile @"c:\temp\google.html" pageContents)
      } |> runWorkflow

This can be built from the following functions

 let runStage (name, f:Async<'a>) = 
     async {
         try
             do printfn "Running %s" name
             let! result = f
             return Choice1Of2(result)
          with e ->
             printfn "Stage %s failed %A" name e
             return Choice2Of2 e
     }

 type WorkflowBuilder() = 
     member x.Bind((name,curr), next:('a -> Async<'b>)) = 
             async { 
                     let! result = runStage (name, curr)
                     match result with
                     | Choice1Of2(r) -> return! next(r)
                     | Choice2Of2(e) -> return raise(e)
             }
     member x.Return(a) = async { return a }
     member x.ReturnFrom(a) = a    

 let workflow = WorkflowBuilder()

you can then compose simple async workflows yourself to get the desired result.

 let readHttp (uri:string) = 
       async { 
         let req = HttpWebRequest.CreateHttp(uri)
         req.Method <- "GET"
         let! response = req.GetResponseAsync() |> Async.AwaitTask
         let sr = new StreamReader(response.GetResponseStream())
         return sr.ReadToEnd()
      }

 let writeFile (path:string) (contents:string) = 
      async {
          let fs = File.OpenWrite(path)
          let sw = new StreamWriter(fs)
          do sw.Write(contents)
      }

Of course this implementation is language specific there is nothing really stopping a translation to C# or Java. The general design pattern being used here is a monad, put simply it is just some wrapping type (in this case Async<'a>) plus, two functions (bind and return), which you can see implemented in the WorkflowBuilder type.

I should mention some work will be required to productionize this code, but the idea is more important I feel.

|improve this answer|||||
  • Isn't the problem just composing of functions? I mean, all readers take an Uri and return a thing Then you transform the thing Then you save the thing. i.e getFromFile srcUri |> transform |> storeFtp destUri – Lazydev Dec 11 '14 at 12:18
  • Which is really all this does, but adds things like protecting the call.. – Colin Bull Dec 11 '14 at 12:28
1

What you are looking for is the Chain of Responsibility pattern, which allows you to compose a pipeline of objects that can act on a message.

In the chain of responsibility (one version of it), you have a Handler interface, and your individual behaviors implement that interface, and have a reference to the next handler. Pseudo-code, and probably not specific enough for your needs, but good for example:

interface Handler {
    protected Handler nextHandler;

    protected void pullData(string name);
    protected void pushData(string name, string data);

    public void process(string name);
}

So you would have an HTTPHandler implementing the interface:

class HTTPHandler implements Handler {
    public HTTPHandler(Handler nextHandler) {
        this.nextHandler = nextHandler;
    }

    protected void pullData(string name) {
        string data = getDataOverHTTP(name);
        if(!this.nextHandler == null) {
            this.nextHandler.pushData(name, data);
        }
    }

    protected void pushData(string name, string data) {
        postDataOverHTTP(name, data);
        if(!this.nextHandler == null) {
            this.nextHandler.pushData(name, data);
        }
    }

    public void process(string name) {
        this.pullData(name);
    }
}

Other classes can be built up accordingly, then composed together. Each Handler implementation knows to call the next handler in the list. And, the Handlers can be composed together dynamically at runtime. With one call to process on the first Handler, the data will be sent all the way down the pipeline. This article details a fuller example of this way of chain of responsibility.

(FYI: An alternate way of doing this could be to create a collection that encapsulates a list of Handlers, and passes data between them using an intermediate data object.)

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.