Is there a protocol or a convention that supports REST (ok, maybe we should use HTTP here instead) processing chain and some neat features to help with that? Let me explain what I mean.

Let's assume I have some public REST service available. Using HTTP GET, I have multiple static pictures, GIFs and movie clips available. Generally, I would like to take this data and send it to another REST endpoint, along with additional data about the recognized visual elements in the content. For example, if the image contains Steve Ballmer drinking tea, a description "Steve Ballmer drinking tea" is normally expected at the endpoint.

However, I don't have an image processing and recognition service available, but if there are some such services available somewhere on the internet, I'm happy. Even if one works exclusively with static images and another one with movies.

So, my application (let's call it MyApp) will do the following:

  1. Call service (let's call it Src) to retrieve the picture/GIF/video or whatever
  2. If that resource is a picture, send it to the picture recognition service, let's call it PRS, and retrieve a picture description
  3. If that resource is a video, send it to the video recognition service, let's call it VRS, and retrieve a description of the video
  4. Combine the picture/video content with description, pack it inside an archive and send it to the endpoint that expects the result, let's call it End

This means the data flow is:

MyApp -> Src -> MyApp -> PRS (or VRS) -> MyApp_> End -> MyApp (confirmation)

I am looking for the solution where data flow is this:

MyApp -> Src -> PRS (or VRS) -> End -> MyApp

This means that I only have to say to the Src: "Get the whatever video resource I want and forward it to PRS or VRS depending on the content; after that forward it to the End". Then Src takes the picture, sends it to PRS and says "process this, after that forward it (along with result of processing) to the End". You see, I don't want MyApp to be orchestrator of everything, additionally creating extra network traffic along the way.

Oh, btw, since I want it to be neatly archived, I need a zipping service in the chain, so the solution should look more like this:

MyApp -> Src -> PRS (or VRS) -> Zip -> End -> MyApp

One more thing is that I want MyApp to be informed about the percentage of processing, errors. I expect some asynchronous processing somewhere along the path (e.g. VRS is a good candidate) and everything to work correctly in that condition as well.

Does something like this exist? Something maybe most similar to the Unix/Linux piping. Like "web-pipe". Or something. If it does, I can't find it.


I am looking for a protocol, convention, whatever fits my need and is neither tied for an existing framework (e.g. Spring, .NET MVC/WebApi, ...) nor a "proprietary" part of some existing technology (Java, .NET etc.) It should be something that just-works with (or via) existing HTTP. So any technology can use it. Maybe "concept" should be the proper term here. If it isn't something widespread already.

For example, there is basic authentication. It just works with any technology. It has its rules, do's and dont's. There are WebSockets, working just the same. I need something in that context.

  • Google by "data flow" or "cloud data flow". You will find some references to projects such as Spring Data Flow and Spring cloud data flow. Or applications like Node-RED. That should give you something to start with. – Laiv Jan 7 at 14:45
  • @Laiv - thanks for the insight. Unfortunately both Spring Cloud Data Flow is and both built around the cloud context, and is part of the Spring framework. Node-red is built for IoT and does look like it's pretty centralized. I will update my question to be more precise of what I'm looking for. – OzrenTkalcecKrznaric Jan 7 at 16:23
  • Note that you could do the very same with Linux, pipes, commands and shell scripts. I have implemented ETLs that way. As for the question, I have never heard about a protocol like that. – Laiv Jan 7 at 17:12
  • I suspect that the thing you are looking for is the "semantic web". – VoiceOfUnreason Jan 10 at 12:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.