0

So, there is a workflow I feel needs to exist. It may already exist, and I don't know about it.

Consider the Linux command line workflow, the "motto" of Linux, more or less. You have a bunch of small programs, accepting input and producing output. Each does one thing well, and the user puts them together to do complicated things. (In theory. There are counterexamples, but on the whole, my experience has been that this is true, and tends to work well.) However, I feel that there are a few problems.

  1. A given program can only easily accept one input, and produce one output.
  2. Both input and output are a linear sequence of characters. (It is debatable whether this is truly a problem.)
  3. The model, as it stands, is somewhat incompatible with GUIs.

Alternatives to each item in order:

  1. A given program has an arbitrary number of input/output ports.
  2. Each port has a specified "type" - raw binary, video, text, arrays of numbers, etc. (The likelihood of standards-war makes me waffle on this point. Perhaps it could just be marked as "16 bit raw PCM" or something, but all data is basically just raw binary streams and it's up to you to make sure they're matched correctly, as when piping output from gzip etc.)
  3. A GUI would display its I/O ports somewhere, and you could drag from one of one program's outputs to one of another program's inputs.

Consider, if you will, the following vision of a glorious future: You open some esoteric video format in program 1, which offers it in a raw format. You drag that output to another program 2, which splits the stream into a video output and an audio output. You drag the video from 2 to some filtering program 3 where you invert the colors, and the audio from 2 to some program 4 resembling Audacity, where you select reverb and equalization or something. You drag from 3 and 4 back to a recombiner 5, from 5 to an encoder 6, and from 6 to a streaming service 7. Then you hit go or something and you're streaming a bizarrely filtered video onto the internet, because you can.

So, my actual questions:

  1. Is there already a way this can be done, functional for an end user right now?
  2. Are there libraries that are intended and designed to allow this kind of thing?
  3. Are there fundamental obstacles to this that I have overlooked?
  4. Any other comments you feel ought to be noted?
  • Thought experiment: instead of programs 1 through 7 accepting typed data blobs to arbitrary ports as input, what if each program simply accepted a string path to a file (in the form of a command-line parameter) as input? Another program (a "controller" of sorts) could establish the file paths and execute the programs in the correct order. You could drive it with an XML configuration file maintained by a GUI. – Robert Harvey Apr 18 '18 at 22:08
  • @RobertHarvey Hmm.... That would probably work, but one of the drawbacks is that unless you had the controller repeatedly execute/trigger each program to operate on small chunks, you'd need to process the entire file one full step at a time, and if it's fundamentally a stream (say, a webcam), you may have problems. Basically, it sounds like it'd remove the ability to stream data. You could bend such a system into a form that could deal with streams, but then why wouldn't you just deal with them as streams to begin with? – Erhannis Apr 18 '18 at 22:22
  • 1
    The most difficult part is probably the GUI - routing is a hard problem. link 1 link 2 (the image) Note that the image is generated from GraphViz (using its DOT language), so it is not an interactive GUI. Given that many DSP, Dataflow software exists but not many good GUI, GUI might actually be a harder problem than those hard problems. – rwong Apr 18 '18 at 23:26
  • 1
    There are already various methods of IPC (shared memory, sockets etc.) which can provide exactly this. Their only disadvantage is that they're not as simple to use as program1 | program 2, precisely because of the ambiguities you mention. The UNIX (not just Linux) pipe-of-tools model has been successful since the 1960s for a reason. – Kilian Foth Apr 19 '18 at 6:22
  • 1
    There are applications that work sort of like this in particular domains, for example Max/MSP allows arbitrary routing of audiovisual data between processing modules using a graph-like interface, and PureData is a similar open-source equivalent. – Sean Burton Apr 19 '18 at 11:02
1

Systems like what you describe have been implemented. CORBA is similar (but not quite the same) as what you describe.

Apple implemented their OpenDoc standard many years ago and it was a major flop. It allowed any app to create data and any other app to embed that data in their documents. The idea (if I recall correctly) was that anyone could make an editor for a given data type and anyone could make a component that would display the given data and you could use these components to build documents rather than making a document in a single application.

Microsoft had something similar with OLE.

Apple's Automator application allows you to use a GUI to tie together various other components like shell scripts, Javascript, AppleScript, and various application-specific features. I'm sure there are probably similar apps for other OSes.

0

Hy, you have quite a few flaws in thinking. Interprocess communication is quite slow so what you are asking for is reading / writing data from and to shared memory space. The next problem is synchronization any streaming for your purpose would include mutexes that would make it even slower. The easiest way to handle things like that is to save images to split the stream but if you have 30 fps that will bee a challenge. In any way you would need to handle the IPC and synchronization.

One thing is that you need to define what is your maximum quantum of time that you consider for one data set. What happens if that protocol of tome gets broken? In general you would need a high performance data handler with triple buffering and ring buffer management that handles the data outputs for other apps. This is not a small task and there are projects that are trying similar stuff.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.