Skip to main content

Questions tagged [dataflow]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer

Using both websocket and REST for the application requiring two-way communication

I am currently implementing front-end part of the collaborative application requiring two-way live communication between server and the web app. Cloud service responsible for the data exchange will be ...
helloFromTheOtherSide's user avatar
1 vote
0 answers

Where should I persist the side effects of actions?

I have a outlining app that I am writing using unidirectional dataflow and some of the concepts from Redux. When the user has selects items in the outline, and clicks indent, an indent action is fired....
bnieland's user avatar
2 votes
1 answer

How (whether to?) include Apache Spark in my Architecture

Brief overview of general data flow The general goal of my system is to allow users to upload many different types of files containing data (PDF, CSV, ZIP, etc.), then index it and perform some basic ...
foxtrotuniform6969's user avatar
-1 votes
1 answer

Do other languages have variables shared between threads?

I guess it would be too complex for Node.js / JavaScript to leverage, but I've been working with clusters in node to break big tasks down so all cores can work at once but the inter-process messaging ...
J.Todd's user avatar
  • 3,823
0 votes
3 answers

Diagram to show how one object "becomes" another?

I am working in a purchasing system where we take calls: public class Call { public List<CallItem> CallItems { get; set; } } The calls can be turned into quotes, where call items become ...
Sydney Park's user avatar
0 votes
0 answers

Program Simulation Techniques

I've noticed that data-flow analysis uses approximations of a programs steps by simulating the value flow in some way (which I don't fully understand yet). They describe it well here. On page 4 they ...
Lance's user avatar
  • 2,605
0 votes
2 answers

CLI to Aid Versioning of Input, Source, and Output

I'm currently working in an environment where I have the following: Multiple Inputs (with multiple versions) Source code to generate output (multiple versions) Output generated by a combination of ...
faulty_ram_sticks's user avatar
4 votes
1 answer

Is undo and redo possible/feasible with two-way data binding (vs uni-directional data flow)?

I'm developing an experimental application for work that requires fully persistent undos and redos. I decided to use react/redux/immutable JS tech stack to solve this problem because I saw it worked ...
Rico Kahler's user avatar
1 vote
2 answers

Application design for step by step filling information data flow

Suppose, my application consists of many components, where users need to fill theirs information, something like this: -Select city `-Select library `-Select book `-... a lot of components like ...
Yurii N.'s user avatar
  • 341
5 votes
3 answers

Patterns to allow for versatile data-flow hooks in Javascript

I have an application where I'd like to add ways for external modules/components to hook during the processes and data-flow and perform actions/modify data. For the first requirement—performing ...
Sunyatasattva's user avatar
3 votes
0 answers

Spark supports cyclic data flow

I read on that Spark supports cyclic data flow. How can data flow be cyclic if the RDDs used by Spark are immutable? I thought the processing done by Spark looks like an ...
bsky's user avatar
  • 173
1 vote
1 answer

How to decide if object should request or receive data?

Deciding whether an object should request data (by calling a function) or receive data (by having a function called from another object) is probably dependent on the exact situation. So how do I ...
Simon Verbeke's user avatar
1 vote
1 answer

Using streams to connect programming languages

I'm trying to run a computation-heavy program that creates an image on a website. Is it possible to compute in C++ and have an output stream that connects to an input stream in Node.js to display an ...
user939687's user avatar
3 votes
1 answer

How to efficiently implement this background processing chain?

I am working on an audio software that uses The EchoNest web service to identify and retrieve metadata about audio songs and I would like to have some advice on implementing a background processing ...
aybe's user avatar
  • 765
12 votes
3 answers

What is a "lifted representation"?

Just ran across this term here: "We'll demonstrate the Flow API—a lifted representation—as well as a pluggable way of transforming the lifted ...
Den's user avatar
  • 4,847
0 votes
1 answer

Good approach for hundreds of comsumers and big files

I have several files (nearly 1GB each) with data. Data is a string line. I need to process each of these files with several hundreds of consumers. Each of these consumers does some processing that ...
shda's user avatar
  • 121
1 vote
2 answers

How to manage data flow between UI components?

I have been trying to use classes that implement ActionListener to respond to user input. I'm consistently noticing that for an action listener to communicate the event to other objects, it has to ...
user2738698's user avatar
11 votes
0 answers

Implement FBP / Dataflow programming in VBA?

Intrigued by Flow Based Programming a la J Paul Morrison. I wonder if one can implement FBP in Excel/VBA. I searched quite a bit for a Basic-Lang FBP implementation to no avail. I've looked at some ...
klausnrooster's user avatar
3 votes
1 answer

How do I transparently cache data in intermediate stages of processing?

I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline. ckt = generate_ckt(ckt_properties); freq = generate_fpoints(fconfig); result = freq_dom_sim(ckt,...
Milind R's user avatar
  • 155
5 votes
4 answers

Is it bad practice for a module to contain more information than it needs?

I just wanted to ask for your opinion on a situation that occurs sometimes and which I don't know what would be the most elegant way to solve it. Here it goes: We have module A which reads an entry ...
Helder AC's user avatar
  • 295
6 votes
3 answers

Real-time Dataflow Programming

Background Recently, I have been fascinated about the possibilities of dataflow-based programming approaches - in particular, signal/event-based concepts as realized by functional reactive ...
Frank's user avatar
  • 14.4k
9 votes
6 answers

Do you see an use for "Spreadsheet programming"? [closed]

A while ago I stumbled upon the concept of using spreadsheets (I mean cells and formulas not Macro code) as a way to specify programming logic. The idea is: create a spreadsheet with a cleanly ...
Paolo Bozzola's user avatar