Questions tagged [dataflow]
The dataflow tag has no usage guidance.
21
questions
1
vote
0
answers
132
views
Where should I persist the side effects of actions?
I have a outlining app that I am writing using unidirectional dataflow and some of the concepts from Redux.
When the user has selects items in the outline, and clicks indent, an indent action is fired....
- 89
2
votes
1
answer
170
views
How (whether to?) include Apache Spark in my Architecture
Brief overview of general data flow
The general goal of my system is to allow users to upload many different types of files containing data (PDF, CSV, ZIP, etc.), then index it and perform some basic ...
-1
votes
1
answer
313
views
Do other languages have variables shared between threads?
I guess it would be too complex for Node.js / JavaScript to leverage, but I've been working with clusters in node to break big tasks down so all cores can work at once but the inter-process messaging ...
- 3,833
0
votes
3
answers
409
views
Diagram to show how one object "becomes" another?
I am working in a purchasing system where we take calls:
public class Call
{
public List<CallItem> CallItems { get; set; }
}
The calls can be turned into quotes, where call items become ...
- 19
0
votes
0
answers
61
views
Program Simulation Techniques
I've noticed that data-flow analysis uses approximations of a programs steps by simulating the value flow in some way (which I don't fully understand yet). They describe it well here. On page 4 they ...
- 2,585
0
votes
2
answers
107
views
CLI to Aid Versioning of Input, Source, and Output
I'm currently working in an environment where I have the following:
Multiple Inputs (with multiple versions)
Source code to generate output (multiple versions)
Output generated by a combination of ...
4
votes
1
answer
885
views
Is undo and redo possible/feasible with two-way data binding (vs uni-directional data flow)?
I'm developing an experimental application for work that requires fully persistent undos and redos. I decided to use react/redux/immutable JS tech stack to solve this problem because I saw it worked ...
- 240
1
vote
2
answers
104
views
Application design for step by step filling information data flow
Suppose, my application consists of many components, where users need to fill theirs information, something like this:
-Select city
`-Select library
`-Select book
`-... a lot of components like ...
- 341
5
votes
3
answers
368
views
Patterns to allow for versatile data-flow hooks in Javascript
I have an application where I'd like to add ways for external modules/components to hook during the processes and data-flow and perform actions/modify data.
For the first requirement—performing ...
- 440
3
votes
0
answers
520
views
Spark supports cyclic data flow
I read on http://spark.apache.org/ that Spark supports cyclic data flow.
How can data flow be cyclic if the RDDs used by Spark are immutable?
I thought the processing done by Spark looks like an ...
- 173
1
vote
1
answer
174
views
How to decide if object should request or receive data?
Deciding whether an object should request data (by calling a function) or receive data (by having a function called from another object) is probably dependent on the exact situation. So how do I ...
- 181
1
vote
1
answer
447
views
Using streams to connect programming languages
I'm trying to run a computation-heavy program that creates an image on a website. Is it possible to compute in C++ and have an output stream that connects to an input stream in Node.js to display an ...
- 13
3
votes
1
answer
308
views
How to efficiently implement this background processing chain?
I am working on an audio software that uses The EchoNest web service to identify and retrieve metadata about audio songs and I would like to have some advice on implementing a background processing ...
- 727
12
votes
3
answers
5k
views
What is a "lifted representation"?
Just ran across this term here:
http://www.codemesh.io/codemesh2014/viktor-klang
"We'll demonstrate the Flow API—a lifted representation—as well as a
pluggable way of transforming the lifted ...
- 4,827
0
votes
1
answer
339
views
Good approach for hundreds of comsumers and big files
I have several files (nearly 1GB each) with data. Data is a string line.
I need to process each of these files with several hundreds of consumers. Each of these consumers does some processing that ...
- 121
1
vote
2
answers
1k
views
How to manage data flow between UI components?
I have been trying to use classes that implement ActionListener to respond to user input. I'm consistently noticing that for an action listener to communicate the event to other objects, it has to ...
- 957
11
votes
0
answers
840
views
Implement FBP / Dataflow programming in VBA?
Intrigued by Flow Based Programming a la J Paul Morrison. I wonder if one can implement FBP in Excel/VBA. I searched quite a bit for a Basic-Lang FBP implementation to no avail. I've looked at some ...
- 231
3
votes
1
answer
565
views
How do I transparently cache data in intermediate stages of processing?
I am working with MATLAB on a model reduction algorithm. It is basically a data processing pipeline.
ckt = generate_ckt(ckt_properties);
freq = generate_fpoints(fconfig);
result = freq_dom_sim(ckt,...
- 155
5
votes
4
answers
614
views
Is it bad practice for a module to contain more information than it needs?
I just wanted to ask for your opinion on a situation that occurs sometimes and which I don't know what would be the most elegant way to solve it.
Here it goes:
We have module A which reads an entry ...
- 295
6
votes
3
answers
2k
views
Real-time Dataflow Programming
Background
Recently, I have been fascinated about the possibilities of dataflow-based programming approaches - in particular, signal/event-based concepts as realized by functional reactive ...
- 14.4k
9
votes
6
answers
4k
views
Do you see an use for "Spreadsheet programming"? [closed]
A while ago I stumbled upon the concept of using spreadsheets (I mean cells and formulas not Macro code) as a way to specify programming logic.
The idea is:
create a spreadsheet with a cleanly ...
- 344