Questions tagged [io]

Input/Output of data to/from a system. This usually implies file storage or network communication, but may also mean interaction with a user through a user interface.

Filter by
Sorted by
Tagged with
46
votes
5answers
6k views

Critique of the IO monad being viewed as a state monad operating on the world

The IO monad in Haskell is often explained as a state monad where the state is the world. So a value of type IO a monad is viewed as something like worldState -> (a, worldState). Some time ago I ...
27
votes
6answers
21k views

Why do we have to wait for I/O?

It's always been known that Disk operations are slow and we know the reasons why they are slow. So the question here is why do we have to wait for I/O or why is there such a thing as IOWait, etc.? I ...
27
votes
2answers
8k views

Why is universal newlines mode deprecated in Python?

I just noticed that the universal newline feature of file operations seems to be on its way out. The documentation for Python 3.5 open's mode parameter indicates that it's deprecated: 'U' ...
25
votes
5answers
7k views

At what point is asynchronous reading of disk I/O more efficient than synchronous?

Assuming there is some bit of code that reads files for multiple consumers, and the files are of any arbitrary size: At what size does it become more efficient to read the file asynchronously? Or to ...
23
votes
4answers
5k views

How bad is it calling println() often than concatenating strings together and calling it once?

I know output to the console is a costly operation. In the interest of code readability sometimes it is nice to call a function to output text twice, rather than having a long string of text as an ...
19
votes
4answers
6k views

Why do we have to mention the data type of the variable in C

Usually in C, we have to tell the computer the type of data in variable declaration. E.g. in the following program, I want to print the sum of two floating point numbers X and Y. #include<stdio.h&...
14
votes
2answers
40k views

How does a DMA controller work?

From Section 5.1.4 Direct Memory Access in Modern Operating Systems by Andrew S. Tanenbaum, Herbert Bos, 2014, To simplify the explanation, we assume that the CPU accesses all devices and memory ...
13
votes
2answers
695 views

Readiness vs. Completion Async IO Memory usage?

I was watching this talk about implementing Async IO in Rust and Carl mentions two potential models. Readiness and Completion. Readiness Model: you tell the kernel you want to read from a socket do ...
12
votes
1answer
981 views

Model-View-Controller (MVC) Which component handles save/load operations?

In a traditional MVC application, which component (model, view, or controller) is responsible for reading/writing the model to/from disk?
10
votes
7answers
1k views

How do I apply TDD to read/write functions?

It seems like a chicken and egg problem. You can make a write function write to some data store, but never know you saved it properly without a tested read function. You can make a read function ...
9
votes
6answers
5k views

How can IO cause side effects in Functional Programming? [duplicate]

Whenever I read about Haskell, I find that IO can cause side effects. But I do not understand how it would do it? Do we mean that we are writing to a file from one lazy sequence and another lazy ...
9
votes
6answers
3k views

How to Unit-Test a parser of a file?

I'm implementing a metadata parser of image files from all formats. I want to write tests for it. One trivial way to do so is to have test image files of all formats as a resources for the tests, and ...
9
votes
3answers
12k views

Reading file during write on linux

As I understand, when a file is being written, the process writing to the file obtains an exclusive lock. So other processes cannot access this file for read. With the above knowledge, I'm unable ...
8
votes
5answers
50k views

How to structure a loop that repeats until success and handles failures

I am a self-taught programmer. I started programming about 1.5 years ago. Now I have started to have programming classes in school. We have had programming classes for 1/2 year and will have another 1/...
8
votes
5answers
8k views

Why do we have to tell printf() the type of data in C?

Let's consider this C code: #include <stdio.h> main() { int x=5; printf("x is "); printf("%d",5); } In this, when we wrote int x=5; we told the computer that x is an integer. The ...
8
votes
3answers
8k views

What is preferred for file I/O in Java? [closed]

Of the three BufferedReader/Writer Binary Streams Channels which is more preferred in real life for file I/O in Java and why? I prefer channels for binary data to be put into file or read from ...
7
votes
2answers
5k views

Separate Thread Pools for I/O and CPU Tasks

I've been puzzling over a good implementation for this for a while. I have a program that does a long-running I/O operation (downloading a file) and a long-running CPU operation (parsing its contents)...
7
votes
1answer
9k views

Benefits of Java ByteArrayInputStream vs byte array?

I have a scenario where my Java process will be receiving a stream of binary data: public class DataHandler { public void handleData(DATA data) { // TODO: Do something with data } } ...
7
votes
1answer
1k views

Does it ever make sense to read from and/or write to a bound, listening socket?

I'm experimenting a little with creating a socket server, in PHP. In doing so I'm trying to abstract away the kinds of sockets I think I'll be needing, that I've tentitively named: ListenSocket — the ...
6
votes
3answers
4k views

What does “address space” means when talking about IO devices?

The following quote is from this page: While some CPU manufacturers implement a single address space in their chips, others decided that peripheral devices are different from memory and, ...
6
votes
1answer
2k views

Who is responsible for stream positioning?

The samples in question are c#, but it applies to any language. If you have a function which reads content from a stream. Who should be responsible for ensuring that the stream position is correct ...
5
votes
7answers
2k views

Why have many programmers moved to using exception handling for input or output? [closed]

Why have many programmers moved to using exception handling for input or output? For these programmers, what is the motivation behind this decision?
5
votes
2answers
504 views

What is constraining cross-platform asynchronous file I/O?

Looking at a range of cross-platform languages, libraries and GUI toolkits, I often notice a conspicuous absence of support for asynchronous file I/O. This seems like too much of a common factor to be ...
5
votes
3answers
395 views

Generic and type safe I/O model in any language

I am looking for an I/O model, in any programming language, that is generic and type safe. By genericity, I mean there should not be separate functions for performing the same operations on different ...
5
votes
1answer
6k views

How does the Base Address Registers (BARs) in a PCI card work?

I am trying to understand how the Base Address Registers (BARs) in a PCI card work, this is how I think they work: Each function in a PCI card have 6 BAR fields, and each BAR field is 32-bit in size. ...
5
votes
2answers
1k views

Design pattern for an automated mechanical test bench

Background I have a test fixture with a number of communication/data acquisition devices on it that is used as an end of line test for a product. Because of all the various sensors used in the ...
4
votes
2answers
654 views

What is a stream, and how does it work?

Like a data stream in general, this can be an I/O stream or memory stream or a file stream or what not. I understand that it's some sort of stream of bytes, that you can read to parse very large ...
4
votes
2answers
449 views

Should a Closeable throw an exception if used after being closed, even if it doesn't have to?

Let's say we have this interface: public interface OutputChannel extends Closeable { public void writeOutput(String output); } Then there are several classes which implement this interface and ...
4
votes
1answer
2k views

Name convention for Reader and Writer [closed]

I'm trying to name a component that will be capable of both reading from an InputStream (file or otherwise), and writing to an OutputStream; thus serving both as a Reader and as Writer. So far the ...
4
votes
3answers
1k views

Providing a non-blocking IO API in a C library

I am working on a C library (SlipRock) for interprocess communication. The library currently exposes a simple, blocking API. This is easy to use, makes misuse (relatively) difficult (this is C after ...
4
votes
2answers
9k views

Using streams as method parameters and/or return types

So let's say I have a 'Transformer' interface that can transform one file-type to another. Let's say I also wish to use streams to provide the data to the Transformer. Which is the approach that is ...
4
votes
2answers
345 views

How does I/O work for large graph databases?

I should preface this by saying that I'm mostly a front end web developer, trained as a musician, but over the past few years I've been getting more and more into computer science. So one idea I have ...
4
votes
1answer
3k views

How to read from a database, and write to a file asynchronously / non blocking way in Java

I'm trying to modify a serial program that reads from a database, and writes results to a file, this is done in a blocking way and I think we can get performance boost of there is a memory buffer and ...
4
votes
1answer
356 views

Which C++ IO interfaces for a complex data source i.e. converter

My question: When you've got a complex converter like, that takes chunks or large result sets out of a database, converts it into a line by line file/resource in the end, should one either design it ...
3
votes
2answers
7k views

How does printf write to stdout? [closed]

I've been looking for an answer to this for a couple days, and all of the results I've come up with are about how printf accepts an arbitrary number of arguments. How does printf write to stdout? ...
3
votes
4answers
169 views

Structuring an input file [closed]

I am in the process of structuring a small program to perform some hydraulic analysis of pipe flow. As I am envisioning this, the program will read an input file, store the input parameters in a ...
3
votes
4answers
603 views

Does a full duplex communication object conflict with single responsibility?

I'm writing a Websocket implementation in Java, and I've set it up so that it basically just wraps an InputStream and OutputStream, and has public methods for both sending and receiving. While input ...
3
votes
5answers
10k views

Sharing buffer between multiple threads

I had a job process that was executing a lot of IO to read and write temporary files. Now I want to (need to) reduce the quantity of IO executions. So I want to create a sort of circular buffer ...
3
votes
1answer
119 views

Difference Between AsyncResult and Task in c#

So, in C#, I understand the historical difference between the two vaguely; a Task is a newer concept and the highest level concurrency native C# offers. AsyncResult is a bit more ambiguous. For ...
3
votes
3answers
5k views

Are “Control register” and “Status register” and “Data register” part of the device itself?

I am studying about Memory-Mapped I/O from here. I have read the following: From the CPU's perspective, an I/O device appears as a set of special-purpose registers, of three general types: ...
3
votes
1answer
106 views

How “close” should a IO class/logic be to a form or thread that controls the IO device?

I am having a difficult time determining where my IO logic should reside within my application. In this application there are multiple IO devices both USB and serial. I currently have the idea to ...
2
votes
1answer
207 views

What does “data bus control” mean?

This video mentions the following: What does it mean for the DMA controller to be granted the data bus control, does that mean the CPU cannot use the bus to access memory and IO devices until the DMA ...
2
votes
1answer
211 views

Are streams of binary data considered a form of bit banging?

Are streams of binary data considered a form of bit banging? Does this definition change if the array is buffered? I am referring software which handles binary data on a general purpose CPU; for ...
2
votes
1answer
310 views

How do peripherals decide which address to use in memory-mapped I/O?

I've been reading about how memory-mapped I/O actually works and I cannot understand how the system assigns an address or address space to a specific device. Let's say I had two devices that I wanted ...
2
votes
1answer
4k views

How data is accessed in Memory-Mapped I/O?

This is an example of Memory-Mapped I/O: So basically you access the device controller registers through memory. Now my question is, when you for example write to the memory location that maps to ...
2
votes
1answer
29k views

How to let multiple threads write on the same file [closed]

I have got a text file called "vholders.txt". I am making multiple threads as you can see here ,those threads work with their own given data and at last they write their own output to the vholders....
2
votes
2answers
213 views

Reading huge (up to 32 GB) file in multithreading env?

Goal: 1) huge file read in chunks each 1 MB long, 2) each one gets compressed and written to an another output file. Note: I am limited to .Net 3.5 only. Is there a known pattern how to parallelize ...
2
votes
2answers
2k views

Reading using non-blocking IO on a fd asynchronously in C++ (moving from Node.js)

I'm trying to switch my brain from Node.js/Objective-C iOS programming to C++ programming, and it's a little bit taxing. Node.js and Objective-C with iOS do not have a run loop that I am supposed to ...
2
votes
1answer
128 views

What are the underlying patterns in systems allowing asynchronous operations?

Say I'm working with a system that allows async, nonblocking operations. If I queue up a set of those operations and specify their result buffer references: nonblocking_write( message, write_buffer )...
2
votes
1answer
217 views

Where do you use Java Channels?

I have only used Java Channels and BufferedOutputStream and the input counter part for reading and writing simple integers and some strings here and there to the file. However I do not understand the ...