Stream Concepts

Data flows. It arrives from a network socket in unpredictable chunks. It leaves through a file descriptor as fast as the disk allows. It passes through encryption, compression, and framing layers—​each transforming it before handing it off to the next. Modeling this flow well is one of the most important things an I/O library can do.

Capy organizes data flow around six concepts, arranged in three complementary pairs. The design reflects a truth about I/O that most libraries gloss over: partial operations and complete operations are fundamentally different things, and conflating them leads to bugs.

A socket might give you 47 bytes when you asked for 1024. That is not an error—​it is the nature of the hardware. Some code needs to handle those 47 bytes immediately and ask for more. Other code needs exactly 1024 bytes and should not return until it has them (or an error occurs). These are different operations with different contracts, and Capy gives them different names: streams for partial I/O, and sources and sinks for complete I/O.

On top of this, Capy adds buffer sources and buffer sinks--concepts that work with dynamic buffers, enabling protocol parsers and message builders to grow their storage as needed without manual bookkeeping.

What You Will Learn

  • Overview — The six stream concepts at a glance, how they relate to each other, and which one to reach for in different situations.

  • Streams (Partial I/O) — ReadStream and WriteStream--the concepts for operations that transfer some data and return immediately. The building blocks for everything else.

  • Sources and Sinks (Complete I/O) — ReadSource and WriteSink--the concepts for operations that transfer all requested data or report an error. Built on top of streams, with well-defined completion guarantees.

  • Buffer Sources and Sinks — BufferSource and BufferSink--concepts that pair complete I/O with dynamic buffers for protocol-level operations.

  • Transfer Algorithms — Generic algorithms that move data between streams, sources, and sinks. Composable, efficient, and independent of any particular transport.

  • Physical Isolation — How Capy’s stream concepts enable you to test, mock, and compose I/O layers without coupling to specific transports. Write your logic once; run it over TCP, TLS, pipes, or in-memory buffers.

These concepts are the vocabulary of Capy’s I/O model. Once you understand them, every I/O operation in the library will feel familiar.