|
1 | | - |
2 | | -`kotlinx-io` is a multiplatform library for |
3 | | -processing binary data, |
4 | | -working with memory blocks, |
5 | | -interacting with the platform, |
6 | | -and performing other low level operations. |
7 | | - |
8 | 1 | [](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub) |
9 | 2 |
|
10 | | -NOTE: master contains version 0.2.x and is *experimental*. Any API is a subject to change. If you're looking for multiplatform releases of older versions, you can check [ktor-io](https://github.com/ktorio/ktor/tree/master/ktor-io). See https://github.com/Kotlin/kotlinx-io/issues/54 for more details. |
11 | | - |
12 | | -# Structure |
13 | | - |
14 | | -NOTE: This part represents the target vision of the package. Most modules are still unavailable. |
15 | | - |
16 | | -`kotlinx-io` package has many modules and you can use as much as you need. |
17 | | - |
18 | | -* `core` – defines all the low-level functionality for working with binary data and basic text. |
19 | | -* `async` – (unavailable) implements asynchronous versions of input and output data streams. |
20 | | -* `platform` – (unavailable) provides low-level platform facilities such as reading or writing from/to a file. |
21 | | -* `sockets` – (unavailable) provides low-level functionality for interacting with network. |
22 | | -* `cryptography` – (unavailable) provides encryption & decryption functionality. |
23 | | -* `compression` – (unavailable) provides compression & decompression functionality. |
24 | | -* `files` – (unavailable) provides advanced file system functionality such as working with paths and directories. |
25 | | - |
26 | | -# Core |
27 | | - |
28 | | -This module provides few core I/O primitives that are used across other modules and can be used to define |
29 | | -custom binary inputs and outputs, as well as processing raw memory. |
30 | | - |
31 | | -* `Buffer` – represents a continuous memory block of specific size. |
32 | | -Provides direct positional read and write operations for primitives and arrays of primitives. |
33 | | -* `Input` – represents a source of bytes. Provides sequential reading functionality and a special `preview` mode |
34 | | -for processing bytes without discarding them. |
35 | | -* `Output` – represents a destination for bytes. Provides sequential writing functionality. |
36 | | -* `Bytes` – represents binary data of arbitrary size, potentially spanned across several buffers. |
37 | | -Can be built using using `Output` and can be read using `Input`. |
38 | | - |
39 | | -It also has basic facilities for working with text, with UTF-8 implemented efficiently in core, |
40 | | -and `Charsets` giving access to platform-dependent functionality for converting text into bytes and back. |
41 | | - |
42 | | -## Buffers |
43 | | - |
44 | | -Buffer is direct representation of memory on the target platform implemented using efficient platform-dependent |
45 | | -mechanisms. A buffer of arbitrary size can be allocated and released using `PlatformBufferAllocator`. |
46 | | -It is user's responsibility to release an allocated buffer. |
47 | | - |
48 | | -```kotlin |
49 | | - val buffer = PlatformBufferAllocator.allocate(8) // allocates a buffer of 8 bytes |
50 | | - buffer.storeLongAt(0, 123451234567890L) // stores a long value at offset 0 |
51 | | - val longValue = buffer.loadLongAt(0) // reads back a long value |
52 | | -``` |
53 | | - |
54 | | -All operations with a `Buffer` are performed in network byte order (Big-Endian). |
55 | | -There are helper functions `reverseByteOrder` defined for all primitive types to reverse the byte order when it is needed. |
56 | | - |
57 | | -## Inputs |
58 | | - |
59 | | -An `Input` is a high-performance buffered entity for reading data from an underlying source. |
60 | | -It is an abstract class with only few abstract methods and a plentiful of convenience built around them. |
61 | | -There are functions to read primitives, arrays of primitives, higher-level extension methods for reading UTF-8 text, |
62 | | -text encoded with a custom `Charset`, and more. One can define any other read methods using extensions and provided primitives. |
63 | | - |
64 | | -`Input` design doesn't provide facilities for direct manipulation of the current reading position, |
65 | | -but instead it has the `preview` mechanism which we believe is a lot safer, efficient and enough for most look-ahead |
66 | | -scenarios. |
67 | | - |
68 | | -Preview operation instructs `Input` to start accumulating buffers instead of discarding them when they are exhausted, |
69 | | -thus making it possible to revert to the initial position without performing additional I/O operations. |
70 | | - |
71 | | -```kotlin |
72 | | - input.readLong() // (0) reads long value and discards bytes |
73 | | - input.preview { // (1) begins preview operation and stops discarding bytes |
74 | | - readShort() // (2) read short value and keep the bytes |
75 | | - } // completes preview operation and rewinds the input to the state (1) |
76 | | - input.readShort() // (3) reads short value from (2) again |
77 | | -``` |
78 | | - |
79 | | -Note that `preview` function provides another, nested `Input` to the lambda as a receiver |
80 | | -which should be used for all preview reads. |
81 | | -Implementation can choose to alter original `Input` state or create a new instance, |
82 | | -so one should always be using the instance provided as a receiver to `preview`. |
83 | | - |
84 | | -Preview operations can be nested, each keeping its own state and position, thus making it possible to compose |
85 | | -operations on Inputs. |
86 | | - |
87 | | -## Outputs |
88 | | - |
89 | | -An `Output` is a high-performance buffered entity for writing data to an underlying destination. |
90 | | -Like `Input`, it provides all the primitive operations as well as a number of convenience functions for text output. |
91 | | - |
92 | | -Similarly, `Output` doesn't provide a mechanism to rewind backwards and update data, but using `Bytes` one can easily |
93 | | -implement complex scenarios such as writing a size before a block, calculating hashes and so on. |
94 | | - |
95 | | -## Bytes |
96 | | - |
97 | | -A `Bytes` type is useful for transferring data between various endpoints, accumulating data in memory or sending |
98 | | -repetitive bytes to different outputs. |
99 | | - |
100 | | -`Bytes` can be produced by building function `buildBytes { … }` where lambda has an `Output` as a receiver, |
101 | | -thus making it possible to conveniently generate content, or use in any I/O operations or custom user's functions. |
102 | | - |
103 | | -```kotlin |
104 | | - val bytes = buildBytes { |
105 | | - writeLong(0x0001020304050607) |
106 | | - writeShort(0x0809) |
107 | | - } |
108 | | -``` |
109 | | - |
110 | | -When you have a `Bytes` instance, you can know the number of bytes stored, and can obtain an `Input` to read these bytes. |
111 | | -Creating an `Input` is a zero-copy operation, underlying mechanics simply reuses buffers for reading data. |
112 | | - |
113 | | -```kotlin |
114 | | - val input = bytes.input() |
115 | | - input.readLong() |
116 | | -``` |
117 | | - |
118 | | -Writing such an instance into `Output` is also zero-copy operation, since implementation will send existing buffers |
119 | | -to the underlying destination. |
120 | | - |
121 | | -```kotlin |
122 | | - output.writeBytes(bytes) |
123 | | -``` |
124 | | - |
125 | | -Combining these features makes it possible to write domain-specific functions for complex data writing: |
126 | | - |
127 | | -```kotlin |
128 | | -fun Output.writeWithSizeAndHash(writer: Output.()->Unit) { |
129 | | - val bytes = buildBytes(writer) |
130 | | - writeInt(bytes.size) |
131 | | - writeBytes(bytes) |
132 | | - val hash = bytes.input().calculateHash() |
133 | | - writeLong(hash) |
134 | | -} |
135 | | -``` |
136 | | - |
137 | | -## Text |
138 | | - |
139 | | -[TBD] Efficient UTF-8 and platform-dependent Charsets |
140 | | - |
141 | | -## Pools |
142 | | - |
143 | | -[TBD] Allocating and releasing a buffer each time one is needed can be inefficient, |
144 | | -so the package provides facilities for buffer pools. |
145 | | - |
146 | | -# Async |
147 | | - |
148 | | -[TBD] `InputChannel` and `OutputChannel` as an asynchronous (suspending) versions of `Input` and `Output` |
149 | | - |
150 | 3 |
|
151 | | -# Platform |
| 4 | +### Obsolete warning |
152 | 5 |
|
153 | | -[TBD] |
154 | | -* `FileInput` and `FileOutput` with a very limited set of operations such as `open`. No paths, no directories, no access control. |
155 | | -* `Process` type for launching an external processes, and interacting with their inputs and outputs. |
156 | | -* `Environment` type for interacting with environment variables. |
| 6 | +The library was an experiment for providing a cross-platform multiplatform IO primitives |
| 7 | +and its development and maintenance was effectively postponed in 2019. |
157 | 8 |
|
158 | | -# Adding a dependency |
| 9 | +Its current API is frozen at version 0.1.16 and won't receive any further API updates, |
| 10 | +improvements or bugfixes. |
159 | 11 |
|
160 | | -```gradle |
161 | | -dependencies { |
162 | | - compile "org.jetbrains.kotlinx:kotlinx-io-jvm:$kotlinx_io_version" |
163 | | -} |
164 | | -``` |
| 12 | +### Future replacement |
165 | 13 |
|
| 14 | +Future replacement is to be discussed, currently we do recomment using [Okio](https://github.com/square/okio/) |
| 15 | +for multiplatform IO primitives. |
0 commit comments