You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+6-14Lines changed: 6 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -62,7 +62,7 @@ julia> encodings()
62
62
(Note that many of these are aliases for standard names.)
63
63
64
64
## The `Encoding` type
65
-
In the examples above, the encoding was specified as a standard string. Though, in order to avoid ambiguities in multiple dispatch and to benefit from type specialization performance benefits, the package offers a special `Encoding` parametric type. Each parameterization of this type represents a character encoding. The [non-standard string literal](http://docs.julialang.org/en/stable/manual/strings/#man-non-standard-string-literals)`enc` can be used to create an instance of this type, like so: `enc"UTF-16"`.
65
+
In the examples above, the encoding was specified as a standard string. Though, in order to avoid ambiguities in multiple dispatch and to increase performance via type specialization, the package offers a special `Encoding` parametric type. Each parameterization of this type represents a character encoding. The [non-standard string literal](http://docs.julialang.org/en/stable/manual/strings/#man-non-standard-string-literals)`enc` can be used to create an instance of this type, like so: `enc"UTF-16"`.
66
66
67
67
Since there is no ambiguity, the `encode` and `decode` functions accept either a string or an `Encoding` object. On the other hand, other functions presented below only support the latter to avoid creating conflicts with other packages extending Julia Base methods.
## Advanced Usage: `StringEncoder` and `StringDecoder`
131
-
The convenience functions presented above are based on the `StringEncoder` and `StringDecoder` types, which wrap I/O streams and offer onthefly character encoding conversion facilities. They can be used directly if you need to work with encoded text on an already existing I/O stream. This can be illustrated using an `IOBuffer`:
131
+
The convenience functions presented above are based on the `StringEncoder` and `StringDecoder` types, which wrap I/O streams and offer on-the-fly character encoding conversion facilities. They can be used directly if you need to work with encoded text on an already existing I/O stream. This can be illustrated using an `IOBuffer`:
132
132
```julia
133
133
julia> b =IOBuffer();
134
134
135
135
julia> s =StringEncoder(b, "UTF-16");
136
136
137
-
julia>write(s, "café");
137
+
julia>write(s, "café");# Encoding happens automatically here
138
138
139
139
julia>close(s); # Essential to complete encoding
140
140
141
-
julia>decode(takebuf_array(b), enc"UTF-16")
142
-
"café"
143
-
```
144
-
145
-
And the reverse operation:
146
-
```julia
147
-
julia> b =IOBuffer();
141
+
julia>seek(b, 0); # Move to start of buffer
148
142
149
143
julia> s =StringDecoder(b, "UTF-16");
150
144
151
-
julia>write(b, encode("café", enc"UTF-16"));
152
-
153
-
julia>decode(takebuf_array(b), enc"UTF-16")
145
+
julia>readstring(s) # Decoding happens automatically here
154
146
"café"
155
147
```
156
148
157
-
Do not forget to call `close` on `StringEncoder` and `StringDecoder` objects to release iconv resources. In the case of `StringEncoder`, this function will also call `flush`, which will write any characters still in the buffer, and possibly some control sequences (for stateful encodings).
149
+
Do not forget to call `close` on `StringEncoder` and `StringDecoder` objects to finish the encoding process. For `StringEncoder`, this function calls `flush`, which writes any characters still in the buffer, and possibly some control sequences (for stateful encodings). For both `StringEncoder` and `StringDecoder`, `close` checks that there are no incomplete sequences left in the input stream, and raise an `IncompleteSequenceError` if that's the case. It will also free iconv resources immediately, instead of waiting for garbage collection.
158
150
159
151
Conversion currently raises an error if an invalid byte sequence is encountered in the input, or if some characters cannot be represented in the target enconding. It is not yet possible to ignore such characters or to replace them with a placeholder.
0 commit comments