jkc.git/src/io.hh, branch WIPUnnamed repository; edit this file 'description' to name the repository.
https://git.spawned.biz/jkc.git/atom?h=WIP2025-09-22T21:38:21ZChange io::Reader and company to return ReadError::Eof instead of 0.2025-09-22T21:38:21ZJoel Klinghedthe_jk@spawned.biz2025-09-22T21:38:21Zurn:sha1:ce271f82f16ee89a18e7bfc9ed8eab7cbd6f37bc
It's debatable if Eof should be considered an error or not.
But it is pretty clear it generally is a special response that
needs special handling, so easier to keep with the unexpected lot.
Also keeps better at higher abstraction levels, such as the line
reader.
uio: Unicode reader2025-09-15T18:52:51ZJoel Klinghedthe_jk@spawned.biz2025-09-15T18:52:51Zurn:sha1:18a622f378b403788c67fc785d30f4609caa3fc7
Reads UTF-8 and UTF-16 into UTF-8 or UTF-16 strings.
If strict is true, fails at first invalid character.
If strict is false, invalid characters are replaced with U+FFFD.
For the replacement, I changed behavior if uN::read_replace to only
jump one byte. Otherwise a common invalid case when ISO-8859-1 or
WIN-1252 are read as UTF-8 would skip many characters.
If skip_bom is true any bom at start of stream is ignored.
If skip_bom is false any bom will be included.
Input format can be forced, if not detect is used which will
try to guess and then fallback to UTF-8.
Add unicode general category lookup2025-09-10T20:12:22ZJoel Klinghedthe_jk@spawned.biz2025-09-10T20:12:22Zurn:sha1:32e14551a90e85000e41b3f0445d34d58a1431e4
Generate the lookup tables from UnicodeData.txt, do to that,
add gen_ugc, which uses csv, buffers, line, io and other modules
to do the job.