jkc.git/meson.build, branch mainUnnamed repository; edit this file 'description' to name the repository.
https://git.spawned.biz/jkc.git/atom?h=main2026-05-02T08:54:02ZWIP2026-05-02T08:54:02ZJoel Klinghedthe_jk@spawned.biz2026-05-01T16:45:23Zurn:sha1:19005581a0d35233f862e57308734d3486569bb9java: Add tokens support for Java 212025-09-29T07:50:47ZJoel Klinghedthe_jk@spawned.biz2025-09-29T07:39:49Zurn:sha1:d196d51e07f50f3510c43ad375c5559b58860023
Some new keywords, I opted to modify java-8 grammar to use the new
names, even if they are not going to match anything. Makes the
tokenizer easier to write.
java: Add tokens2025-09-29T07:39:17ZJoel Klinghedthe_jk@spawned.biz2025-09-28T20:53:30Zurn:sha1:1e9e51dae1c01bab7562911b958c47528b8011c8
Only parses Java 8 tokens for now.
Add simple prefix_tree2025-09-27T16:49:23ZJoel Klinghedthe_jk@spawned.biz2025-09-27T16:25:10Zurn:sha1:2f13baa843bd1fb5db6630a2823681ffaff9fb11
Will be used by tokenizer for short lists of strings
java::uescape: Unicode reader that knows about Java's \uXXXX escapes2025-09-18T21:57:56ZJoel Klinghedthe_jk@spawned.biz2025-09-18T21:57:56Zurn:sha1:50348284f5d82ccfd65b0c803ba0ba895912ceffuline: Add unicode line reader2025-09-16T22:48:46ZJoel Klinghedthe_jk@spawned.biz2025-09-16T22:48:46Zurn:sha1:2a9e59adb5db8630ab7bdbdeedac623e3397989buio: Unicode reader2025-09-15T18:52:51ZJoel Klinghedthe_jk@spawned.biz2025-09-15T18:52:51Zurn:sha1:18a622f378b403788c67fc785d30f4609caa3fc7
Reads UTF-8 and UTF-16 into UTF-8 or UTF-16 strings.
If strict is true, fails at first invalid character.
If strict is false, invalid characters are replaced with U+FFFD.
For the replacement, I changed behavior if uN::read_replace to only
jump one byte. Otherwise a common invalid case when ISO-8859-1 or
WIN-1252 are read as UTF-8 would skip many characters.
If skip_bom is true any bom at start of stream is ignored.
If skip_bom is false any bom will be included.
Input format can be forced, if not detect is used which will
try to guess and then fallback to UTF-8.
Improve test coverage of io and unicode2025-09-10T21:57:26ZJoel Klinghedthe_jk@spawned.biz2025-09-10T21:57:26Zurn:sha1:28c6425e4ed1cd2eab538e7cba08c18aa83d8af5Add unicode general category lookup2025-09-10T20:12:22ZJoel Klinghedthe_jk@spawned.biz2025-09-10T20:12:22Zurn:sha1:32e14551a90e85000e41b3f0445d34d58a1431e4
Generate the lookup tables from UnicodeData.txt, do to that,
add gen_ugc, which uses csv, buffers, line, io and other modules
to do the job.
Disable RTTI and exceptions2025-09-08T21:10:04ZJoel Klinghedthe_jk@spawned.biz2025-09-08T21:10:04Zurn:sha1:f0ba930314e05c8d949ce92ffbda41fdb133198a
Are not going to use them