packages / ply

ucto

advanced rule-based (regular-expression) and unicode-aware tokenizer for various languages

license: GPL-3.0-onlyhttps://github.com/LanguageMachines/ucto

use it

[dependencies]
ucto = "0.33"

versions