tokenizer

名词 n.
中文释义

英文释义

名词 n.
  1. A system that parses an input stream into its component tokens.

词形变化

tokenizers plural

词源

Etymology tree
English tokenize
Proto-Indo-European *-yósder.
Proto-Italic *-āzijos
Latin -āriusnom.
Latin -āriusbor.
Proto-Germanic *-ārijaz
Proto-West Germanic *-ārī
Old English -ere
Middle English -ere
English -er
English tokenizer
From tokenize + -er.
0 次浏览 数据来源: Wiktionary