[FIG(quote)[
[FIGCAPTION[
[1] [CITE@EN[XQuery and XPath Full Text 3.0]]
([TIME[2015-11-20 06:32:18 +09:00]] 版)
<http://www.w3.org/TR/2015/REC-xpath-full-text-30-20151124/#TokenDef>
]FIGCAPTION]

> A token is a non-empty sequence of characters returned by a tokenizer as a basic unit to be searched. Beyond that, tokens are implementation-defined.

]FIG]