字句。
MIME と HTTP でやや定義が異なる。差は「{」「}」の有無。
RFC 2045 token = qr/[\x21\x23-\x27\x2A\x2B\x2D\x2E\x30-\x39\x41-\x5A\x5E-\x7E]+/;
RFC 2616 token = qr/[\x21\x23-\x27\x2A\x2B\x2D\x2E\x30-\x39\x41-\x5A\x5E-\x7A\x7C\x7E]+/;
CHAR = alpha | digit | separator | "!" | "#" | "$" | "%" | "&" | "'" | "*" | "+" | "-" | "." | "`" | "^" | "_" | "{" | "|" | "}" | "~" | CTL CTL = <any control character> separator = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <"> | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT token = 1*<any CHAR except CTLs or separators>
[3] この定義を ASCII に当てはめると RFC 2045 の定義と一致します。 (なぜか RFC 2616 の定義ではありません。)