[36] Encoding は、 Web における文字符号化の扱いについて規定した仕様です。

仕様書

概念

[79]

歴史

[99] 元々 Web における文字コード処理の原則は参照処理モデルとして緩やかに確立されていったのですが、 その詳細が明確に標準化されているとは言い難い状態でした。

Web Encodings

[37] 当初 WHATWG Wiki の「Web Encodings」ページに annevk が Webブラウザーが実装する charset の名前の情報を集めていました。

またこの頃までに、公式な仕様と現実の Webブラウザーで文字符号化の実装が異なっており、 例えば ISO-8859-1 は現実には Windows-1252 であることなどが認識され、 HTML Living Standard の規定に組み込まれていきました。

  • [1] Web Encodings - WHATWG Wiki ( 版) http://wiki.whatwg.org/wiki/Web_Encodings
  • [2] [whatwg] Superset encodings [Re: ISO-8859-* and the C1 control range] ( 版) http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-July/021207.html
  • [3] IRC logs: freenode / #whatwg / 20090813 ( 版) http://krijnhoetmer.nl/irc-logs/whatwg/20090813
  • [4] Registering a charset alias ( 版) http://mail.apps.ietf.org/ietf/charsets/msg01830.html
  • [5] Re: Registering a charset alias ( 版) http://mail.apps.ietf.org/ietf/charsets/msg01868.html
  • [6] I18n:Charset Aliases - MozillaWiki ( 版) https://wiki.mozilla.org/I18n:Charset_Aliases
  • [7] (X)HTML5 Tracking ( 版) http://html5.org/tools/web-apps-tracker?from=4174&to=4175
  • [8] MIME charsets — coq.no ( 版) http://coq.no/character-tables/mime/en
  • [9] IRC logs: freenode / #whatwg / 20100908 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20100908#l-372
  • [10] Encodings: Analysis — Anne’s Blog ( ( 版)) http://annevankesteren.nl/2010/12/encodings-analysis
  • [11] annevk / webencodings / overview – Bitbucket ( ( 版)) https://bitbucket.org/annevk/webencodings
  • [12] IRC logs: freenode / #whatwg / 20101221 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20101221
  • [13] Web Encodings ( ( 版)) http://webencodings.org/
  • [14] Web Encodings ( ( 版)) http://webencodings.org/

Encoding Standard

[38] 2011年12月、 annevk は文字符号化の名前に加えて、符号化と復号の調査と文書化も始めました。

[39] 仕様書は W3C の Mercurial リポジトリーで管理されていましたが、 この時点では W3C の文書ではありませんでした。
  • [15] IRC logs: freenode / #whatwg / 20111212 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20111212#l-267
  • [16] Encoding Standard ( ( 版)) http://dvcs.w3.org/hg/encoding/raw-file/tip/Overview.html
  • [17] 8-bit labels and their encodings ( ( 版)) http://annevankesteren.nl/2010/8-bit-labels
  • [18] IRC logs: freenode / #whatwg / 20111215 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20111215
  • [19] annevk / webencodings / overview — Bitbucket ( ( 版)) https://bitbucket.org/annevk/webencodings
  • [20] Single-octet encoding research ( ( 版)) http://dvcs.w3.org/hg/encoding/raw-file/tip/single-octet-research.html
  • [21] Single-octet labels and their support various browsers ( ( 版)) http://lists.w3.org/Archives/Public/www-archive/2011Dec/att-0020/encoding-labels.html
  • [22] Single-octet labels and their support various browsers ( ( 版)) http://lists.w3.org/Archives/Public/www-archive/2011Dec/att-0021/encoding-labels.html
  • [23] [whatwg] Encodings and the web ( ( 版)) http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-December/034241.html
  • [24] [whatwg] Character-encoding-related threads ( ( 版)) http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-February/034818.html
  • [25] Web Applications 1.0 r6991 Rejig the wording of the character encoding section to make it more precise and in particular to not make CR processing require look-ahead. ( ( 版)) http://html5.org/tools/web-apps-tracker?from=6990&to=6991
  • [26] IRC logs: freenode / #whatwg / 20120410 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20120410
  • [27] [whatwg] Encoding Standard (mostly complete) ( 版) http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-April/035481.html
  • [28] Encoding - WHATWG Wiki ( ( 版)) http://wiki.whatwg.org/wiki/Encoding

[40] 2012年の春には TextEncoder/TextDecoder が提案されていますが、 その参照実装において Encoding 仕様に基づく符号化と復号が実装されています。

  • [29] stringencoding - String encoding API for the Web - Google Project Hosting ( ( 版)) http://code.google.com/p/stringencoding/

[41] 2012年4月末には仕様書は完成に至ります。ただし Big5 については実装間の互換性に乏しく、 実情も十分わかっていないということで、未確定の部分を残しています。

  • [30] Encodings: status update and big5 — Anne’s Blog ( ( 版)) http://annevankesteren.nl/2012/04/big5
  • [31] RE: Encoding Standard (mostly complete) ( ( 版)) http://mail.apps.ietf.org/ietf/charsets/msg02053.html
  • [32] 690225 – Universal XSS likely with MultiByte charset (e.g. japanese sites) ( ( 版)) https://bugzilla.mozilla.org/show_bug.cgi?id=690225

[34] 2012年4月26日には WHATCG の仕様となり、 CC0 から W3C CLA にライセンスが変更されました。また「Living Standard」から「Living Specification」 に変わっています。

[42] annevk は今後の展望として、文字符号化の sniffing の規定も含めたいと述べています。

[43] Microsoft は自社の実装を変更するつもりはないと述べています。

  • [33] IRC logs: freenode / #whatwg / 20120418 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20120418

[44] Re: Encoding Standard (mostly complete) ( ( 版)) http://mail.apps.ietf.org/ietf/charsets/msg02054.html

[87] Contributions by Opera (Anne van Kesteren 著, 版) http://lists.w3.org/Archives/Public/public-whatwg-contrib/2012Apr/0008.html

[45] IRC logs: freenode / #whatwg / 20120421 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20120421

[46] Bug 16773 – Expand the label list ( ( 版)) https://www.w3.org/Bugs/Public/show_bug.cgi?id=16773

[47] IRC logs: freenode / #whatwg / 20120620 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20120620#l-445

[48] Encoding Standard ( ( 版)) http://encoding.spec.whatwg.org/

[49] IRC logs: freenode / #whatwg / 20121010 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20121010

[50] IRC logs: freenode / #whatwg / 20121013 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20121013

[51] Single-byte encoding decoders: standard versus browsers ( ( 版)) http://lists.w3.org/Archives/Public/www-archive/2012Apr/att-0058/spectable.html

[52] IRC logs: freenode / #whatwg / 20121016 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20121016

[53] Web Applications 1.0 r7647 Embrace the Encodings specification. ( ( 版)) http://html5.org/tools/web-apps-tracker?from=7646&to=7647

[54] IRC logs: freenode / #whatwg / 20130124 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20130124#l-765

[55] Web Applications 1.0 r7782 Strip a leading BOM from scripts in workers, if any. Also, use more of the encoding spec. ( ( 版)) http://html5.org/tools/web-apps-tracker?from=7781&to=7782

[56] IRC logs: freenode / #whatwg / 20130329 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20130329#l-533

[57] [rust-dev] String encode/decode ( ( 版)) https://mail.mozilla.org/pipermail/rust-dev/2013-April/003575.html

[58] [whatwg] Requiring the Encoding Standard preferred name is too strict for no good reason ( ( 版)) http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039917.html

[59] Web Applications 1.0 r8081 Closer integration with encoding.spec.whatwg.org ( ( 版)) http://html5.org/tools/web-apps-tracker?from=8080&to=8081

[60] Web Applications 1.0 r8109 Allow any label for referencing encodings, not just the official name ( ( 版)) http://html5.org/tools/web-apps-tracker?from=8108&to=8109

[61] [whatwg] Requiring the Encoding Standard preferred name is too strict for no good reason ( ( 版)) http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/040313.html

[62] IRC logs: freenode / #whatwg / 20130912 ( ( 版)) http://krijnhoetmer.nl/irc-logs/whatwg/20130912

[63] Re: Encoding Standard ( (Anne van Kesteren 著, 版)) http://lists.w3.org/Archives/Public/public-i18n-core/2013OctDec/0005.html

[64] 805374 – Remove Character Encoding menu entries that are not in the Encoding Standard ( ( 版)) https://bugzilla.mozilla.org/show_bug.cgi?id=805374

[65] Web Applications 1.0 r3368 Clarify the encoding aliases mess. ( ( 版)) http://html5.org/tools/web-apps-tracker?from=3367&to=3368

[66] Encoding ( ( 版)) http://www.w3.org/TR/2014/WD-encoding-20140128/

[67] Encoding ( ( 版)) http://www.w3.org/TR/2014/WD-encoding-20140603/

[68] Encoding ( ( 版)) http://www.w3.org/TR/2014/CR-encoding-20140916/

[69] Encoding specification Disposition of Comments for 2014-06-03 LCWD ( ( 版)) http://www.w3.org/International/docs/encoding/encoding-doc.html

[70] Issue 412053 - chromium - Align windows-12xx and 874 with the encoding spec - An open-source project to help move the web forward. - Google Project Hosting ( ( 版)) https://code.google.com/p/chromium/issues/detail?id=412053

[71] Re: Call for Encoding Tests [I18N-ACTION-407] (Anne van Kesteren 著, 版) https://lists.w3.org/Archives/Public/www-international/2015JanMar/0078.html

[72] Encoding ( ( 版)) http://www.w3.org/TR/2015/CR-encoding-20151020/

[73] Fix #13: rename streaming flag to do not flush flag · whatwg/encoding@6de9a24 ( 版) https://github.com/whatwg/encoding/commit/6de9a2449937736d0e7f544791bb6c657fc3b47c

[74] Fix #12: clarify how decode() handles end-of-stream · whatwg/encoding@45431e6 ( 版) https://github.com/whatwg/encoding/commit/45431e6ce485ad50bc489b6e4cf46903398aff2a

[75] web-platform-tests/encoding at master · w3c/web-platform-tests ( ()) https://github.com/w3c/web-platform-tests/tree/master/encoding

[76] utf8.js/tests at master · mathiasbynens/utf8.js ( ()) https://github.com/mathiasbynens/utf8.js/tree/master/tests

[77] Search · user:mathiasbynens encoding standard ( ()) https://github.com/search?q=user%3Amathiasbynens+encoding+standard

[78] hsivonen/encoding_rs: An experimental Gecko-oriented implementation of the Encoding Standard in Rust () https://github.com/hsivonen/encoding_rs

[80] Encoding () https://www.w3.org/TR/2016/CR-encoding-20161110/

[81] Editorial: convert to Bikeshed (annevk著, ) https://github.com/whatwg/encoding/commit/bcc0f278c08856f67bdf28b6b82f5d3c482d5cb3

[82] Add visualizations for the indexes (hsivonen著, ) https://github.com/whatwg/encoding/commit/7696b7abf42ac13ccb59f183ee4e9e836ade13c2

[83] Meta: enable PR preview, add editor config, and link web-platform-tests (annevk著, ) https://github.com/whatwg/encoding/commit/24eb4377bd6d448e71d0f666b550fe9984bc9d05

[84] Meta: fix typo in test URL made in previous commit (annevk著, ) https://github.com/whatwg/encoding/commit/f2c66828cebdf1fd79b1735e76ee984b796ba8b5

[85] Encoding () https://www.w3.org/TR/2017/CR-encoding-20170413/

[86] web-platform-tests/encoding at master · w3c/web-platform-tests () https://github.com/w3c/web-platform-tests/tree/master/encoding

[88] Meta: HTML check only if needed, suppress warnings (sideshowbarker著, ) https://github.com/whatwg/encoding/commit/8785c3ffe476c1413bf71c1b16f4eecc916e0bc7

[89] Require utf-8 when specifying character encoding by sideshowbarker · Pull Request #3091 · whatwg/html () https://github.com/whatwg/html/pull/3091

[90] Host encoding.spec.whatwg.org (foolip著, ) https://github.com/whatwg/misc-server/commit/6e3ad9718916a6c13089eba7b90c244c2e2b3c8d

[91] Meta: use new WHATWG boilerplate and license (domenic著, ) https://github.com/whatwg/encoding/commit/d55cb85655eab662da03e3552275c035f1d91daa

[92] Meta: use new WHATWG boilerplate and license by domenic · Pull Request #132 · whatwg/encoding () https://github.com/whatwg/encoding/pull/132

[93] Be clearer on which user agents the API applies to (annevk著, ) https://github.com/whatwg/encoding/commit/e72995345427463fb81ec5811a4404d591b49296

[94] Non-browser versus JavaScript · Issue #87 · whatwg/encoding () https://github.com/whatwg/encoding/issues/87

[95] Be clearer on which user agents the API applies to by annevk · Pull Request #136 · whatwg/encoding () https://github.com/whatwg/encoding/pull/136

[96] Review Draft Publication: June 2018 (annevk著, ) https://github.com/whatwg/encoding/commit/19af047b7d3c669843f97ae18496951ebd3b5bc4

[97] Review Draft Publication: June 2018 by annevk · Pull Request #147 · whatwg/encoding () https://github.com/whatwg/encoding/pull/147

[98] Encoding Standard Review Draft June 2018 () https://encoding.spec.whatwg.org/review-drafts/2018-06/

[100] Give clearer advice on hooks for standards (annevk著, ) https://github.com/whatwg/encoding/commit/b579018b406d7752f8b7a3aa9c2bc800519c6f1a

[101] Give advice around specification hooks · Issue #125 · whatwg/encoding () https://github.com/whatwg/encoding/issues/125

[102] Give clearer advice on hooks for standards by annevk · Pull Request #137 · whatwg/encoding () https://github.com/whatwg/encoding/pull/137

[103] Be clearer on which user agents the API applies to (annevk著, ) https://github.com/whatwg/encoding/commit/e72995345427463fb81ec5811a4404d591b49296

[104] Non-browser versus JavaScript · Issue #87 · whatwg/encoding () https://github.com/whatwg/encoding/issues/87

[105] Be clearer on which user agents the API applies to by annevk · Pull Request #136 · whatwg/encoding () https://github.com/whatwg/encoding/pull/136

[106] Memorandum of Understanding Between W3C and WHATWG () https://www.w3.org/2019/04/WHATWG-W3C-MOU.html

[107] Review Draft Publication: December 2018 (annevk著, ) https://github.com/whatwg/encoding/commit/c2879dd1673704c51aac86fd4279598324ca792c

[108] Review Draft Publication: December 2018 by annevk · Pull Request #165 · whatwg/encoding () https://github.com/whatwg/encoding/pull/165

[109] Encoding Standard Review Draft December 2018 () https://encoding.spec.whatwg.org/review-drafts/2018-12/

[110] Review Draft Publication: June 2019 (annevk著, ) https://github.com/whatwg/encoding/commit/b4ff31c7c423acbe10052b63cbc5f433ae4cecb1

[111] Review Draft Publication: June 2019 by annevk · Pull Request #177 · whatwg/encoding () https://github.com/whatwg/encoding/pull/177

[112] Encoding Standard Review Draft June 2019 () https://encoding.spec.whatwg.org/review-drafts/2019-06/

[113] Encoding Standard の文字符号化を採用する、しないの基準は Webプラットフォームの他の機能同様、 (仕様書が形つくられた時点で) 複数の Webブラウザーで安定的に実装されているかどうかが基本になっているようですが、 これが Anne の官僚主義的な悪いところで、 古くから Webブラウザーがきっちり対応してきた欧米の文字符号化はマイナーなものまで細かくカバーしている一方で、 ハック的な方法 (例えば フォント依存符号化 ) で現地の人が対処してきて Webブラウザーがまともに取り組んで来なかった中央アジアや南アジアの文字符号化は地域内でメジャーでも省かれている、 という文化的不平等感、いわゆる欧米中心主義が技術的に固定化されてしまってるんですよねえ。