Edit

Page

Page title:

Page body:

[1] [[改行]]区切りの [[JSON]] 値の列が [[JSON]] によって表現可能なデータの[[交換][情報交換]]に使われることがあります。
これを[DFN[[[改行]]区切りの [[JSON]]]]、[DFN[[[LDJSON]]]]、[DFN[[[NDJSON]]]]、
[DFN[[[JSON Lines]]]] などと呼ぶことがあります。

;; [85] 統一的な名前はなく色々な呼ばれ方をしています。
特に名称なしで利用されていることもあります。

[101] いろいろな[[アプリケーション]]で採用されていて、
[[JSONデータのストリーミング転送][JSONストリーミング]]の[[事実上の標準]]となっています。

* 構文

[2] 0個以上の [[JSON]] 値を[[改行]]で区切ります。

[FIG(railroad)[
= ?
== [[JSON]] 値
== *
=== [[改行]]
=== [[JSON]] 値
]FIG]

[3] 連続した[[改行]] (= [[空行]]) やファイル末尾の[[改行]]をどう解釈するかは、実装によります。

[4] [[改行]]が [[LF]] か [[CRLF]] か、いずれでもよいかは実装によります。

;; [38] [[LF]] を区切りとして解釈すれば、 [[CR]] は [[JSON]] の[[空白]]なので、
自然に無視されます。

[5] [[JSON]] 値には[[字句]]間の[[空白]]として[[改行]]を含めることも本来は可能ですが、
本形式では使用しません。これにより、1行 = 1 [[JSON]] 値となり、処理しやすくなります。

;; [6] [[JSON]] 値における[[空白]]は[[人間]]の[[可読性]]の向上以外に意味を持たないので、
[[情報交換]]用には不要です。なお[[文字列]]中の[[改行]]は [[JSON]] 値内では必ず[[エスケープ]]が必要なので、
元々[[改行]]そのままの形で現れることはありません。
(ですから、[[改行]]が含まれる [[JSON]] 値から何も考えずに[[改行]]を除去すれば、
ここで使える[[JSON]]値となります。)

;; [7] [[JSON]] 値内部の[[改行]]をそのままにしても、意図通り[[構文解析]]することは可能ではあります。
[[行指向]]の処理はできなくなりますが、[[JSON]]値の終わりが現れる行までを読み込んで1つの
[[JSON]]値として処理するだけです。
なお [[Wikipedia]] はそのようなものも含め、区切りを含まないものを
[[Concatenated JSON]] [SRC[>>44]] と呼んでいます。

[37] [[JSON]] 値としてすべて認めるものと、[[オブジェクト]]や[[配列]]など特定の形のもののみ認めるものがあります。

[50] [[文字コード]]としては、 [[UTF-8]] を使うのが普通です。

* 処理

[86] 入力を[[行]]ごとに、それぞれ [[JSON]] として[[復号]]していきます。

[87] [[行指向]]の処理は [[Unix]] などで一般的に用いられており、
ほとんどの[[プログラミング言語]]の[[標準ライブラリー]]で簡単に実装できます。

[88] [[JSON]] の処理も、特別なことはせず、単に [[JSON]] から内部データ構造に変換する一般的な[[ライブラリー]]を呼び出すだけで十分です。

[89] 後は、利用する場面に応じて[[空行]]を受信した時の処理、
不正な [[JSON]] を受信した時の処理、
入力ストリームの末端に到達した時の処理を実装し忘れないように注意しましょう。

;; [90] こうした処理をまとめて行う専用の[[ライブラリー]]もいくつもありますが、
その場その場で都度実装しても良いくらい単純な処理で済みます。

* MIME 型

[51] [[NDJSON]] 仕様は [[MIME型]]として [DFN[[CODE(MIME)@en[[[application/x-ndjson]]]]]]
を規定しています [SRC[>>23]]。

[56] [[Wikipedia]] は [[MIME型]]として [DFN[[CODE(MIME)@en[[[application/x-ldjson]]]]]]
を使うべきと述べています [SRC[>>14]]。

[62] たまに [CODE(MIME)@en[[[application/json]]]] が使われることもあります。 (正しくはありません。)

* 拡張子

[108] 
[DFN[[CODE[.jsonl]]]],
[DFN[[CODE[.jsonlines]]]],
[DFN[[CODE[.ld]]]],
[DFN[[CODE[.ldjson]]]]
が使われているようです。

[109] 
[CODE[.json]]
が使われることもあるようです。
ただの
[[JSON]]
と紛らわしいので、避けるべきでしょう。

* 実装

[10] [[jq]] は、結果が複数の [[JSON]] 値となるとき[[改行]]区切りで出力します。
ただし、標準では [[JSON]] 値に[[改行]]を含む[[空白]]を入れて[[人間]]が見やすい形にします。
[CODE[-c]] / [CODE[--compact-output]] オプションを指定すると、[[空白]]のない、
本項の形式となります。

[12] [[Twitter]] は [[CRLF]] 区切りの [[JSON]] 値を使っています [SRC[>>292]]。
[[ツイート]]に [[LF]] が含まれる可能性がある ([[CR]] が含まれる可能性はない) [SRC[>>292]]
とされていますが、[[文字列]]に生の [[LF]] が含まれることがあるのか、それとも字句間に含まれることがあるだけなのかは不明です (前者は [[JSON]] 仕様上あり得ないはずですが)。

[13] [[Twitter]] はオプション形式として、[[JSON]]値の前に [[JSON]]値の長さを[[十進数]]で出力する形式も用意しています [SRC[>>292]]。
[[CRLF]] の出現を探すのではなく、指定文字列長分を読み込む方式の方が都合がいい実装のために用意されているようです。

[REFS[
- [292] [CITE[Processing streaming data | Twitter Developers]] ([TIME[2015-03-05 11:53:33 +09:00]] 版) <https://dev.twitter.com/streaming/overview/processing>
]REFS]

[83] >>82 は、[[改行]]区切りにされることがよくあるが、
他の[[空白]]区切りでも (曖昧でなければ[[空白]]なしでも) 受け付ける、としています。
この形式は他の実装では [[Concatenated JSON]] と呼ばれています。

[11] その他次の実装が[[改行]]区切りの [[JSON]] 値列を使っています。

[REFS[
- [291] [CITE[Bulk API]]
([TIME[2015-02-24 04:40:27 +09:00]] 版)
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html>
]REFS]

[63] いろいろな [[Webアプリケーション]]の [[Web API]] が本項の形式 (の何らかのバリエーション)
を採用しています。

[57] [[Chrome]] の[[開発者ツール]]は[[応答]]が [[JSON]] の列の時も [[JSON]] 値の[[配列]]のように表示できます。

* 文脈

[114] 
[CITE@en-US[DynamoDB table export output format - Amazon DynamoDB]], [TIME[2024-03-05T19:46:54.000Z]], [TIME[2024-03-06T03:52:07.270Z]] <https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataExport.Output.html>

* 関連

[8] [[JSON]] 値の列全体を [[JSON]] [[配列]]とすることも可能ですが、
末尾まで読み終えてから処理するのでない ([[ストリーミング]]処理したい)
場合に扱いづらいのが問題です。

[84] [[JSON]] をストリーミング形式で構文解析することも可能で、
そのような実装も存在はしますが、本方式の方がずっと簡単に実現できます。
[WEAK[(ストリーミングに対応していない普通の [[JSON]] の実装を流用できます。)]]

[9] [[JSON text sequences]] 
([[RFC 7464]])
は本項の方式から派生したもののようですが、
特殊な[[制御文字]]を使っています。本項の方式と比べてそこまで複雑でもありませんが、
メリットも特にありません。本項の方式の方が [[Unix]] その他の[[行指向]]の既存の[[ライブラリー]]等との親和性は高そうです。

;; 本項の方式と [[JSON text sequences]] を混同した解説の類もありますから、
要注意。

* 歴史

[81] [[NDJSON]] は、昔は [[LDJSON]] と言っていました [SRC[>>80]]。

[REFS[
- [80] [CITE@en[Update specification_draft2.md · ndjson/ndjson-spec@c658c26]] ([TIME[2016-02-21 20:56:12 +09:00]] 版) <https://github.com/ndjson/ndjson-spec/commit/c658c26ba15bea718228b5a3d54f0d4db22ec0ba>
]REFS]

[14] [CITE@en[Line Delimited JSON - Wikipedia, the free encyclopedia]] ([TIME[2015-02-24 09:07:32 +09:00]] 版) <http://en.wikipedia.org/wiki/Line_Delimited_JSON>

[15] [CITE[standard content-type for "streaming JSON" (newline delimited JSON objects)? - Google グループ]]
([TIME[2015-03-13 19:49:54 +09:00]] 版)
<https://groups.google.com/forum/#!topic/nodejs/0ohwx0vF-SY>

[16] [CITE[websocket - Choice of transports for JSON over TCP - Stack Overflow]]
([TIME[2015-03-13 19:47:01 +09:00]] 版)
<http://stackoverflow.com/questions/6573870/choice-of-transports-for-json-over-tcp>

[17] [CITE[How We Built Filmgrain, Part 2 of 2 | Filmgrain Blog]]
([TIME[2013-09-15 11:20:31 +09:00]] 版)
<http://blog.filmgrainapp.com/2013/07/02/how-we-built-filmgrain-part-2-of-2/>

[18] [CITE@en[ldjson-stream]]
([TIME[2015-03-13 19:51:08 +09:00]] 版)
<https://www.npmjs.com/package/ldjson-stream>

[19] [CITE@en[maxogden/ndjson]]
([TIME[2015-03-13 19:51:19 +09:00]] 版)
<https://github.com/maxogden/ndjson>

[55] [CODE[[[mongoimport]]]] [SRC[>>53]] / [CODE[[[mongoexport]]]] [SRC[>>54]]
における [[JSON]] とは原則として[[改行]]区切りの [[JSON]] の列のことを言っています。

[REFS[
- [53] [CITE@en[mongoimport — MongoDB Manual 3.0.1]] ([TIME[2015-03-28 05:46:01 +09:00]] 版) <http://docs.mongodb.org/manual/reference/program/mongoimport/>
- [54] [CITE@en[mongoexport — MongoDB Manual 3.0.1]] ([TIME[2015-03-28 05:45:59 +09:00]] 版) <http://docs.mongodb.org/manual/reference/program/mongoexport/>
]REFS]

[20] [CITE@en[ld-jsonstream]]
([TIME[2015-03-13 19:51:24 +09:00]] 版)
<https://www.npmjs.com/package/ld-jsonstream>

[21] [CITE@en[timkuijsten/node-ld-jsonstream]]
([TIME[2015-03-13 19:51:34 +09:00]] 版)
<https://github.com/timkuijsten/node-ld-jsonstream>

[22] [CITE[how to parse a large, Newline-delimited JSON file by JSONStream module in node.js? - Stack Overflow]]
([TIME[2015-03-13 19:51:46 +09:00]] 版)
<http://stackoverflow.com/questions/15121584/how-to-parse-a-large-newline-delimited-json-file-by-jsonstream-module-in-node-j>

-[23] [CITE@en[NDJSON - Newline delimited JSON - Data Protocols - Open Knowledge Foundation]]
([[Open Knowledge Foundation and others]] 著, [TIME[2015-03-04 16:29:27 +09:00]] 版)
<http://dataprotocols.org/ndjson/>
-[117] [CITE@en[NDJSON - Newline delimited JSON - Data Protocols - Open Knowledge Labs]], [[Open Knowledge Foundation and others]], [TIME[2024-03-21T13:36:47.000Z]], [TIME[2017-07-13T05:47:17.303Z]] <https://web.archive.org/web/20170713054509/http://specs.okfnlabs.org/ndjson/index.html>

[SEE[ [[Data Protocols]] ]]

[24] [CITE[Storing Data as Newline Delimited JSON Entries - Will Anderson]]
([[Will Anderson]] 著, [TIME[2015-02-27 16:14:27 +09:00]] 版)
<http://willi.am/blog/2014/07/16/storing-data-as-newline-delimited-json-entries/>

[25] [CITE[JSON Lines]]
([TIME[2015-02-25 07:23:03 +09:00]] 版)
<http://jsonlines.org/>

[26] [CITE[ndjson]]
([TIME[2015-01-02 21:04:22 +09:00]] 版)
<http://ndjson.org/>

[FIG(quote)[
[FIGCAPTION[
[27] [CITE[Streaming API | Plotly Developers]]
([TIME[2015-03-13 19:59:16 +09:00]] 版)
<https://plot.ly/streaming/>
]FIGCAPTION]

> Once the stream has been connected over http, write to the request stream with newline separated JSON.
> httpRequestSocket.write('{ "x": 3, "y": 1 }\n')
> The newline is extremely important. Without this delimiter the Streaming Endpoint will not delineate your data, and will terminate the stream. You can send multiple streams to the same plot by nesting stream tokens within the corresponding data trace object. Similarly you can use the same token for multiple traces in a plot (they will show the same stream, so this is useful only in when using subplots).

]FIG]

[52] [CITE@en[draft-hallambaker-jsonl-01 - JSON Log Format (JSON-L)]]
( ([TIME[2014-05-07 04:59:18 +09:00]] 版))
<http://tools.ietf.org/html/draft-hallambaker-jsonl-01>

[28] [CITE[Lesspress.net]]
([[Christopher Brown, io@henrian.com]] 著, [TIME[2014-12-04 07:34:04 +09:00]] 版)
<http://lesspress.net/#38>

[29] [CITE@en[ndjson/ndjson-spec]]
([TIME[2015-03-13 20:44:22 +09:00]] 版)
<https://github.com/ndjson/ndjson-spec>

[FIG(quote)[
[FIGCAPTION[
[30] [CITE@en[dat/rest-api.md at master · maxogden/dat]]
([TIME[2015-03-13 20:46:00 +09:00]] 版)
<https://github.com/maxogden/dat/blob/master/docs/rest-api.md#post-apibulk>
]FIGCAPTION]

> Data can be either newline-delimited JSON (with content-type application/json) or CSV (with content-type text/csv).
> The response will be a stream of newline-delimited JSON objects.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[31] [CITE@en[Bubbles: Python ETL Framework (prototype) - Open Knowledge Labs]]
([TIME[2015-03-10 23:46:38 +09:00]] 版)
<http://okfnlabs.org/blog/2014/09/01/bubbles-python-etl.html>
]FIGCAPTION]

> JSON newlline delimited file in a store represented by a directory of JSOND files

]FIG]

[FIG(quote)[
[FIGCAPTION[
[32] [CITE[logstash - open source log management]]
([TIME[2015-03-03 06:25:28 +09:00]] 版)
<http://logstash.net/docs/1.4.2/codecs/json_lines>
]FIGCAPTION]

> This codec will decode streamed JSON that is newline delimited. For decoding line-oriented JSON payload in the redis or file inputs, for example, use the json codec instead. Encoding will emit a single JSON string ending in a ‘\n’

]FIG]

[FIG(quote)[
[FIGCAPTION[
[33] [CITE[Storing alerts as JSON — OSSEC 2.8.1 documentation]]
([TIME[2015-03-09 23:01:50 +09:00]] 版)
<http://ossec-docs.readthedocs.org/en/latest/manual/output/json-alert-log-output.html>
]FIGCAPTION]

> With the json output, you can write alerts as a newline separated json file which other programs can easily consume.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[34] [CITE[Class: Traject::MarcReader — Documentation for traject/traject (master)]]
([TIME[2015-03-13 20:50:41 +09:00]] 版)
<http://www.rubydoc.info/github/traject/traject/master/Traject/MarcReader>
]FIGCAPTION]

> "json" The "marc-in-json" format, encoded as newline-separated json. (synonym 'ndj'). A simplistic newline-separated json, with no comments allowed, and no unescpaed internal newlines allowed in the json objects -- we just read line by line, and assume each line is a marc-in-json.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[35] [CITE[log-defer-viz - search.cpan.org]]
([TIME[2015-03-13 20:52:16 +09:00]] 版)
<http://search.cpan.org/dist/Log-Defer-Viz/bin/log-defer-viz>
]FIGCAPTION]

> INPUT FORMAT
>     $ log-defer-viz --input-format=json  ## default is newline separated JSON

]FIG]

[36] [CITE[python-stream-json]]
([TIME[2013-12-04 07:00:02 +09:00]] 版)
<http://www.enricozini.org/2011/tips/python-stream-json/>

[39] [CITE@en[Feed exports — Scrapy 0.24.5 documentation]]
([TIME[2015-02-26 06:37:57 +09:00]] 版)
<http://doc.scrapy.org/en/latest/topics/feed-exports.html#topics-feed-format-jsonlines>

[40] [CITE[Add support for the JSON lines format. ''''''[''''''#937474'''''']'''''' | Drupal.org]]
([TIME[2015-03-13 21:55:28 +09:00]] 版)
<https://www.drupal.org/node/937474>

[FIG(quote)[
[FIGCAPTION[
[41] [CITE@en[OpenCPU - High performance JSON streaming in R: Part 1]]
([[Jeroen Ooms]] 著, [TIME[2015-03-12 16:39:22 +09:00]] 版)
<https://www.opencpu.org/posts/jsonlite-streaming/>
]FIGCAPTION]

> The json streaming format
> Because parsing huge JSON strings is difficult and inefficient, JSON streaming is done using lines of minified JSON records.

]FIG]

[42] [CITE@en[bitdivine/jline]]
([TIME[2015-03-13 22:03:56 +09:00]] 版)
<https://github.com/bitdivine/jline>

[FIG(quote)[
[FIGCAPTION[
[43] [CITE[API Basics — Scrapinghub documentation]]
([TIME[2015-03-05 23:55:58 +09:00]] 版)
<http://doc.scrapinghub.com/api.html#data-formats>
]FIGCAPTION]

> To better support streaming with many popular JSON parsers, we provide JSON Lines format by default, but JSON and CSV are also available.

]FIG]

[44] [CITE@en[JSON Streaming - Wikipedia, the free encyclopedia]]
([TIME[2015-03-21 21:13:36 +09:00]] 版)
<http://en.wikipedia.org/wiki/JSON_Streaming>

[FIG(quote)[
[FIGCAPTION[
[45] [CITE@en[Streaming | Flowdock API]]
([TIME[2015-03-29 18:06:53 +09:00]] 版)
<https://www.flowdock.com/api/streaming>
]FIGCAPTION]

> JSON Stream (application/json)
> The JSON stream returns messages as JSON objects that are delimited by carriage return (\r). Newline characters (\n) may occur in messages, but carriage returns should not.
> Parsers must be tolerant of occasional extra newline characters placed between messages. These characters are sent as periodic "keep-alive" messages to tell clients and NAT firewalls that the connection is still alive during low message volume periods.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[46] [CITE[The JSON Streaming Record (JSRec) data format « Huy Nguyen]]
([TIME[2014-07-23 13:00:27 +09:00]] 版)
<http://www.huyng.com/posts/json-streaming-record-data-format/>
]FIGCAPTION]

> Files of this format have .jsrec as their file extension
> Each line in the file is a json hash map
> Empty lines and lines beginning with ‘#’ are considred comments and ignored during parsing

]FIG]

[FIG(quote)[
[FIGCAPTION[
[47] [CITE@en[Docker Remote API v1.17 - Docker Documentation]]
([TIME[2015-03-26 12:53:56 +09:00]] 版)
<https://docs.docker.com/reference/api/docker_remote_api_v1.17/>
]FIGCAPTION]

>     Content-Type: application/json
>     {"stream": "Step 1..."}
>     {"stream": "..."}
>     {"error": "Error...", "errorDetail": {"code": 123, "message": "Error..."}}

]FIG]

[FIG(quote)[
[FIGCAPTION[
[48] [CITE@en[Extend your community - Meetup]]
([TIME[2015-03-29 18:43:17 +09:00]] 版)
<http://www.meetup.com/meetup_api/docs/stream/2/open_events/>
]FIGCAPTION]

> A response message is one HTTP chunk, the body of which is a single json object, described below, terminated by a newline.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[49] [CITE@en[Streaming]]
([TIME[2015-02-21 05:45:15 +09:00]] 版)
<http://developer.oanda.com/rest-live/streaming/>
]FIGCAPTION]

> All data written to the stream are encoded in the JSON format. Events sent to the stream are either heartbeats (every 15 seconds) to ensure that HTTP connection remains active or transactions reporting the following events:

]FIG]

[58] [CITE@en[Talk:Line Delimited JSON - Wikipedia, the free encyclopedia]]
([TIME[2015-02-26 19:59:49 +09:00]] 版)
<http://en.wikipedia.org/wiki/Talk:Line_Delimited_JSON>

[FIG(quote)[
[FIGCAPTION[
[59] [CITE[Catmandu::Importer::JSON - search.cpan.org]]
([TIME[2015-03-29 20:19:56 +09:00]] 版)
<http://search.cpan.org/dist/Catmandu/lib/Catmandu/Importer/JSON.pm>
]FIGCAPTION]

> multiline switches optionally between line-delimited JSON and multiline JSON or arrays. the default is line-delimited JSON.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[60] [CITE@en[Farsight Security/Blog/NMSG and JSON encoding]]
([[Mike Schiffman]] 著, [TIME[2015-06-02 08:28:56 +09:00]] 版)
<https://www.farsightsecurity.com/Blog/20150506-mschiffm-nmsg-nmsgjsontool/>
]FIGCAPTION]

> Currently, the only type of JSON njt expects and emits is newline-delimited JSON. Each record is expected to be terminated with a literal \n and no \n's can appear inside the JSON.

]FIG]

[61] [CITE@en[Re: Please help with writing spec for async JSON APIs]]
([TIME[2015-08-18 01:44:10 +09:00]] 版)
<https://www.mail-archive.com/es-discuss@mozilla.org/msg36495.html>

[FIG(quote)[
[FIGCAPTION[
[64] [CITE[Streaming API | DataSift Developers]]
([TIME[2015-08-28 21:50:10 +09:00]] 版)
<http://dev.datasift.com/docs/api/streaming-api>
]FIGCAPTION]

> The response body contains a list of JSON objects separated by new lines.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[65] [CITE[pull | DataSift Developers]]
([TIME[2015-08-28 23:07:20 +09:00]] 版)
<http://dev.datasift.com/docs/api/rest-api/endpoints/pull>
]FIGCAPTION]

> 
> X-DataSift-Format: json_new_line
> Content-Type: application/ldjson

]FIG]

[FIG(quote)[
[FIGCAPTION[
[66] [CITE@en[Preparing Data for BigQuery - BigQuery — Google Cloud Platform]]
([[Google]] 著, [TIME[2015-09-17 06:50:38 +09:00]] 版)
<https://cloud.google.com/bigquery/preparing-data-for-bigquery>
]FIGCAPTION]

> One JSON object, including any nested/repeated fields, must appear on each line.
> The following example shows sample nested/repeated data:
> {"kind": "person", "fullName": "John Doe", "age": 22, "gender": "Male", "citiesLived": '''['''{ "place": "Seattle", "numberOfYears": 5}, {"place": "Stockholm", "numberOfYears": 6}''']'''}
> {"kind": "person", "fullName": "Jane Austen", "age": 24, "gender": "Female", "citiesLived": '''['''{"place": "Los Angeles", "numberOfYears": 2}, {"place": "Tokyo", "numberOfYears": 2}''']'''}
>

]FIG]

[FIG(quote)[
[FIGCAPTION[
[67] [CITE[ニコニコ動画 『スナップショット検索API』 ガイド]]
([TIME[2015-09-15 12:49:32 +09:00]] 版)
<http://search.nicovideo.jp/docs/api/snapshot.html#toc5>
]FIGCAPTION]

> 【検索レスポンス】
> {"dqnid":"c0676eea-cc77-4317-b442-d626c5f34558","type":"hits","values":'''['''{"_rowid":0,"cmsid":"sm13208019","title":"【DIVA 2nd】 鏡音八八花合戦 【EDIT PV】","view_counter":9999},{"_rowid":1,"cmsid":"sm12215733","title":"メイドイン俺でミニゲーム その5","view_counter":9997},{"_rowid":2,"cmsid":"sm3495465","title":"【初音ミクオリジナル】~プレゼント~【Independence Free】","view_counter":9997}''']'''}
> {"dqnid":"c0676eea-cc77-4317-b442-d626c5f34558","type":"stats","values":'''['''{"_rowid":0,"service":"video","total":213353}''']'''}
> {"dqnid":"c0676eea-cc77-4317-b442-d626c5f34558","endofstream":true,"type":"hits"}
> {"dqnid":"c0676eea-cc77-4317-b442-d626c5f34558","endofstream":true,"type":"stats"}
> 検索レスポンス仕様
> 改行(\n)区切りを一つのチャンクとして、いくつかのチャンクに分かれて返ってきます。

]FIG]

[68] [CITE@en[thenativeweb/json-lines-client]]
([TIME[2015-10-09 23:01:06 +09:00]] 版)
<https://github.com/thenativeweb/json-lines-client>

[FIG(quote)[
[FIGCAPTION[
[69] [CITE[API Basics — Scrapinghub documentation]]
([TIME[2015-10-09 21:51:46 +09:00]] 版)
<http://doc.scrapinghub.com/api.html>
]FIGCAPTION]

> To better support streaming with many popular JSON parsers, we provide JSON Lines format by default, but JSON and CSV are also available.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[70] [CITE@en[mattdesl/garnish]]
([TIME[2015-11-29 22:52:44 +09:00]] 版)
<https://github.com/mattdesl/garnish>
]FIGCAPTION]

> Typically, you would use bole or ndjson to write the content to garnish. You can also write ndjson to stdout like so:

]FIG]

[FIG(quote)[
[FIGCAPTION[
[71] [CITE@en[rvagg/bole]]
([TIME[2015-11-29 22:53:23 +09:00]] 版)
<https://github.com/rvagg/bole>
]FIGCAPTION]

> Newline separated JSON output to arbitrary streams

]FIG]

[72] [CITE[JSON Lines | Hacker News]]
([TIME[2015-11-29 22:59:09 +09:00]] 版)
<https://news.ycombinator.com/item?id=10280483>

[73] [CITE[Storing Data as Newline Delimited JSON Entries - Will Anderson]]
([[Will Anderson]] 著, [TIME[2015-09-15 10:20:59 +09:00]] 版)
<http://willi.am/blog/2014/07/16/storing-data-as-newline-delimited-json-entries/>

[FIG(quote)[
[FIGCAPTION[
[74] [CITE[json(1) - JSON love for your command line]]
([TIME[2014-08-25 16:28:55 +09:00]] 版)
<http://trentm.com/json/>
]FIGCAPTION]

> "Adjacent" objects means objects separated by a newline, or by no space at all. Adjacent arrays means separate by a newline. These conditions are chosen as a balance between (a) not being ambiguous to parse with a simple regex and (b) enough to be useful for common cases.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[75] [CITE@en[Everyone loves R markdown and Github; stories from the R Summit, day two | 4D Pie Charts]]
([TIME[2015-11-29 23:04:26 +09:00]] 版)
<http://4dpiecharts.com/2015/06/28/everyone-loves-r-markdown-and-github-stories-from-the-r-summit-day-two/>
]FIGCAPTION]

> jsonlite also supports ndjson, where each line is a JSON object. This is important for large files: you can just parse one line at a time, then return the whole thing as a list or data frame, or you can define a line specific behaviour.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[76] [CITE[R: Streaming JSON input/output]]
([TIME[2015-11-29 23:05:45 +09:00]] 版)
<http://finzi.psych.upenn.edu/library/jsonlite/html/stream_in.html>
]FIGCAPTION]

> Because parsing huge JSON strings is difficult and inefficient, JSON streaming is done using lines of minified JSON records, a.k.a. ndjson. This is pretty standard: JSON databases such as dat or MongoDB use the same format to import/export datasets. Note that this means that the total stream combined is not valid JSON itself; only the individual lines are. Also note that because line-breaks are used as separators, prettified JSON is not permitted: the JSON lines must be minified.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[77] [CITE@en[stephenplusplus/jsonl]]
([TIME[2015-11-29 23:09:11 +09:00]] 版)
<https://github.com/stephenplusplus/jsonl>
]FIGCAPTION]

> Transform a stream of JSON into a stream of Line Delimited JSON

]FIG]

[FIG(quote)[
[FIGCAPTION[
[78] [CITE[embulk-parser-jsonlをリリースしました! - Qiita]]
([TIME[2015-11-29 23:09:57 +09:00]] 版)
<http://qiita.com/shun0102/items/8989e6ed2ee0f46a0fa9>
]FIGCAPTION]

> 僕がリリースしたのはjsonlということで、1行につき1つのJSONというフォーマットを読み込むためのparserです。

]FIG]

[79] [CITE@en[axiaoxin/vim-json-line-format]]
([TIME[2015-12-29 18:12:22 +09:00]] 版)
<https://github.com/axiaoxin/vim-json-line-format>

[FIG(quote)[
[FIGCAPTION[
[82] [CITE[Orchestrate]]
([[Orchestrate]] 著, [TIME[2016-03-08 10:22:00 +09:00]] 版)
<https://orchestrate.io/docs/apiref#bulk>
]FIGCAPTION]

> Or alternatively, to send a JSON stream, set the Content-Type header to "application/orchestrate-export-stream+json". JSON stream data is often sent with one JSON object per line, using newline characters to delimit adjacent objects, but this isn't strictly necessary with Orchestrate. You can format the data with arbitrary whitespace (or put them direclty adjacent to one another, as long as the stream can be parsed as a sequence of valid JSON objects.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[91] [CITE@en[Event endpoint · Appuri]]
( ([TIME[2016-08-24 16:22:38 +09:00]]))
<http://developer.appuri.com/v1/docs/event-endpoint>
]FIGCAPTION]

> Events are sent via an HTTPS POST request with the content type set to application/x-ldjson (Line-Delimited JSON).

]FIG]

[FIG(quote)[
[FIGCAPTION[
[92] [CITE@en[REST API]]
( ([TIME[2016-02-27 07:16:28 +09:00]]))
<http://quasar-analytics.org/docs/restapi/>
]FIGCAPTION]

> The following values are supported for the Accept header:
> Value	Description
> None	“Human-Readable” results, one result per line. Note: not parseable as a single JSON object.
> application/json	Nicely formatted JSON array
> application/ldjson;mode=precise	One result per line

]FIG]

[FIG(quote)[
[FIGCAPTION[
[93] [CITE@en[REST API]]
( ([TIME[2016-02-27 07:16:28 +09:00]]))
<http://quasar-analytics.org/docs/restapi/>
]FIGCAPTION]

> Note that if you remove the Accept header, then you will receive Precise JSON, which is the default format. The response would then look like this:
> { "city": "WESTOVER AFB", "state": "MA", "pop": 1764, "loc": '''[''' -72.558657, 42.196672 ''']''' }
> { "city": "CUMMINGTON", "state": "MA", "pop": 1484, "loc": '''[''' -72.905767, 42.435296 ''']''' }

]FIG]

[FIG(quote)[
[FIGCAPTION[
[94] [CITE@en[trentm/node-bunyan: a simple and fast JSON logging module for node.js services]]
( ([TIME[2016-10-17 20:37:05 +09:00]]))
<https://github.com/trentm/node-bunyan>
]FIGCAPTION]

> Bunyan log records are JSON. A few fields are added automatically: "pid", "hostname", "time" and "v".
> $ node hi.js
> {"name":"myapp","hostname":"banana.local","pid":40161,"level":30,"msg":"hi","time":"2013-01-04T18:46:23.851Z","v":0}
> {"name":"myapp","hostname":"banana.local","pid":40161,"level":40,"lang":"fr","msg":"au revoir","time":"2013-01-04T18:46:23.853Z","v":0}
>

]FIG]

[FIG(quote)[
[FIGCAPTION[
[95] [CITE@en[trentm/node-bunyan: a simple and fast JSON logging module for node.js services]]
( ([TIME[2016-10-17 20:37:28 +09:00]]))
<https://github.com/trentm/node-bunyan>
]FIGCAPTION]

> Bunyan log output is a stream of JSON objects.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[96] [CITE[AnyEvent::Handle - search.cpan.org]]
( ([TIME[2016-10-26 11:48:41 +09:00]]))
<http://search.cpan.org/dist/AnyEvent/lib/AnyEvent/Handle.pm>
]FIGCAPTION]

> A simple RPC protocol that interoperates easily with other languages is to send JSON arrays (or objects, although arrays are usually the better choice as they mimic how function argument passing works) and a newline after each JSON text:

]FIG]

[FIG(quote)[
[FIGCAPTION[
[97] [CITE[Documentation]]
([TIME[2016-09-27 20:12:27 +09:00]])
<http://docs.appbase.io/scalr/rest/intro.html#quick-start-to-the-rest-api-step-1-making-requests>
]FIGCAPTION]

> In the new document update, we can see the price change (5595 -> 6034) being reflected. Subsequent changes will be streamed to the resonse as raw JSON objects. As we see, there are no delimiters between between two consecutive JSON responses.

]FIG]

[98] [CITE[Fun hacks for faster content - JakeArchibald.com]]
([TIME[2016-12-20 19:09:26 +09:00]])
<https://jakearchibald.com/2016/fun-hacks-faster-content/#newline-delimited-json>

[FIG(quote)[
[FIGCAPTION[
[99] [CITE@en[Streaming | Flowdock API]]
([TIME[2016-12-21 22:49:33 +09:00]])
<https://www.flowdock.com/api/streaming>
]FIGCAPTION]

> JSON Stream (application/json)
> The JSON stream returns messages as JSON objects that are delimited by carriage return (\r). Newline characters (\n) may occur in messages, but carriage returns should not.
> Parsers must be tolerant of occasional extra newline characters placed between messages. These characters are sent as periodic “keep-alive” messages to tell clients and NAT firewalls that the connection is still alive during low message volume periods.

]FIG]

[FIG(quote)[
[FIGCAPTION[
[100] [CITE@en[Event V1/V2 (deprecated) - SendGrid Documentation | SendGrid]]
( ([[SendGrid]]著, [TIME[2017-03-18 02:16:02 +09:00]]))
<https://sendgrid.com/docs/API_Reference/Webhooks/event_deprecated.html>
]FIGCAPTION]

> Batched event POSTs have a content-type header of application/json, and contain exactly one JSON string per line, with each line representing one event. Please note that currently the POST headers define this post as application/json, though it’s not; each line is a valid JSON string, but the overall POST body is not.

]FIG]

[102] 
[[NDJSON]]、
徐々にメジャーになってきてる感ある

[103] [CITE@en[Introducing streaming newline-delimited JSON (NDJSON) with ReactPHP]], [TIME[2019-10-30 19:52:35 +09:00]] <https://clue.engineering/2018/introducing-reactphp-ndjson>

[104] [CITE@en[GitHub - ndjson/ndjson-spec: Specification]], [TIME[2019-10-31 18:10:26 +09:00]] <https://github.com/ndjson/ndjson-spec>

[105] [CITE@en[Invalid mime type application/stream+json '''['''SPR-16742''']''' · Issue #21283 · spring-projects/spring-framework · GitHub]], [TIME[2019-10-31 18:12:28 +09:00]] <https://github.com/spring-projects/spring-framework/issues/21283>

[106] [CITE@en[Relation to JSON Text Sequences standard · Issue #25 · ndjson/ndjson-spec · GitHub]], [TIME[2019-10-31 18:13:18 +09:00]] <https://github.com/ndjson/ndjson-spec/issues/25>

>application/x-json-stream and application/x-jsonlines

[FIG(quote)[
[FIGCAPTION[
[107] [CITE@en[GeoJSONSeq: sequence of GeoJSON features — GDAL documentation]]
([TIME[2021-12-08T15:14:16.000Z]], [TIME[2021-12-09T05:17:47.386Z]])
<https://gdal.org/drivers/vector/geojsonseq.html>
]FIGCAPTION]

> This driver implements read/creation support for features encoded individually as GeoJSON Feature objects, separated by newline (LF) (Newline Delimited JSON)

]FIG]

[110] 
[CITE@en[jsonlines.org and ndjson.org · Issue #22 · wardi/jsonlines · GitHub]], [TIME[2024-03-06T03:42:22.000Z]] <https://github.com/wardi/jsonlines/issues/22>

[111] 
[CITE@en[jsonlines · Issue #35 · ndjson/ndjson-spec · GitHub]], [TIME[2024-03-06T03:44:32.000Z]] <https://github.com/ndjson/ndjson-spec/issues/35>

[112] 
[CITE@en[jsonlines.org · Issue #8 · ndjson/ndjson-spec · GitHub]], [TIME[2024-03-06T03:45:16.000Z]] <https://github.com/ndjson/ndjson-spec/issues/8>

[113] 
[CITE@en[Standard MIME content-type · Issue #19 · wardi/jsonlines · GitHub]], [TIME[2024-03-06T03:47:13.000Z]] <https://github.com/wardi/jsonlines/issues/19>

[115] 
[CITE@en[1603986 - Support newline delimited JSON (NDJSON) with MIME type application/x-ndjson]], [TIME[2024-03-06T03:53:01.000Z]] <https://bugzilla.mozilla.org/show_bug.cgi?id=1603986>

[116] 
[CITE@en[Domain ndjson.org has expired · Issue #24 · ndjson/ndjson.github.io · GitHub]], [TIME[2024-03-06T03:54:03.000Z]] <https://github.com/ndjson/ndjson.github.io/issues/24>

Historical Legal Sensitive Parent: [Help / License]

Edit

Page

Errors

Page name(s)