jsonl

jsonl is a module that parses and stringifies JSON Lines files.

jsonl/parser.js

This module defines the parser function: parser([reviver]). It takes an optional reviver function as an argument and returns a function chain that parses JSON Lines files.

As of 3.3.0 reviver can be an object with two properties: reviver and ignoreErrors. The first one is the function defined above and the second one is a boolean. If ignoreErrors is truthy then all JSON parsing errors are ignored silently. It defaults to false.

The returned function is constructed with gen().

Examples

// const chain = require('stream-chain');
// const parser = require('stream-chain/jsonl/parser.js');

import chain from 'stream-chain';
import parser from 'stream-chain/jsonl/parser.js';

import fs from 'node:fs';
import zlib from 'node:zlib';

chain([
  fs.createReadStream(fileName),
  zlib.createGunzip(),
  parser(),
  x => console.log(x) // prints objects
]);

jsonl/parserStream.js

This module defines the parser as a stream: parserStream([options]). It takes an optional options object as an argument and returns a stream that parses JSON Lines files.

options is an object used by asStream(). It defaults writableObjectMode to false and readableObjectMode to true. Additionally, it defines an optional reviver function (see parser() above).

As of 3.3.0 it defines an optional property ignoreErrors. If it is truthy then all JSON parsing errors are ignored silently. It defaults to false.

Effectively, it is parser() wrapped with asStream() with the proper default for stream object modes.

Examples

// const chain = require('stream-chain');
// const parserStream = require('stream-chain/jsonl/parserStream.js');

import chain from 'stream-chain';
import parserStream from 'stream-chain/jsonl/parserStream.js';

import fs from 'node:fs';
import zlib from 'node:zlib';

chain([
  fs.createReadStream(fileName),
  zlib.createGunzip(),
  parserStream(),
  x => console.log(x) // prints objects
]);

jsonl/stringerStream.js

This module defines the stringer as a stream: stringerStream([options]). It takes an optional options object as an argument and returns a stream that stringifies JSON Lines files.

options is an object used by Duplex. It defaults writableObjectMode to true and readableObjectMode to false. Additionally, it defines the following optional properties to customize the output:

• reviver — a function. It is used as a second argument to JSON.stringify().
• space — a number or a string. It is used as a third argument to JSON.stringify().
• prefix — a string. It is used as a text to be prepended to the output. Defaults to an empty string.
• suffix — a string. It is used as a text to be appended to the output. Defaults to an empty string.
• separator — a string. It is used as a separator between items in the output. Defaults to a newline.
• emptyValue — a string. It is used as a text to be used when no values were present but the stream was closed. Defaults to prefix + suffix.

Warning: specifying prefix, suffix, separator, or emptyValue may result in an output that is not valid JSON Lines.

Examples

A simple JSONL output:

// const chain = require('stream-chain');
// const parser = require('stream-chain/jsonl/parser.js');
// const stringerStream = require('stream-chain/jsonl/stringerStream.js');

import chain from 'stream-chain';
import parser from 'stream-chain/jsonl/parser.js';
import stringerStream from 'stream-chain/jsonl/stringerStream.js';

import fs from 'node:fs';
import zlib from 'node:zlib';

chain([
  fs.createReadStream(fileName),
  zlib.createGunzip(),
  parser(),
  x => [x], // wrap each object in an array
  stringerStream() // produces a valid JSON Lines file
]);

Return an array of objects:

const asArray = stringerStream({
  prefix: '[',
  suffix: ']',
  separator: ','
});

// for a stream of numbers: 1, 2, 3
// it produces this string:
// [1,2,3]