D3 · all subjects

d3-dsv

30 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

d3-dsv module overview

d3-dsv parses and formats delimiter-separated values (CSV and TSV). It provides functions for parsing strings into objects or rows, formatting objects or rows back to strings, creating custom DSV parsers, and automatically inferring value types.

tsvFormatValue convenience method

tsvFormatValue(value) is equivalent to d3.dsvFormat("\t").formatValue(value).

tsvFormatBody convenience method

tsvFormatBody(rows, columns) is equivalent to d3.dsvFormat("\t").formatBody(rows, columns).

tsvParseRows convenience method

tsvParseRows(string, row) is equivalent to d3.dsvFormat("\t").parseRows(string, row).

dsvFormat constructor signature

dsvFormat(delimiter) constructs a new DSV parser and formatter for the specified delimiter. The delimiter must be a single character (a single 16-bit code unit); ASCII delimiters are fine, but emoji delimiters are not.

dsv.parse field values default to strings

When using dsv.parse without a row conversion function, field values are strings. There is no automatic conversion to numbers, dates, or other types for safety reasons. Use d3.autoType or a custom row conversion function to coerce types.

dsv.parseRows method signature and behavior

dsv.parseRows(string, row) parses a delimiter-separated values string and returns an array of arrays representing parsed rows. Unlike dsv.parse, it treats the header line as a standard row and does not require a header. Each row is an array rather than an object. Rows may have variable length. If a row conversion function is specified, it is invoked for each row with arguments (d, i, columns) where d is the row array, i is the index starting at zero for the first row, and columns is the array of column names.

dsv.format method signature and behavior

dsv.format(rows, columns) formats an array of object rows as delimiter-separated values, returning a string. This is the inverse of dsv.parse. Each row is separated by a newline (\n), and columns are separated by the delimiter. Values containing the delimiter, double-quote ("), or newline are escaped using double-quotes. If columns is not specified, the column header is determined by the union of all properties on all objects in rows (order is nondeterministic). If columns is specified, it is an array of column name strings. All field values are coerced to strings; null or undefined becomes empty string; Date values use ECMAScript date-time string format (YYYY-MM-DD for UTC midnight).

dsv.formatBody method signature

dsv.formatBody(rows, columns) is equivalent to dsv.format but omits the header row. Useful for appending rows to an existing file.

dsv.formatRow method signature

dsv.formatRow(row) formats a single array row of strings as delimiter-separated values, returning a string. Values containing the delimiter, double-quote ("), or newline are escaped using double-quotes.

dsv.formatValue method signature

dsv.formatValue(value) formats a single value or string as a delimiter-separated value, returning a string. A value containing the delimiter, double-quote ("), or newline is escaped using double-quotes.

csvParseRows convenience method

csvParseRows(string, row) is equivalent to d3.dsvFormat(",").parseRows(string, row).

csvFormat convenience method

csvFormat(rows, columns) is equivalent to d3.dsvFormat(",").format(rows, columns).

csvFormatBody convenience method

csvFormatBody(rows, columns) is equivalent to d3.dsvFormat(",").formatBody(rows, columns).

csvFormatRow convenience method

csvFormatRow(row) is equivalent to d3.dsvFormat(",").formatRow(row).

csvFormatValue convenience method

csvFormatValue(value) is equivalent to d3.dsvFormat(",").formatValue(value).

autoType function signature and behavior

autoType(object) infers the types of values on the given object (or array) representing a parsed row and coerces them accordingly, returning the mutated object. Intended to be used as a row accessor function with dsv.parse or dsv.parseRows. Type inference applies to each trimmed value in the following order: if empty then null; if exactly "true" then true; if exactly "false" then false; if exactly "NaN" then NaN; otherwise if coercible to a number then a number; otherwise if a date-only or date-time string in ECMAScript's ISO 8601 subset then a Date; otherwise a string (original untrimmed value).

autoType number coercion with leading zeros

autoType coerces values with leading zeros to numbers; for example "08904" coerces to 8904. Extra characters such as commas or units ("$1.00", "(123)", "1,234", "32px") prevent number coercion, resulting in a string.

autoType date string handling

autoType requires date strings in ECMAScript's subset of ISO 8601 format. For date-only strings like YYYY-MM-DD, the inferred time is midnight UTC. For date-time strings like YYYY-MM-DDTHH:MM without a time zone, it is assumed to be local time.

autoType usage example

d3.csvParse(string, d3.autoType) parses a CSV string and automatically infers and coerces types for numbers and dates. For a CSV with Year, Make, Model, Length columns, numeric columns become numbers and date columns become Date objects.

dsv.parse content security policy requirement

dsv.parse requires unsafe-eval in the script-src directive of a content security policy due to safe use of dynamic code generation for fast parsing. Use dsv.parseRows as an alternative if unsafe-eval cannot be used.

Byte order mark handling in DSV files

DSV files sometimes begin with a byte order mark (BOM), such as when saving from Microsoft Excel in CSV UTF-8 format. Web browsers remove the BOM via the UTF-8 decode algorithm, but Node.js does not. If the BOM is not removed, the first character becomes a zero-width non-breaking space, causing the first column name to begin with this invisible character. Use the strip-bom npm package to remove the BOM before parsing.

dsv.parseRows row conversion example

Example of using dsv.parseRows with a row conversion function: const data = d3.csvParseRows(string, (d, i) => { return { year: new Date(+d[0], 0, 1), make: d[1], model: d[2], length: +d[3] }; });

dsv.format columns parameter example

To control which columns appear in the output and their order, pass a columns array to dsv.format: const string = d3.csvFormat(data, ["year", "make", "model", "length"]);

dsv.formatRows custom transformation example

To format rows with custom transformations, map data to an array of arrays and use dsv.formatRows: const string = d3.csvFormatRows(data.map((d, i) => { return [ d.year.getUTCFullYear(), d.make, d.model, d.length ]; }));

dsv.formatRows with header row example

To include a header row with dsv.formatRows, concatenate the header array with the data rows: const string = d3.csvFormatRows([[ "year", "make", "model", "length" ]].concat(data.map((d, i) => { return [ d.year.getUTCFullYear(), d.make, d.model, d.length ]; })));

Duplicate column names in dsv.parse

If column names are not unique in dsv.parse, only the last value is returned for each column name. To access all values when duplicate column names exist, use dsv.parseRows instead.

Number coercion comparison in row conversion

When converting field values to numbers in a row conversion function, using + or Number() is typically faster than parseInt or parseFloat, though more restrictive. For example, "30px" with + returns NaN, while parseInt and parseFloat return 30.

tsvFormatRow convenience method

tsvFormatRow(row) is equivalent to d3.dsvFormat("\t").formatRow(row).

Custom delimiter example with dsvFormat

To parse or format values with a custom delimiter like pipe, use dsvFormat: const psv = d3.dsvFormat("|"); const data = psv.parse("foo|bar\n1|2");

Give your agent this brain