spec/introduction
Introduction
Text Tables is a line-oriented text format for tables. A file holds one or more tables; each table's rows are ordinary CSV/TSV, and system lines — lines that start with the file's prefix — describe the file and its tables: dialect, column schema, aggregates, metadata, SQL, notes and charts.
A descriptive format (normative)
Text Tables describes data; it does not define or alter it.
- A table's data rows MUST be valid CSV/TSV in the table's declared dialect. System lines only annotate them.
- A parser MUST NOT rewrite values from metadata.
default=is a schema/DDL attribute and is not substituted on read; null markers read as null. - Consistency signals (
rows=against the actual row count,default=against nulls,agg-*against the data) are advisory unless a rule says otherwise: a mismatch is a warning and MUST NOT block reading. The exceptions are listed in errors.md (for example,tables-mismatchis always an error). - A file asserts its own data. Aggregates, row counts and other derived facts describe exactly the rows the file declares — see aggregates.md.
- Rewriting data (materializing a computed column, slicing, filtering) is an explicit tool operation, never a side effect of reading.
Forms
| Form | Extension | MIME type |
|---|---|---|
| Text | .txtab |
text/txtab |
| Archive | .txtab.zip |
application/txtab+zip |
A table's data can live in three places, declared per table (see tables.md and archive.md):
- inline — rows in the same file, after the table's block;
- external rows — an ordinary CSV/TSV file named by
reference=; - external columns — a directory of
.colfiles, one per column (storage=columns), normally inside a.txtab.zip.
The vocabulary is the same everywhere; only the packaging differs. There is no JSON form.
Conformance
- A parser reads Text Tables files and reports the errors and warnings defined in errors.md. A conforming parser MUST reject a file on any FAIL-level error and MUST NOT reject it on a WARN-level one.
- A writer produces Text Tables files. A conforming writer MUST produce files that a conforming parser reads without FAIL-level errors, and SHOULD produce files without warnings.
- A validator (
txtab verifyand similar) is a parser that additionally escalates the warnings marked "→ FAIL" in errors.md.
Text Tables is not ExCSV. A conforming parser MUST NOT treat an ExCSV file (#!excsv …) as Text Tables; such a file is plain CSV/TSV from its point of view (file-structure.md).
Terminology
The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119.
| Term | Meaning |
|---|---|
| prefix (P) | The characters before the first space of the first line; starts every system line |
| system line | A line that starts with P followed by a space, or that is exactly P |
| first line | Line 1: P texttables=<version> … |
| keyword | The first token after P on a system line, e.g. table, column, agg-sum |
| comment | A system line whose first token is -- |
| file block | System lines between the first line and the first P table |
| table block | System lines from a P table line up to the table's data (or the next P table) |
| data section | A table's inline rows: the header row (when header=1) followed by rows data rows |
| record | One line of a data section |
Prior work
Text Tables is a new take on ExCSV 0.6, which built on Astropy ECSV, InfluxDB Annotated CSV, MetaCSV, CSVW, CSVY and Frictionless Table Schema. It does not read ExCSV files.