Outputs
An output is where a source run writes its exports. You pick it at run time with --output; it is not part of
the source. The same source can write files on your laptop and load a warehouse in production, with no change.
In short
The source says what to write (its exports). --output says where.
streamwright run my_source --output parquet:out
This writes one Parquet file per export into out/, plus a state.json with the run's bookmarks.
Pick an output
--output value | Result |
|---|---|
singer (default) | Singer SCHEMA / RECORD / STATE messages on stdout |
jsonl:DIR · csv:DIR · tsv:DIR · parquet:DIR | One file per export plus state.json, written atomically |
duckdb:PATH[:SCHEMA] · ducklake:CATALOG[:SCHEMA] | Load into a DuckDB file or a DuckLake catalog |
dlt:DESTINATION[:DATASET] | Load into any dlt destination, such as dlt:duckdb or dlt:bigquery:marketing |
Logs always go to stderr, so singer output on stdout can be piped to another tool.
Files
| Format | Each export's file |
|---|---|
jsonl | One JSON object per line |
csv / tsv | A header row, then one row per record |
parquet | One row per record, with the export step's column types, zstd-compressed |
Files are named <export>.<date>.<time>.<unique>.<ext> unless you name them. --file-name sets the path of each
export's file inside DIR:
streamwright run my_source --output parquet:out --file-name "{{ export }}/{{ today }}.parquet"
- The template can use
{{ export }},{{ source }},{{ today }},{{ timestamp }},{{ format }}and{{ config.NAME }}. - Subfolders are created. Every export needs its own path. A file with the same name is replaced.
--file-namereplaces anyfile_nameset in the YAML (see Export).--file-nameworks with file outputs only; with another output it is an error.
Warehouses
| Output | Use it to... |
|---|---|
duckdb:PATH[:SCHEMA] | Load straight into a local DuckDB file |
ducklake:CATALOG[:SCHEMA] | Load into a DuckLake catalog: a file, or postgres:DSN for a Postgres catalog. Put data files on S3 with STREAMWRIGHT_DUCKLAKE_DATA_PATH. |
dlt:DESTINATION[:DATASET] | Use dlt's destinations (BigQuery, Snowflake, Postgres and more) and its schema evolution. dlt also keeps the run's state. |
note
These outputs are for sources. A job (kind: DuckDBJob, kind: SparkJob) declares its writes in the manifest's
outputs. See DuckDB jobs.