Skip to main content

StreamWright documentation

Fetch, shape and write data in YAML

Learn the three ideas below, pick what you want to build, and you can read any page in these docs.

What do you want to do?

Three ideas to remember

  1. 1

    Fetch → Shape → Write

    Everything reads records, shapes them with SQL, then writes them. In a source: requests → transform → export. In a job: read steps → sql steps → outputs.

  2. 2

    connection · operation · policy

    Every call to the outside world says where (the connection), what (the operation) and how (retries, rate limits, timeouts).

  3. 3

    Same YAML, many runtimes

    Run it from the CLI, behind a REST service, on Spark, or per user and network with Dagster.