Quickstart

This page copies one folder into DuckDB with no configuration, then sets up a project that loads a Postgres database into DuckDB and keeps it up to date. It needs SQLLoad (Install) and, for the second part, Docker.

Copy one table

sqlload copy loads one table once and needs no configuration file. Here, a folder of Parquet files, orders/, is loaded into a DuckDB table:

$ sqlload copy orders/ duckdb://shop.duckdb#raw.orders
✓ Plan complete  1 selected, 1 object to create, 1 to load   (sqlload plan shows the tree)

created  raw.orders  9 columns
loaded  raw.orders      3 rows 0.0s

✓ Completed  1 loaded, 3 rows, 0.1s
├── slowest      raw.orders 0.0s
└── peak memory  72.3 MiB

The plan’s closing line comes first: one table to create and load. DuckDB reads the Parquet files itself, so the rows never pass through SQLLoad. The table’s 9 columns are the files’ 4 and the _sqlload_* columns, which record which file and row each row came from, and when it loaded.

Add --print-toml to print the same copy as a sqlload.toml project instead of running it. See sqlload copy.

Load a database with a project

A project is a sqlload.toml file that names connections, targets and sources. plan, apply and load then work on every table in it.

1. Start Postgres with some data

Save this as seed.sql:

CREATE TABLE customers (
    customer_id bigint PRIMARY KEY,
    email text NOT NULL,
    country text,
    created_at timestamptz NOT NULL
);
CREATE TABLE orders (
    order_id bigint PRIMARY KEY,
    customer_id bigint NOT NULL REFERENCES customers,
    status text NOT NULL,
    total numeric(12,2) NOT NULL,
    updated_at timestamptz NOT NULL
);
CREATE INDEX ON orders (updated_at);

INSERT INTO customers
SELECT i, 'customer' || i || '@example.com', (ARRAY['GB', 'FR', 'DE', 'US'])[1 + i % 4],
       timestamptz '2026-01-01' + i * interval '1 day'
FROM generate_series(1, 50) AS i;

INSERT INTO orders
SELECT i, 1 + i % 50, (ARRAY['placed', 'paid', 'shipped'])[1 + i % 3],
       round(10 + i * 1.37, 2), timestamptz '2026-09-01' + i * interval '1 hour'
FROM generate_series(1, 1000) AS i;

Then start Postgres and load it:

docker run -d --name shop-db -e POSTGRES_PASSWORD=secret -p 5432:5432 postgres:17
docker exec -i shop-db psql -q -U postgres < seed.sql

2. Write sqlload.toml

default_target = "local"

[connections.app]
kind = "postgres"
host = "localhost"
user = "postgres"
password = "${ENV:PGPASSWORD}"
database = "postgres"

[connections.warehouse]
kind = "duckdb"
database = "shop.duckdb"

[targets.local]
connection = "warehouse"
loader_schema = "raw"

[sources.app]
kind = "postgres"
connection = "app"
description = "The shop's application database."
include = ["public.*"]
write_strategy = "merge"
unique_key = "{primary_key}"

[sources.app.tables."public.orders"]
description = "Orders, one row per order."
cursor_column = "updated_at"

[sources.app.tables."public.customers"]
description = "Customers, one row per customer."

• [connections] say how to reach each system. ${ENV:PGPASSWORD} reads the password from the environment, so it stays out of the file.
• [targets.local] loads into the DuckDB file, in schema raw.
• [sources.app] syncs every table in public. Every table needs a write_strategy; here the source sets merge for all of them, and unique_key = "{primary_key}" upserts on each table’s primary key.
• orders is incremental: each run continues from the highest updated_at already loaded.
• Each table SQLLoad writes needs a description, which becomes the table’s comment.

3. Check, plan, apply and load

export PGPASSWORD=secret

sqlload debug validates the file and tests each connection:

$ sqlload debug
ok    config  2 connections, 1 targets, 1 sources
ok    target local  DuckDB v2.0.0
ok    source app  postgres@postgres, PostgreSQL 17.11 (Debian 17.11-1.pgdg13+2)

sqlload plan shows what apply and load would do, and changes nothing:

$ sqlload plan
Plan ready  2 selected, 2 objects to create, 2 to load
to  DuckDB local

Objects (2)                                         sqlload apply
├── raw.app__customers                              create table
│   ├── customer_id int            -> BIGINT
│   ├── email       text           -> VARCHAR
│   ├── country     text           -> VARCHAR
│   ├── created_at  timestamp_tz   -> TIMESTAMP WITH TIME ZONE  (stored as a UTC instant; the offset is not kept)
│   └── + _sqlload_load_id, _loaded_at
└── raw.app__orders                                 create table
    ├── order_id    int             -> BIGINT
    ├── customer_id int             -> BIGINT
    ├── status      text            -> VARCHAR
    ├── total       decimal(12,2)   -> DECIMAL(12,2)
    ├── updated_at  timestamp_tz    -> TIMESTAMP WITH TIME ZONE  (stored as a UTC instant; the offset is not kept)
    └── + _sqlload_load_id, _loaded_at

Loads (2)                                           sqlload load
├── raw.app__customers                              merge (unique_key: customer_id ← {primary_key})
│   ├── source  postgres  app  public.customers
│   └── route   read as Arrow -> DuckDB takes Arrow
└── raw.app__orders                                 merge (unique_key: order_id ← {primary_key})
    ├── source  postgres  app  public.orders
    ├── route   read as Arrow -> DuckDB takes Arrow
    └── reads   updated_at: first run, everything

✓ Plan complete  2 selected, 2 objects to create, 2 to load

Tables are named {source}__{table} unless you set table_name. ← {primary_key} shows that each unique_key is the table’s primary key. reads shows where an incremental table’s read starts.

sqlload apply creates the tables, and sqlload load fills them. apply prints the plan first, here cut to the last lines; load prints the plan’s closing line, then a line per table:

$ sqlload apply
…
created  raw.app__customers  6 columns
created  raw.app__orders     7 columns

✓ Completed  0.1s
$ sqlload load
✓ Plan complete  2 selected, 2 to load   (sqlload plan shows the tree)

loaded  raw.app__customers     50 rows 0.0s
loaded  raw.app__orders     1,000 rows 0.0s

✓ Completed  2 loaded, 1,050 rows, 0.1s
├── slowest      raw.app__orders 0.0s
└── peak memory  85.4 MiB

4. Load again

Change one order and add another:

docker exec -i shop-db psql -q -U postgres -c "UPDATE orders SET status = 'refunded', updated_at = now() WHERE order_id = 42; INSERT INTO orders VALUES (1001, 7, 'placed', 99.90, now());"

Plan only app__orders. A plan of one table shows every line. reads shows where the read starts: the destination’s maximum updated_at, read from the table itself. Because an index leads with updated_at, orders is read in cursor order and can commit along the way, as commits shows.

$ sqlload plan app__orders
Plan ready  1 selected, 1 to load

Loads (1)                                           sqlload load
└── raw.app__orders                                 merge (unique_key: order_id ← {primary_key})
    ├── source  postgres  app  public.orders
    ├── route   read as Arrow -> DuckDB takes Arrow
    ├── key     order_id, the primary key the source reports (unique_key = "{primary_key}", sqlload.toml:24)
    ├── reads   updated_at >= 2026-10-12 16:00:00.000+00 (warehouse maximum)
    ├── columns 5
    └── commits about every 1 GB, in updated_at order (an index leads with it, NULLs first), between distinct values

✓ Plan complete  1 selected, 1 to load
$ sqlload load app__orders
✓ Plan complete  1 selected, 1 to load   (sqlload plan shows the tree)

loaded  raw.app__orders      3 rows 0.0s

✓ Completed  1 loaded, 3 rows, 0.1s
├── slowest      raw.app__orders 0.0s
└── peak memory  80.8 MiB

Three rows: the two changed ones, and the row at the previous maximum, which the cursor includes. merge updates order 42 in place. SQLLoad keeps no state file for this. Delete the DuckDB file, and the next run loads everything again.

Next

• Projects: how connections, targets and sources fit together.
• Load from Python and HTTP APIs: tables defined in Python, including HTTP APIs.
• Snowflake load methods: how rows and files get into Snowflake.
• sqlload.toml reference: every key.