Resumable load walkthrough

This page shows a complete Load-mode example that uses the automatic resumable snapshot-copy path. You will create one eligible source table, start a Load stream, stop it after at least one chunk completes, resume the same stream config, and verify that DBConvert Streams continues from the next incomplete primary-key range instead of restarting the whole table.

For the routing rules and state semantics, read Resumable Load first.

What you will build

Source table public.orders_resumable_demo
Target schema rsnap_demo
One Load stream using schemaPolicy=create_missing_only and writeMode=upsert
One stop/start cycle that resumes from durable loadProgress state

Prerequisites

PostgreSQL source and PostgreSQL target connections already exist in DBConvert Streams
API base URL, API key, and install ID are available
No custom SQL on the selected table
The target schema is empty or dedicated to this walkthrough
The source table uses a single numeric primary key

This walkthrough uses source IDs 1..250000. In the current implementation, resumable Load uses internal pk_range chunking with a width of 50000, so this table plans 5 chunks.

Set shell variables

export HOST="http://127.0.0.1:8020/api/v1"
export API_KEY="YOUR_API_KEY"
export INSTALL_ID="YOUR_INSTALL_ID"

export SOURCE_CONNECTION_ID="conn_pg_source"
export TARGET_CONNECTION_ID="conn_pg_target"

export SOURCE_DATABASE="postgres"
export SOURCE_SCHEMA="public"
export TARGET_DATABASE="postgres"
export TARGET_SCHEMA="rsnap_demo"

Replace the connection IDs and database names with values from your environment.

1. Create the source table and clean target state

On the PostgreSQL source:

DROP TABLE IF EXISTS public.orders_resumable_demo;

CREATE TABLE public.orders_resumable_demo (
    id BIGINT PRIMARY KEY,
    customer_id BIGINT NOT NULL,
    amount NUMERIC(10,2) NOT NULL,
    status TEXT NOT NULL,
    created_at TIMESTAMP NOT NULL
);

INSERT INTO public.orders_resumable_demo (id, customer_id, amount, status, created_at)
SELECT i,
       ((i - 1) % 10000) + 1,
       ROUND((10 + (i % 5000) * 0.25)::numeric, 2),
       CASE
           WHEN i % 5 = 0 THEN 'paid'
           WHEN i % 5 = 1 THEN 'pending'
           WHEN i % 5 = 2 THEN 'packed'
           WHEN i % 5 = 3 THEN 'shipped'
           ELSE 'delivered'
       END,
       TIMESTAMP '2026-01-01 00:00:00' + ((i - 1) * INTERVAL '1 second')
FROM generate_series(1, 250000) AS i;

SELECT COUNT(*) FROM public.orders_resumable_demo;
-- 250000

On the PostgreSQL target:

CREATE SCHEMA IF NOT EXISTS rsnap_demo;
DROP TABLE IF EXISTS rsnap_demo.orders_resumable_demo;

The target table must not contain old rows from a previous run. This walkthrough lets DBConvert Streams create the missing table on the first start.

2. Create the Load stream config

Create a stream config that selects only the demo table and uses an eligible target policy:

curl -s -X POST "$HOST/stream-configs" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID" \
  -d "{
    \"name\": \"resumable-load-demo\",
    \"mode\": \"load\",
    \"source\": {
      \"connections\": [
        {
          \"connectionId\": \"$SOURCE_CONNECTION_ID\",
          \"database\": \"$SOURCE_DATABASE\",
          \"schema\": \"$SOURCE_SCHEMA\",
          \"tables\": [
            { \"name\": \"orders_resumable_demo\" }
          ]
        }
      ]
    },
    \"target\": {
      \"id\": \"$TARGET_CONNECTION_ID\",
      \"spec\": {
        \"db\": {
          \"database\": \"$TARGET_DATABASE\",
          \"schema\": \"$TARGET_SCHEMA\",
          \"schemaPolicy\": \"create_missing_only\",
          \"writeMode\": \"upsert\"
        }
      }
    },
    \"reportingInterval\": 1
  }"

Save the returned stream-config id as CONFIG_ID.

3. Start the first run and wait for one completed chunk

Start the stream:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/start" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Save the returned stream id as STREAM_ID.

Check the stats endpoint until loadProgress.chunksCompleted is at least 1 while the top-level stream status is still RUNNING:

curl -s -X GET "$HOST/streams/$STREAM_ID/stats" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Expected shape during the first run:

{
  "status": "RUNNING",
  "loadProgress": {
    "status": "copying",
    "strategy": "pk_range",
    "mode": "load",
    "tablesPlanned": 1,
    "tablesCompleted": 0,
    "chunksPlanned": 5,
    "chunksCompleted": 1,
    "currentTable": "public.orders_resumable_demo",
    "currentChunk": 2
  }
}

Exact timestamps and the in-flight currentChunk can vary, but chunksCompleted must advance before you stop the run.

4. Stop the first run

Stop the active stream:

curl -s -X POST "$HOST/streams/$STREAM_ID/stop" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Poll the same stats endpoint until the stream reaches STOPPED. The stopped run should still show at least one completed chunk:

{
  "status": "STOPPED",
  "loadProgress": {
    "status": "copying",
    "strategy": "pk_range",
    "mode": "load",
    "chunksCompleted": 1
  }
}

The copy is incomplete, but the completed chunk boundary is already durable.

In the desktop app, expand the config in the Streams rail and open this stopped run. Its Monitor tab shows Load Progress, Rows Saved, and completed Chunks. When the saved state remains compatible, the config action is Resume Load. If the run reports that saved state can no longer be resumed, use Reset before starting a fresh load instead.

5. Resume the same config

Start the same stream config, not a new one. In the desktop app, select Resume Load; via API, call the same start endpoint:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/start" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Save the new stream id as RESUMED_STREAM_ID.

Check the stats endpoint again:

curl -s -X GET "$HOST/streams/$RESUMED_STREAM_ID/stats" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Expected terminal result:

{
  "status": "FINISHED",
  "loadProgress": {
    "status": "complete",
    "strategy": "pk_range",
    "mode": "load",
    "tablesPlanned": 1,
    "tablesCompleted": 1,
    "chunksPlanned": 5,
    "chunksCompleted": 5
  }
}

This is the key check: the resumed run finishes the planned chunks for the same config instead of rebuilding the target from chunk 0.

6. Verify the target rows

On the PostgreSQL target:

SELECT COUNT(*) AS rows,
       COUNT(DISTINCT id) AS distinct_ids,
       MIN(id) AS min_id,
       MAX(id) AS max_id
FROM rsnap_demo.orders_resumable_demo;

-- rows=250000
-- distinct_ids=250000
-- min_id=1
-- max_id=250000

If you also compare with the source row count, both sides should report 250000.

What this walkthrough proves

The table is eligible for automatic resumable Load routing
loadProgress state is persisted by stream config ID
Stopping the run after a completed chunk does not force a full re-copy
The resumed run converges on the same final target row set without duplicate primary keys

Troubleshooting

`loadProgress` is missing from `/streams/{id}/stats`

The table is not on the resumable path. Re-check the active config:

source and target are both databases
target writeMode is upsert
target schemaPolicy is create_missing_only or validate_existing
selected table uses a single numeric primary key
no custom SQL query is attached to the table

Restart returns `reset_required`

The saved snapshot-copy state no longer matches the target or selected table shape. Reset the config state explicitly:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/reset" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID" \
  -H "X-Confirm-Reset: true"

For Load mode, reset clears the durable recovery keys and truncates the target rows loaded by this config, so the next start can run as a fresh first load.

The run finishes before you can stop it

Increase the source row count and repeat the walkthrough on a clean target schema. The resume check is easiest when the first run stays active long enough for chunksCompleted to move past 0.