Resumable load walkthrough

This page shows a complete Load-mode example that uses the automatic resumable snapshot-copy path. You will create one eligible source table, start a Load stream, stop it after at least one chunk completes, start the same stream config again, and verify that DBConvert Streams resumes from the next incomplete primary-key range instead of restarting the whole table.

For the routing rules and state semantics, read Resumable Load first.

What you will build

• Source table public.orders_resumable_demo
• Target schema rsnap_demo
• One Load stream using schemaPolicy=create_missing_only and writeMode=upsert
• One stop/start cycle that resumes from durable loadProgress state

Prerequisites

• PostgreSQL source and PostgreSQL target connections already exist in DBConvert Streams
• API base URL, API key, and install ID are available
• No custom SQL on the selected table
• The target schema is empty or dedicated to this walkthrough
• The source table uses a single numeric primary key

This walkthrough uses source IDs 1..250000. In the current implementation, resumable Load uses internal pk_range chunking with a width of 50000, so this table plans 5 chunks.

Set shell variables

export HOST="http://127.0.0.1:8020/api/v1"
export API_KEY="YOUR_API_KEY"
export INSTALL_ID="YOUR_INSTALL_ID"

export SOURCE_CONNECTION_ID="conn_pg_source"
export TARGET_CONNECTION_ID="conn_pg_target"

export SOURCE_DATABASE="postgres"
export SOURCE_SCHEMA="public"
export TARGET_DATABASE="postgres"
export TARGET_SCHEMA="rsnap_demo"

Replace the connection IDs and database names with values from your environment.

1. Create the source table and clean target state

On the PostgreSQL source:

DROP TABLE IF EXISTS public.orders_resumable_demo;

CREATE TABLE public.orders_resumable_demo (
    id BIGINT PRIMARY KEY,
    customer_id BIGINT NOT NULL,
    amount NUMERIC(10,2) NOT NULL,
    status TEXT NOT NULL,
    created_at TIMESTAMP NOT NULL
);

INSERT INTO public.orders_resumable_demo (id, customer_id, amount, status, created_at)
SELECT i,
       ((i - 1) % 10000) + 1,
       ROUND((10 + (i % 5000) * 0.25)::numeric, 2),
       CASE
           WHEN i % 5 = 0 THEN 'paid'
           WHEN i % 5 = 1 THEN 'pending'
           WHEN i % 5 = 2 THEN 'packed'
           WHEN i % 5 = 3 THEN 'shipped'
           ELSE 'delivered'
       END,
       TIMESTAMP '2026-01-01 00:00:00' + ((i - 1) * INTERVAL '1 second')
FROM generate_series(1, 250000) AS i;

SELECT COUNT(*) FROM public.orders_resumable_demo;
-- 250000

On the PostgreSQL target:

CREATE SCHEMA IF NOT EXISTS rsnap_demo;
DROP TABLE IF EXISTS rsnap_demo.orders_resumable_demo;

The target table must not contain old rows from a previous run. This walkthrough lets DBConvert Streams create the missing table on the first start.

2. Create the Load stream config

Create a stream config that selects only the demo table and uses an eligible target policy:

curl -s -X POST "$HOST/stream-configs" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID" \
  -d "{
    \"name\": \"resumable-load-demo\",
    \"mode\": \"load\",
    \"source\": {
      \"connections\": [
        {
          \"connectionId\": \"$SOURCE_CONNECTION_ID\",
          \"database\": \"$SOURCE_DATABASE\",
          \"schema\": \"$SOURCE_SCHEMA\",
          \"tables\": [
            { \"name\": \"orders_resumable_demo\" }
          ]
        }
      ],
      \"options\": {
        \"dataBundleSize\": 5000
      }
    },
    \"target\": {
      \"id\": \"$TARGET_CONNECTION_ID\",
      \"spec\": {
        \"db\": {
          \"database\": \"$TARGET_DATABASE\",
          \"schema\": \"$TARGET_SCHEMA\",
          \"schemaPolicy\": \"create_missing_only\",
          \"writeMode\": \"upsert\"
        }
      }
    },
    \"reportingInterval\": 1
  }"

Save the returned stream-config id as CONFIG_ID.

3. Start the first run and wait for one completed chunk

Start the stream:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/start" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Save the returned stream id as STREAM_ID.

Check the stats endpoint until loadProgress.chunksCompleted is at least 1 while the top-level stream status is still RUNNING:

curl -s -X GET "$HOST/streams/$STREAM_ID/stats" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Expected shape during the first run:

{
  "status": "RUNNING",
  "loadProgress": {
    "status": "copying",
    "strategy": "pk_range",
    "mode": "load",
    "tablesPlanned": 1,
    "tablesCompleted": 0,
    "chunksPlanned": 5,
    "chunksCompleted": 1,
    "currentTable": "public.orders_resumable_demo",
    "currentChunk": 2
  }
}

Exact timestamps and the in-flight currentChunk can vary, but chunksCompleted must advance before you stop the run.

4. Stop the first run

Stop the active stream:

curl -s -X POST "$HOST/streams/$STREAM_ID/stop" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Poll the same stats endpoint until the stream reaches STOPPED. The stopped run should still show at least one completed chunk:

{
  "status": "STOPPED",
  "loadProgress": {
    "status": "copying",
    "strategy": "pk_range",
    "mode": "load",
    "chunksCompleted": 1
  }
}

The copy is incomplete, but the completed chunk boundary is already durable.

5. Start the same config again

Start the same stream config, not a new one:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/start" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Save the new stream id as RESUMED_STREAM_ID.

Check the stats endpoint again:

curl -s -X GET "$HOST/streams/$RESUMED_STREAM_ID/stats" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID"

Expected terminal result:

{
  "status": "FINISHED",
  "loadProgress": {
    "status": "complete",
    "strategy": "pk_range",
    "mode": "load",
    "tablesPlanned": 1,
    "tablesCompleted": 1,
    "chunksPlanned": 5,
    "chunksCompleted": 5
  }
}

This is the key check: the resumed run finishes the planned chunks for the same config instead of rebuilding the target from chunk 0.

6. Verify the target rows

On the PostgreSQL target:

SELECT COUNT(*) AS rows,
       COUNT(DISTINCT id) AS distinct_ids,
       MIN(id) AS min_id,
       MAX(id) AS max_id
FROM rsnap_demo.orders_resumable_demo;

-- rows=250000
-- distinct_ids=250000
-- min_id=1
-- max_id=250000

If you also compare with the source row count, both sides should report 250000.

What this walkthrough proves

• The table is eligible for automatic resumable Load routing
• loadProgress state is persisted by stream config ID
• Stopping the run after a completed chunk does not force a full re-copy
• The resumed run converges on the same final target row set without duplicate primary keys

Troubleshooting

loadProgress is missing from /streams/{id}/stats

The table is not on the resumable path. Re-check the active config:

• source and target are both databases
• target writeMode is upsert
• target schemaPolicy is create_missing_only or validate_existing
• selected table uses a single numeric primary key
• no custom SQL query is attached to the table

Restart returns reset_required

The saved snapshot-copy state no longer matches the target or selected table shape. Reset the config state explicitly:

curl -s -X POST "$HOST/stream-configs/$CONFIG_ID/reset" \
  -H "X-API-Key: $API_KEY" \
  -H "X-Install-ID: $INSTALL_ID" \
  -H "X-Confirm-Reset: true"

For Load mode, reset clears the durable recovery keys and truncates the target rows loaded by this config, so the next start can run as a fresh first load.

The run finishes before you can stop it

Increase the source row count and repeat the walkthrough on a clean target schema. The resume check is easiest when the first run stays active long enough for chunksCompleted to move past 0.

Related docs

• Resumable Load
• Load Mode
• Target Data Policies
• Stream Configuration Reference