Resumable Load

Resumable Load breaks an eligible table load into primary-key ranges and persists the boundary of each completed chunk. If the stream is interrupted mid-copy, the next start continues from the next incomplete chunk instead of re-reading rows that already made it to the target.

For a copy-paste lab that starts a Load stream, stops it after one completed chunk, and verifies resume on the next start, see Resumable Load Walkthrough.

Under the hood, DBConvert Streams implements Resumable Load as resumable chunked snapshot copy. The same mechanism backs both supported load paths:

• Load mode — the whole run is one bounded load.
• Initial Load + CDC for MySQL sources — the bootstrap copy that runs before CDC apply takes over. See Initial Load + CDC for the surrounding lifecycle.

DBConvert Streams chooses this path automatically per table. There is no public enable/disable flag.

When the chunked path is used

Tables are routed to the resumable chunked path when all of the following are true:

• the source is MySQL or PostgreSQL
• the target is a MySQL or PostgreSQL database
• the target write mode is upsert
• the schema policy is create_missing_only or validate_existing
• the table has a single-column numeric primary key

Tables outside those rules stay on the plain snapshot path in the same run.

Supported engines and modes

PostgreSQL CDC bootstrap still uses the non-resumable path. Its exported snapshot name is only valid while the replication-slot connection stays open, so a crash-resume cannot replay into the same source snapshot yet.

What automatic routing means

• eligible tables use chunked resumable load with durable progress state
• ineligible tables continue on the plain snapshot path in the same run
• the saved stream configuration does not expose a separate resumable-snapshot option

This lets one run mix chunked and plain tables without changing the public API.

Chunk strategy

The chunking strategy is pk_range:

1. Preflight reads MIN(pk) and MAX(pk) per selected table.
2. The range is split into inclusive numeric chunks sized by an internal default.
3. Each chunk runs as SELECT … WHERE pk BETWEEN <lower> AND <upper> ORDER BY pk.
4. The chunk boundary is written to durable state after the target acknowledges the writes.

Chunks are processed sequentially per table. Row counts, bytes, and elapsed time keep flowing through the existing source and target table stats — the new state tracks only the copy boundary, not a second progress counter.

Recovery on restart

On the next start, preflight reconciles the current config against the stored state:

The matching test is strict: if the selected table set, primary-key metadata, or source engine changed, the run fails instead of quietly re-copying into a target that may already contain rows.

Reset

When preflight returns reset_required, or when you want to rerun a failed load cleanly, clear the stored state with the reset endpoint:

curl -X POST \
  -H "X-API-Key: $API_KEY" \
  -H "X-Confirm-Reset: true" \
  "$HOST/api/v1/stream-configs/$CONFIG_ID/reset"

Reset clears saved state and prepares a clean new run in one atomic sequence:

1. Truncates target rows already loaded by this stream config — so the next run starts into an empty target.
2. Purges all durable recovery keys tied to the config: resumable load state, CDC bootstrap state, and CDC apply checkpoint.

The X-Confirm-Reset: true header is required to prevent accidental destructive resets. The stream must be stopped before reset; the API returns an error if an active run is detected.

For CDC streams with initialSnapshot on a PostgreSQL source, reset also drops the inactive replication slot.

Monitor visibility

In the Monitor tab, the stream header shows a load-behavior badge for mode=load configs:

The badge is a quick operator hint. The actual routing criteria are the ones listed in When the chunked path is used above.

When the badge shows Resume after Stop, a stopped run shows a Resume Load action in the monitor header. Click it to continue from the last saved chunk.

The GET /streams/{id}/stats response includes a loadProgress summary while resumable copy is active:

{
  "loadProgress": {
    "status": "copying",
    "strategy": "pk_range",
    "mode": "load",
    "sourceEngine": "mysql",
    "targetEngine": "postgresql",
    "tablesPlanned": 2,
    "tablesCompleted": 0,
    "chunksPlanned": 20,
    "chunksCompleted": 7,
    "currentTable": "public.orders",
    "currentChunk": 8,
    "updatedAt": "2026-04-19T09:12:03Z",
    "ageSeconds": 4
  }
}

Row-level progress remains on the existing per-table stats; loadProgress is about where the copy is in its planned boundary.

Stored state

State lives in the embedded NATS JetStream KV bucket streams:

• <configID>.loadprogress — chunked copy cursor (version, table list, per-table min/max/next-lower, current chunk, last error).
• <configID>.cdc.bootstrap — CDC bootstrap phase and handoff position when initial snapshot is enabled.
• <configID>.cdc.checkpoint — CDC apply checkpoint after bootstrap hands off.

Keep the stream config to keep the state. Deleting the config removes all three keys automatically.

Guarantees and boundaries

The current release provides:

• crash-safe resume from the last completed chunk for eligible chunked tables
• strict preflight instead of silent fallback when durable state and current config disagree
• a reset endpoint for the reset_required path

It does not yet provide:

• composite primary-key chunking
• non-numeric key strategies
• chunked resume for PostgreSQL CDC bootstrap
• file or S3 target resume
• distributed chunk workers

Related docs

• Initial Load + CDC
• CDC Reliability and Resume
• Resumable Load Walkthrough
• Target Data Policies
• Stream Configuration Reference