Initial Load + CDC

Use Initial Load + CDC when a source table already has rows and you want DBConvert Streams to copy those rows before continuous CDC starts.

With this option enabled, a CDC run captures a consistent source snapshot, copies existing rows through the normal stream pipeline, then starts CDC from the source log position that belongs to that snapshot.

For a copy-paste lab with setup SQL, live CDC changes, and stop/start catch-up checks, see Initial Load + CDC Walkthrough.

Why it is opt-in

Initial snapshot is disabled by default because it changes CDC startup from "apply new changes only" to "copy all selected existing rows first, then apply changes". That copy can be large and must pass the target requirements below before CDC starts.

If a target table already contains rows that are not present in the source snapshot, an upsert snapshot cannot remove those rows. DBConvert Streams fails the run instead of reporting success with stale target data.

When to use it

Use this flow when:

the source is MySQL or PostgreSQL
the stream is in cdc mode
the stream has one database source connection
the selected target tables are empty before the first data copy
the target write mode is upsert
selected tables have primary keys required by upsert writes
you want one CDC run to perform the initial load and then continue with ongoing changes

Use Load mode instead when you need a bounded migration job, custom SQL extraction, multi-source input, file/S3 sources, or a manual migration step before CDC.

What happens at runtime

DBConvert Streams captures a source-side snapshot boundary.
Existing rows are copied to the target.
Changes committed while the snapshot is being copied remain in the source log.
CDC starts from the handoff position captured with the snapshot.
The target receives later INSERT, UPDATE, and DELETE changes.
The CDC checkpoint advances after target writes are acknowledged.

PostgreSQL uses a logical replication slot with an exported snapshot. MySQL uses a consistent InnoDB snapshot with captured binlog coordinates.

For PostgreSQL sources, the Database Overview and CDC wizard show replication slot usage, for example 3 / 5 · 2 free. If no slot is free, PostgreSQL cannot create a new CDC slot until an unused slot is dropped or max_replication_slots is increased.

Create a CDC stream with Initial Load + CDC

In the UI, create a CDC stream and enable Initial Load + CDC in the source options. The Monitor tab shows the CDC Continuity card while bootstrap runs and after CDC handoff.

For API-created configs, set source.options.initialSnapshot.enabled:

{
  "mode": "cdc",
  "source": {
    "connections": [
      {
        "connectionId": "conn_source",
        "database": "source",
        "schema": "public",
        "tables": [
          { "name": "customers" }
        ]
      }
    ],
    "options": {
      "initialSnapshot": {
        "enabled": true,
        "restartPolicy": "fail_until_reset"
      },
      "operations": ["INSERT", "UPDATE", "DELETE"]
    }
  },
  "target": {
    "id": "conn_target",
    "spec": {
      "db": {
        "database": "target",
        "schema": "public",
        "schemaPolicy": "create_missing_only",
        "writeMode": "upsert"
      }
    }
  }
}

Expected behavior

Existing source rows, automatic bootstrap

If initialSnapshot.enabled=true, DBConvert Streams copies the existing source rows first, then continues CDC from the matching handoff position.

You do not need to run a separate Load stream for this flow.

Stop and start after bootstrap completes

After the bootstrap phase is complete, stopping and starting the same stream config resumes normal CDC from the saved checkpoint. The initial snapshot is not copied again.

Source changes while the stream is stopped

If the source database keeps the required binlog or WAL range, starting the same config later applies changes that accumulated while the stream was stopped.

If the required source log range has expired, the stream cannot safely resume from its checkpoint. Create a new target state with a fresh initial snapshot or run a controlled repair procedure.

Target requirements

Initial snapshot requires upsert-compatible table metadata and empty target tables before the first data copy.

For database targets, DBConvert Streams can create target structure before the snapshot when structure creation is enabled and the schema policy allows missing tables to be created. Use Create missing only for a new target so missing selected tables are created, while existing selected tables are validated instead of replaced.

Create missing only does not bypass the empty-target requirement. After structure creation, DBConvert Streams verifies that selected target tables are empty and fails instead of truncating or cleaning target data automatically. If the stream uses Validate existing or structure creation is disabled, provide compatible empty target tables yourself.

For the schema policy details, see Target Schema and Data Policies.

Interrupted bootstrap

Default behavior is restartPolicy: "fail_until_reset". If the process is interrupted while the snapshot is being copied, the next start fails until the target state is reset. This avoids continuing from a partial snapshot that may contain stale rows.

For MySQL sources, DBConvert Streams automatically routes eligible tables to the resumable chunked snapshot path and keeps ineligible tables on the plain path. Chunked tables can resume from the last completed primary-key range when the same stream config is started again. Plain-path tables still follow fail_until_reset behavior. PostgreSQL CDC bootstrap still uses the plain non-resumable path.

Monitor visibility

When initial snapshot is enabled, the Monitor tab shows CDC Continuity:

Snapshot shows copying, handoff, or complete.
Snapshot Ended At shows the handoff source log position.
CDC Apply shows the latest acknowledged CDC checkpoint.
For PostgreSQL, the footer also shows the replication slot name.
The footer identifies the source position type, retry-safe apply behavior, ordered-per-table processing, and last update time.

File and S3 targets

For file and S3 targets in CDC mode, initial snapshot plus CDC produces current-state snapshot files, not append-only CDC log files. The file target applies inserts, updates, and deletes to its staged table state and writes a deterministic part-*-current.* file for each table and writer. S3 targets use the same file behavior, then upload the current files to the configured bucket.

Resume state

CDC resume state is stored in the embedded NATS JetStream KV bucket streams, keyed by stream config ID:

<configID>.cdc.bootstrap stores initial snapshot phase, handoff position, selected table progress, and PostgreSQL slot details when available.
<configID>.cdc.checkpoint stores the durable CDC apply checkpoint after target writers acknowledge changes.
<configID>.loadprogress — chunked copy progress for MySQL bootstrap; see Resumable Load.

Keep the stream config when you want to resume from the saved checkpoint. Deleting the config removes the associated CDC bootstrap and checkpoint state, and (since v2.4.2) drops the inactive PostgreSQL replication slot so it stops retaining WAL on the source.

Reset

To restart a failed or completed bootstrap from scratch, use the reset endpoint:

curl -X POST \
  -H "X-API-Key: $API_KEY" \
  -H "X-Confirm-Reset: true" \
  "$HOST/api/v1/stream-configs/$CONFIG_ID/reset"

Reset performs the following in order:

Truncates target rows already loaded by this config.
For PostgreSQL sources, drops the inactive replication slot created by the bootstrap.
Clears all durable recovery keys: load-progress state, CDC bootstrap state, and CDC apply checkpoint.

The stream must be stopped before reset. After reset, the next start runs as a fresh first run.

Guarantees and boundaries

Initial Load + CDC provides a consistent initial copy followed by retry-safe CDC apply. The current release does not include:

automatic source schema evolution after stream start
automatic recovery from an interrupted plain-path snapshot (MySQL chunked tables resume; PostgreSQL bootstrap and plain-path MySQL tables require reset)
multi-source CDC bootstrap
file or S3 source bootstrap, because file and S3 sources do not provide database transaction logs

For schema changes, apply and validate compatible target schema changes as part of your operational procedure.