Review and Verify a Load

DBConvert Streams checks a Load at both ends:

• Before it starts, you review a per-table plan: which tables will be created, used as they are, or dropped and recreated. Nothing is written to the target to build this plan.
• After it finishes, Verify data compares source and target. It first compares totals for every table, then compares the rows themselves wherever the totals differ, and names the rows that do not match.

Review before start

When you start a stream, DBConvert Streams reads the source and target catalogs and shows the plan for every selected table, based on the stream's schema policy and write mode.

Building the plan is read-only: no table is created, dropped, or truncated, and no test table is written to find out whether you have permissions.

Replacing tables needs a second confirmation. When the plan drops existing tables, Replace and start stays disabled until you tick the checkbox that names how many tables will lose their data.

A blocked plan cannot be started. The dialog lists the tables and why they are blocked, and offers only Close. Common cases:

• Fail if exists with tables that already exist on the target: choose another schema policy or drop those tables.
• A target database that does not exist: DBConvert Streams does not create databases. Create it, or choose an existing database in the stream configuration.

The plan you start is the plan you reviewed. If the source, the target, or the stream configuration changes between the review and the start, the start is refused and the review opens again with the current plan.

Verify data after a Load

Open a finished Load run and select Verify data in the Tables card. Verification runs in the background; the header shows its progress and a Cancel button, and you can leave the page while it works.

Each table goes through up to three checks.

1. Schema

The target table must exist and its columns, types, and primary key must be compatible with the source. A table whose schema does not match shows Schema differs, and its data is not scanned.

2. Totals

DBConvert Streams runs one aggregate query per table on the source and one on the target, at the same time, and compares:

A total is computed only when both sides store the column in the same kind of type. A MySQL tinyint(1) loaded into a PostgreSQL boolean, binary and JSON columns, and timestamps that depend on the session time zone are compared by non-null count only.

If the stream selects rows with a filter or a limit, the source totals cover exactly the rows the run was configured to copy.

Sums of floating-point columns are compared with a tolerance: MySQL and PostgreSQL add millions of values in a different order and the result can differ in the 13th significant digit. These values are shown with ≈, and hovering one shows both exact sums.

When every total matches, the table shows Totals match with its row count, and the totals are folded into one line. Show values opens them as a table of columns by total.

3. Rows

Totals show that a table differs, not where. Rows are compared:

• automatically, for every table whose totals differ, and
• on request, for any finished table, with Find differing rows in the row actions or in the expanded result, even when the totals match.

Both tables are read in primary-key order, in ranges of 50,000 keys, and every row present on both sides is compared column by column. Source values are first converted the way the Load converted them when writing, so a type mapping such as tinyint(1) to boolean is not reported as a difference.

Each difference is one of:

Up to 10,000 differences are counted per table, and the first 1,000 are stored with their values. The expanded result lists the first 50.

When every row matches, the table shows All N rows identical.

Reading a result

Every result shows when it was verified. Verification compares the target with the source as the source is at that moment: rows written to the source after the run finished also appear as differences.

Verify automatically after every run

In the Structure and Data step, turn on Verify data after run under the write mode. Every Load run that finishes successfully then verifies itself: totals for every table, and rows for tables whose totals differ.

The option is off by default because it reads every table on both databases once more after the load.

Cost

Totals are one full scan per table on each database. Comparing rows reads each table from both databases in full and compares the values in DBConvert Streams, so most of the work is done by the application rather than the databases. One row comparison runs at a time.

For scale, on a developer workstation with MySQL 8.0 and PostgreSQL 16 running locally in Docker, a 20,000,000-row, 3.8 GB table took:

Limitations

• Verification is available for finished Load runs. CDC runs are not verified.
• Source and target must be MySQL or PostgreSQL. Other targets report Not checked.
• Custom SQL queries on a single database source are verified by totals: the query runs again on the source and its totals are compared with the target table. Rows are not compared, because a query output has no primary key. A query using NOW(), RANDOM(), or LIMIT without ORDER BY can return different rows when it runs again.
• Federated queries and queries over files or S3, which run through DuckDB, are listed as Not checked.
• Comparing rows needs a single integer primary key. Tables with a composite, text, or missing primary key are verified by totals only.
• A selection that combines a limit with a custom sort order cannot be matched by key; its rows are not compared.
• On an evaluation Load, text values after the first rows of each table are marked, so their values are not compared; counts, numbers, dates, and keys still are.
• Verification reports differences. It does not change the target.

API

Start verification of a finished run:

curl -X POST "$API_URL/api/v1/stream-configs/$CONFIG_ID/runs/$RUN_ID/verify" \
  -H "X-API-Key: $API_KEY" -H "Content-Type: application/json" \
  -d '{"level": "aggregate"}'

The request returns 202 and the result is stored on the run: poll GET /api/v1/stream-configs/{id}/runs/{runId} and read verification. Stop a running verification with POST /api/v1/stream-configs/{id}/runs/{runId}/verify/cancel.

To turn on verification after every run, set verifyAfterRun in the database target spec:

{
  "target": {
    "spec": {
      "db": {
        "database": "analytics",
        "schemaPolicy": "create_missing_only",
        "writeMode": "truncate_and_load",
        "verifyAfterRun": true
      }
    }
  }
}

Starting a stream through the API follows the same review contract as the UI: request the plan with POST /api/v1/stream-configs/{id}/readiness, then pass its fingerprint to POST /api/v1/stream-configs/{id}/start as readinessFingerprint, with confirmDestructive: true when the plan replaces tables. A plan that changed in between is refused with 409 and code stale_plan; a plan with blocked tables is refused with 400 before a run is created. See the API reference for the full schemas.

Related Docs

• Target Schema and Data Policies
• Resumable Load
• Stream States and Lifecycle