Docs/Streams/Configuration

Stream Configuration Guide

This guide explains the main decisions you make when configuring a stream in DBConvert Streams.

Use this page for the operational workflow. Use Stream Configuration Overview for the mental model, Stream Configuration Reference for exact field paths, and Stream Configuration Examples for full payloads.

Use the Capability Matrix first when you need to confirm whether a source/target/mode combination is in scope.

Core stream decisions

Every stream starts with five decisions:

  1. choose the source
  2. choose the target
  3. choose the mode: Load or CDC
  4. define the scope: tables, files, or query-driven selection
  5. define execution settings such as reporting intervals, bundle size, and optional limits

Source and target selection

The selected connections determine what modes and scopes are available. Not every source supports every target, and explorer support does not imply CDC support.

Set up connections following Connection Management.

Mode selection

Choose Load for bounded transfers and CDC for ongoing synchronization. See Understanding DBConvert Streams for when to use each mode.

CDC is only valid for a subset of source/target combinations. Confirm support before selecting CDC.

Scope selection

The stream scope can be based on selected tables, schemas, custom SQL (Load only), or file paths depending on the source type. Use smaller scopes during first-run validation to reduce risk.

For detailed selection patterns, see the Schema Selection Guide. For query-driven scopes, see Custom SQL Queries.

Execution settings

Data bundle size (dataBundleSize)

Controls how many records are processed together per batch. Larger bundles improve throughput for simple datasets; smaller bundles are safer for wide rows or constrained environments. See the Configuration Reference for allowed values.

Reporting interval (reportingInterval)

Controls how often progress is surfaced, in seconds (default: 5). Use shorter intervals for closer visibility and longer intervals to reduce reporting overhead.

Optional limits (limits)

Streams can stop automatically based on:

  • numberOfEvents — maximum processed events
  • elapsedTime — maximum elapsed time in seconds

These are useful for testing, bounded CDC runs, and controlled trial executions.

Structure and target behavior

Structure handling determines how DBConvert Streams treats target-side tables and existing data. Two fields control this: schemaPolicy (what to do if tables exist) and writeMode (what to do if tables contain rows).

Use conservative defaults — especially in shared environments. See Target Schema and Data Policies for the full decision matrix and defaults per mode.

Validation workflow

Before starting a production-relevant stream:

  1. Confirm object visibility in Data Explorer.
  2. Validate representative rows with the SQL Console.
  3. Start with a smaller scope or use limits for a bounded first run.
  4. Monitor the first execution in Observability — verify the stream reaches finished state and the transferred row count matches expectations.
PreviousOverviewNextSource Configuration