Target Configuration

Use this page when you need to define where a stream writes and what writer-specific settings apply to that target.

Every target configuration starts with target.id plus one target family under target.spec.

Target shape

The target object always looks like this:

{
  "id": "conn_target",
  "spec": {
    "db": {}
  }
}

Rules:

target.id selects the stored destination connection
target.spec contains exactly one target family
valid target families are db, files, and s3

Database targets

Use target.spec.db when the stream writes to MySQL or PostgreSQL.

Typical shape:

{
  "id": "conn_postgres_target",
  "spec": {
    "db": {
      "database": "analytics",
      "schema": "public",
      "schemaPolicy": "create_missing_only",
      "writeMode": "upsert"
    }
  }
}

Key points:

database selects the target database
schema is optional and depends on the target engine
structureOptions controls what structure objects may be created
schemaPolicy controls how existing tables are treated
writeMode controls how existing rows are treated
skipData turns the run into structure-only output

Use Target Schema and Data Policies and Database Structure Options for the behavioral choices behind those fields.

Local file targets

Use target.spec.files when the stream should write CSV, JSONL, or Parquet files to a local filesystem connection.

Typical shape:

{
  "id": "conn_files_local",
  "spec": {
    "files": {
      "fileFormat": "parquet",
      "format": {
        "compression": "zstd"
      }
    }
  }
}

Key points:

fileFormat chooses csv, jsonl, or parquet
format holds compression and format-specific settings
outputDirectory is optional and is often omitted because the connection base path determines the final location

S3 targets

Use target.spec.s3 when the stream should write files locally and then upload them to S3-compatible storage.

Typical shape:

{
  "id": "conn_s3_target",
  "spec": {
    "s3": {
      "fileFormat": "parquet",
      "format": {
        "compression": "zstd"
      },
      "upload": {
        "bucket": "analytics-exports",
        "prefix": "daily/"
      }
    }
  }
}

Key points:

outputDirectory is the local staging area before upload
fileFormat and format work the same way as local file targets
upload holds bucket, prefix, retention, and encryption behavior

Shared format settings

Use nested format settings under either target.spec.files.format or target.spec.s3.format.

Important notes:

compression and parquet are consumed by file-based targets in the current backend
csv exists in the schema, but custom CSV target formatting is not wired into the writer path yet
Parquet settings are available through the API and saved configs even when the UI does not expose them

Upload settings

Use target.spec.s3.upload when the target is S3-compatible storage.

Important fields:

bucket for the destination bucket
prefix for an optional object key prefix
keepLocalFiles when staged files should remain after upload
storageClass, serverSideEnc, and kmsKeyId for storage and encryption behavior

Verification

Before saving or using a target config:

confirm target.spec contains exactly one family object
confirm database targets keep policy fields under target.spec.db
confirm file targets keep format fields under target.spec.files.format or target.spec.s3.format
confirm S3 targets include upload settings when remote upload behavior matters
confirm CSV target formatting expectations match the current writer defaults