Docs/Streams/Features

Table Structure Conversion

DBConvert Streams automatically converts table structure between source and target when they use different engines or formats. This includes data types, indexes, foreign keys, and check constraints.

Structure conversion applies to both Load and CDC modes.

Supported conversion paths

Database ↔ database

  • MySQL → PostgreSQL
  • PostgreSQL → MySQL
  • MariaDB → PostgreSQL
  • PostgreSQL → MariaDB
  • MySQL / PostgreSQL → Snowflake

Files / S3 → database

  • CSV → MySQL / PostgreSQL
  • JSON / JSONL → MySQL / PostgreSQL
  • Parquet → MySQL / PostgreSQL

Database → files / S3

  • MySQL / PostgreSQL → CSV, JSONL, or Parquet files (local or S3)

Structure options

Structure creation is controlled per-stream via the structureOptions object in the target spec. All options are booleans and default to true.

OptionDefaultEffect
tablestrueCreate tables on the target if they do not exist
indexestrueCreate indexes (primary keys, unique, secondary)
foreignKeystrueCreate foreign key constraints
checkConstraintstrueCreate check constraints

Set an option to false to skip that structure type. For example, disabling indexes on large tables can speed up the initial load — you can create them manually afterward.

API example

{
  "target": {
    "id": "conn_TARGET_ID",
    "spec": {
      "db": {
        "database": "target_db",
        "structureOptions": {
          "tables": true,
          "indexes": false,
          "foreignKeys": false,
          "checkConstraints": true
        }
      }
    }
  }
}

In the UI, the stream wizard Structure & Data step exposes these as checkboxes under the advanced section.

Data type mapping

These tables are generated from the conversion code itself, so they describe what the current release actually does. Each row is one column as it exists in the source — a MySQL database, a PostgreSQL database, or a file — and the columns marked with an arrow are the type it gets on that target.

Moving between two databases of the same engine is not in the tables. MySQL to MySQL, or PostgreSQL to PostgreSQL, recreates each column exactly as it was declared — nothing is mapped. File targets (Parquet, CSV, JSONL) go through DuckDB, so its type is what lands in the file's schema.

Notes say what a mapping cannot carry. No note means nothing is lost.

Snowflake appears in the tables because the mapping exists, but the Snowflake target is still in development.

From a MySQL source

Each row is one MySQL column and what it becomes on each target.

MySQL column→ PostgreSQL→ Snowflake→ Parquet / CSV / JSONLNotes
TINYINT(1)BOOLEANNUMBER(3,0)TINYINTMySQL's boolean
TINYINT UNSIGNEDSMALLINTNUMBER(3,0)UTINYINTwidened, because a signed TINYINT cannot hold 255
INTINTEGERNUMBER(10,0)INTEGER
INT UNSIGNEDBIGINTNUMBER(10,0)UINTEGERwidened
BIGINT UNSIGNEDNUMERIC(20,0)NUMBER(19,0)UBIGINTwidened
DECIMAL(20,6)NUMERIC(20,6)NUMBER(20,6)DECIMAL(20,6)
FLOATREALFLOATFLOAT
DOUBLEDOUBLE PRECISIONFLOATDOUBLE
BIT(1)BIT(1)VARIANTVARCHAR
CHAR(10)CHAR(10)VARCHAR(10)CHAR(10)
VARCHAR(255)VARCHAR(255)VARCHAR(255)VARCHAR(255)
TEXTTEXTVARCHARVARCHAR
LONGTEXTTEXTVARCHARVARCHAR
BINARY(16)BYTEABINARY(8388608)BLOB
BLOBBYTEAVARCHAR(25000000)BLOB
LONGBLOBBYTEAVARCHAR(25000000)BLOB
DATEDATEDATEDATE
DATETIMETIMESTAMP(0)TIMESTAMP_NTZTIMESTAMP
DATETIME(3)TIMESTAMP(3)TIMESTAMP_NTZTIMESTAMP
TIMESTAMPTIMESTAMP(0)TIMESTAMP_NTZTIMESTAMPMySQL normalizes to UTC on write; the target column is not zone-aware, so that conversion does not follow the data
TIME(6)INTERVALTIMETIME
YEARSMALLINTNUMBER(4,0)VARCHAR
ENUMTEXTVARCHARVARCHARon PostgreSQL: text plus a CHECK constraint holding the labels
SETTEXTVARCHARVARCHAR
JSONJSONBVARIANTJSON
GEOMETRYTEXTGEOGRAPHYVARCHARcarried as text; spatial indexes are not recreated

From a PostgreSQL source

Each row is one PostgreSQL column and what it becomes on each target.

PostgreSQL column→ MySQL→ Snowflake→ Parquet / CSV / JSONLNotes
SMALLINTSMALLINTNUMBER(5,0)SMALLINT
INTEGERINTNUMBER(10,0)INTEGER
BIGINTBIGINTNUMBER(19,0)BIGINT
NUMERIC(20,6)DECIMAL(20,6)NUMBER(20,6)DECIMAL(20,6)
REALFLOATFLOATFLOAT
DOUBLE PRECISIONDOUBLEFLOATDOUBLE
MONEYDECIMAL(19,2)VARIANTDECIMAL(19,2)the currency symbol belongs to the server's locale, not the value, so only the amount travels
BOOLEANTINYINT(1)BOOLEANBOOLEAN
CHAR(10)CHAR(10)VARCHAR(10)CHAR(10)
VARCHAR(255)VARCHAR(255)VARCHAR(255)CHAR(255)
TEXTLONGTEXTVARCHARVARCHAR
BYTEALONGBLOBVARCHAR(25000000)BLOB
DATEDATEDATEDATE
TIMESTAMPDATETIME(6)TIMESTAMP_NTZTIMESTAMP
TIMESTAMPTZTIMESTAMP(6)TIMESTAMP_TZTIMESTAMP
TIMETIME(6)TIMETIME
TIMETZTEXTVARIANTTIMETZonly PostgreSQL has a zone-aware time; elsewhere the offset is kept in text
INTERVALVARCHAR(255)VARCHAR(255)VARCHAR
UUIDCHAR(36)VARCHAR(36)VARCHAR
JSONJSONVARIANTJSON
JSONBJSONVARIANTJSON
TEXTJSONVARIANTVARCHARno other engine has array columns; the values are carried as JSON
INTEGERJSONVARIANTVARCHARcarried as JSON
ENUM typeENUM('sad','ok')VARIANTVARCHAR
INETVARCHAR(45)VARCHAR(45)VARCHAR
CIDRTEXTVARIANTVARCHAR
MACADDRVARCHAR(17)VARCHAR(17)VARCHAR
BIT(8)BIT(8)VARIANTVARCHAR
TSVECTORTEXTVARIANTVARCHARthe searchable form is rebuilt by the target, not copied
POINTTEXTGEOGRAPHYVARCHARcarried as text; spatial indexes are not recreated
XMLTEXTVARIANTVARCHAR

From a file source

Each row is one column as DuckDB reports it from a Parquet, CSV or JSON file, and what it becomes on each database target. CSV and JSON have no types of their own, so these are the types inferred from their contents.

File column type→ MySQL→ PostgreSQL→ SnowflakeNotes
BOOLEANTINYINT(1)BOOLEANBOOLEAN
TINYINTTINYINTSMALLINTNUMBER(3,0)
UTINYINTTINYINT UNSIGNEDSMALLINTNUMBER(3,0)unsigned; widened where the target has no unsigned type
SMALLINTSMALLINTSMALLINTNUMBER(5,0)
INTEGERINTINTEGERNUMBER(10,0)
UINTEGERINT UNSIGNEDBIGINTNUMBER(10,0)unsigned; widened
BIGINTBIGINTBIGINTNUMBER(19,0)
UBIGINTBIGINT UNSIGNEDNUMERIC(20,0)NUMBER(19,0)unsigned; widened
HUGEINTTEXTTEXTVARIANT128-bit; no target has one, so it is carried as text
FLOATFLOATREALFLOAT
DOUBLEDOUBLEDOUBLE PRECISIONFLOAT
DECIMAL(20,6)DECIMAL(20,6)NUMERIC(20,6)NUMBER(20,6)precision and scale are carried, not rounded to a float
VARCHARLONGTEXTTEXTVARCHARDuckDB strings carry no length, so a length is not invented
DATEDATEDATEDATE
TIMESTAMPDATETIME(6)TIMESTAMPTIMESTAMP_NTZ
TIMESTAMP WITH TIME ZONETIMESTAMP(6)TIMESTAMPTZTIMESTAMP_TZ
TIMETIME(6)TIMETIME
BLOBBLOBBYTEAVARCHAR(25000000)
UUIDCHAR(36)TEXTVARCHAR(36)
INTERVALVARCHAR(255)TEXTVARCHAR(255)

Types not listed map to text on a database target and VARCHAR in a file.

File source → database target

When the source is a local file or S3 object, DBConvert Streams uses DuckDB to infer the schema:

  • Parquet: schema is read from the file metadata (column names, types, nullability)
  • CSV: read_csv_auto() infers column names from headers and types from sampled values
  • JSON / JSONL: read_json_auto() infers structure from the document fields

What each inferred type becomes on a database target is in From a file source above.

Database source → file target

When the target is a local directory or an S3 bucket, columns are written through DuckDB — the Parquet / CSV / JSONL column of the tables above is the type that ends up in the file's schema. Parquet carries those types in its metadata; CSV and JSONL are text formats, so the type shapes how each value is written rather than being stored.

Two properties worth knowing:

  • Decimals stay exact. A DECIMAL(20,6) is written as a decimal, not as a floating-point approximation.
  • Unsigned integers keep their range. A MySQL TINYINT UNSIGNED holding 255 is written as an unsigned type, not as -1.

Output format is determined by the target configuration:

FormatFile extensionDefault compression
CSV.csvuncompressed
JSONL.jsonluncompressed
Parquet.parquetzstd

All formats also support gzip and zstd compression. Parquet additionally supports snappy.

Default value conversion

DBConvert Streams also converts default values between database engines:

  • MySQL → PostgreSQL: MySQL zero dates (0000-00-00) convert to NULL. Boolean defaults 0/1 convert to false/true.
  • PostgreSQL → MySQL: PostgreSQL sequences (nextval()) are skipped. Boolean defaults true/false convert to 1/0. Type casts (::type) are removed.

Check constraint conversion

Check constraints are translated between SQL dialects:

  • PostgreSQL → MySQL: ANY(ARRAY[...]) converts to IN(...). PostgreSQL functions like jsonb_typeof map to MySQL equivalents like JSON_TYPE.
  • MySQL → PostgreSQL: MySQL functions like json_valid map to PostgreSQL equivalents like jsonb_typeof. Backtick identifiers convert to double quotes.