Importing and exporting data with federated queries
Note
When importing or exporting data to or from S3 in Parquet format, take into account the YQL and Apache Arrow type mapping.
Importing data
Federated queries let you import data from connected external sources into YDB tables. To import data, use a read query from an external data source or external table and write into a YDB table.
For column-oriented tables, massively parallel import from an external source is supported when using UPSERT and INSERT: several worker threads read data from the external source in parallel and write into the table. For row-oriented tables, this functionality is under development.
| Operation | Writes to row-oriented tables | Writes to column-oriented tables |
|---|---|---|
| UPSERT | single-threaded | parallel |
| REPLACE | single-threaded | parallel |
| INSERT | single-threaded | parallel |
Tip
The recommended import options using federated queries are UPSERT and REPLACE — the import path is heavily optimized for them.
Example: import data from a PostgreSQL table into a YDB table:
UPSERT INTO target_table
SELECT * FROM postgresql_datasource.source_table
For more on creating external data sources and external tables, and on read queries, see:
Exporting data
Currently, exporting data with federated queries is supported only for S3-compatible storage; see Exporting data to S3 object storage.