You do not yet have any connection for {{dataset.type|datasetTypeToName}}. You need to create a new connection before creating datasets on this connection.

Connection

Manage connections

BigQuery project Database Database Catalog

Leave blank to use the Project ID associated with the connection Leave blank to use the Snowflake database associated with the connection Leave blank to use the MS SQL Server database associated with the connection Leave blank to use the Databricks catalog associated with the connection Leave blank to use the catalog associated with the connection

BigQuery dataset Schema

Table

Datetime mapping

Use DATETIME instead of TIMESTAMP to store "datetime with tz" values

Normalize floats & doubles

Normalize floating point values (force '42' to '42.0')

Teradata settings

Primary index strategy

Index columns

Greenplum settings

Distribution strategy

Distribution key

For partitioned datasets, the partitioning dimensions are always prepended to the sort keys

Sort key

Use an hash based distribution

(Azure documentation on distributed tables)

Distribution column

Forbids writing to a non-partitioned BigQuery table if the dataset is partitioned.

Partitioning configuration mismatch between the dataset and the BigQuery native partitioning.

DSS will create a non-partitioned BigQuery table for this partitioned dataset, but writing to the table will fail as it is forbidden. Configure BigQuery native partitioning or disable 'Partitioning consistency'.

Partitioning

Create BigQuery partitioned table

(Google BigQuery documentation on partitioned tables)

Partition by column

(inclusive)

End (exclusive)

Interval (must be a positive integer)

Cluster by column(s)

Table creation SQL

Note: The $DKU_CREATE_TABLE_FIELDS will be replaced by the list of fields and types from the schema

Pre-write statements

These statements will be executed before Data Science Studio writes data to the table. You might want to remove temporarily some indices for example

Post-write statements

These statements will be executed after Data Science Studio writes data to the table. You might want to create some indices for example

Teradata settings

Greenplum settings

SQL Spark integration