Configuration parameters

pg_lake is configured through PostgreSQL settings. Set them in postgresql.conf, with ALTER SYSTEM, per database or user with ALTER DATABASE ... SET and ALTER ROLE ... SET, or per session with SET. For each parameter, set by says who can change it and when the change takes effect:

  • user: any user, with SET.
  • superuser: superusers, with SET or in the configuration.
  • reload: in the configuration, followed by SELECT pg_reload_conf().
  • restart: in the configuration, followed by a server restart.

The query engine itself, pgduck_server, is configured with command-line options; see pgduck_server options.

  1. Query engine connection and cache
  2. Iceberg tables
  3. Writes and files
  4. Query planning
  5. Maintenance
  6. Catalogs
  7. Security

Query engine connection and cache

Parameter Description
pg_lake_engine.hostDefault host=/tmp port=5332, set by restart Connection string for pgduck_server.
pg_lake_engine.enable_cache_managerDefault on, set by superuser Run a background worker that keeps the pgduck_server file cache under max_cache_size and fills it with recently queried files.
pg_lake_engine.max_cache_sizeDefault 20GB, set by superuser Size limit for the file cache.
pg_lake_engine.cache_manager_intervalDefault 10s, set by superuser Delay between cache manager runs.
pg_lake_engine.max_parallel_file_uploadsDefault 12, set by user Maximum number of concurrent file uploads to object storage per write.
pg_lake_engine.log_engine_errorsDefault on, set by user Log a short classification of query engine errors in the PostgreSQL log.

Iceberg tables

Parameter Description
pg_lake_iceberg.default_location_prefixDefault none, set by superuser URL prefix under which new Iceberg tables store their files, such as s3://bucket/iceberg.
pg_lake_iceberg.default_catalogDefault postgres, set by user Catalog for new Iceberg tables: postgres, rest, object_store, or the name of a catalog server.
pg_lake_iceberg.default_compatibility_modeDefault auto, set by user compatibility_mode for new Iceberg tables: auto or snowflake.
pg_lake_iceberg.unsupported_numeric_as_doubleDefault on, set by user Store unbounded numeric and numeric with precision above 38 as double precision. When off, such columns are rejected.
pg_lake_iceberg.max_snapshot_ageDefault 1800, set by superuser Seconds to retain old snapshots before VACUUM expires them. Overridden per table by max_snapshot_age.
pg_lake_iceberg.default_avro_writer_block_size_kbDefault 64kB, set by superuser Block size for manifest (Avro) files.

Writes and files

Parameter Description
pg_lake_table.target_file_size_mbDefault 512MB, set by user Target size of data files; larger files are split during writes and compaction. A value below 1 disables splitting.
pg_lake_table.target_row_group_size_mbDefault 128MB, set by superuser Target Parquet row group size. 0 disables the target.
pg_lake_table.default_parquet_versionDefault v1, set by superuser Parquet format version for new files: v1 or v2.
pg_lake_table.copy_on_write_thresholdDefault 20, set by user Percentage of deleted rows in a file above which a delete rewrites the file instead of writing a position delete file. 0 always rewrites, 100 never does.
pg_lake_table.max_open_files_for_partitioned_writeDefault 5000, set by superuser Maximum number of partition staging files a write keeps open. See partitioning.
pg_lake_table.enable_partitioned_write_pushdownDefault off, set by user Let DuckDB write partitioned INSERT ... SELECT and COPY FROM statements directly.

Query planning

Parameter Description
pg_lake_table.enable_full_query_pushdownDefault on, set by user Push whole queries down to DuckDB when every part of the query can run there.
pg_lake_table.enable_strict_pushdownDefault on, set by user Only push down functions, operators and types whose DuckDB behavior is known to match PostgreSQL.
pg_lake_table.enable_data_file_pruningDefault on, set by superuser Skip Iceberg data files whose column statistics rule out a match.
pg_lake_table.enable_partition_pruningDefault on, set by superuser Skip Iceberg data files whose partition values rule out a match.

See performance for how these affect queries.

Maintenance

Parameter Description
pg_lake_iceberg.autovacuumDefault on, set by reload Run the Iceberg autovacuum worker.
pg_lake_iceberg.autovacuum_naptimeDefault 10min, set by reload Time between autovacuum runs.
pg_lake_iceberg.autovacuum_lock_timeoutDefault 1min, set by reload How long autovacuum waits for a table lock before skipping the table.
pg_lake_iceberg.log_autovacuum_min_durationDefault 10min, set by reload Log autovacuum runs that take longer than this. -1 disables.
pg_lake_table.max_compactions_per_vacuumDefault 100, set by superuser Maximum compactions in a single VACUUM run.
pg_lake_table.max_file_removals_per_vacuumDefault 10000, set by superuser Maximum file deletions in a single VACUUM run.
pg_lake_engine.orphaned_file_retention_periodDefault 10d, set by superuser How long unreferenced files are kept before VACUUM deletes them.
pg_lake_engine.vacuum_file_remove_max_retriesDefault 145, set by superuser Attempts to delete a file before giving up on it.
pg_lake_table.enable_delete_file_functionDefault off, set by superuser Allow lake_file.delete() to delete files.

Catalogs

Parameter Description
pg_lake_iceberg.rest_catalog_hostDefault http://localhost:8181/api/catalog, set by superuser Base URL of the built-in rest catalog.
pg_lake_iceberg.rest_catalog_client_idDefault none, set by superuser OAuth2 client ID for the rest catalog.
pg_lake_iceberg.rest_catalog_client_secretDefault none, set by superuser OAuth2 client secret for the rest catalog.
pg_lake_iceberg.rest_catalog_oauth_host_pathDefault none, set by superuser OAuth2 token endpoint URL, if not the catalog’s default.
pg_lake_iceberg.rest_catalog_scopeDefault PRINCIPAL_ROLE:ALL, set by superuser OAuth2 scope.
pg_lake_iceberg.rest_catalog_enable_vended_credentialsDefault off, set by superuser Request temporary storage credentials from the catalog.
pg_lake_iceberg.enable_object_store_catalogDefault on, set by reload Publish tables to the object store catalog.
pg_lake_iceberg.object_store_catalog_location_prefixDefault none, set by reload Storage location of the object store catalog.
pg_lake_iceberg.object_store_catalog_max_ageDefault 1min, set by reload Time after which the object store catalog file is rewritten even without changes.

Security

Parameter Description
pg_lake.allowed_azure_host_suffixesDefault Azure public, US government and China cloud endpoints, set by superuser Host suffixes that user-supplied Azure URLs may point to.

Access to object storage from SQL is controlled by roles rather than settings; see roles and permissions.