Iceberg Foreign Table

Apache Iceberg is a table format designed for data lakes, providing a unified way to manage table storage and metadata at scale. Iceberg supports advanced features like partitioning, data updates, time travel, and snapshots. Its Catalog interface manages and discovers metadata across multiple tables, supporting consistent table operations across environments and engines. Multiple Catalog backends—such as Hive Metastore, REST, and AWS Glue—are available for metadata storage and retrieval.

Tacnode integrates with Iceberg REST Catalog using foreign tables, enabling accelerated queries along with efficient data import and export.

Install the Extension

Install the extension once; skip if already installed.

CREATE EXTENSION iceberg_fdw;

Create an Iceberg REST Catalog Foreign Server

CREATE SERVER <server_name> FOREIGN DATA WRAPPER iceberg_fdw
OPTIONS (
 endpoint '<endpoint_url>',
 catalog '<catalog_name>'
);

• endpoint: The API endpoint of the Iceberg REST Catalog
• catalog: Root directory of the table metadata files

-- Example
CREATE SERVER iceberg_server FOREIGN DATA WRAPPER iceberg_fdw
OPTIONS (
 endpoint 'http://localhost:8181/api/catalog',
 catalog 'manual_spark'
);

Create a User Mapping

Grant access to object storage for the local user account.

CREATE USER MAPPING FOR public SERVER iceberg_server
OPTIONS (
 token 'principal:root;realm:default-realm'
);

Import Foreign Tables

Import metadata for Delta-type tables from the Iceberg REST Catalog.

IMPORT FOREIGN SCHEMA remote_schema_name
    { LIMIT TO (table_name [, ...])
    | EXCEPT (table_name [, ...]) }
    FROM SERVER server_name
    INTO local_schema_name
    [ OPTIONS (option 'value' [, ...]) ];

• remote_schema_name: Namespace in the Iceberg REST Catalog
• table_name: Table name in the Iceberg REST Catalog
• server_name: Previously defined foreign server

-- Create a foreign table for t1 under iceberg_namespace with metadata in the public schema.
IMPORT FOREIGN SCHEMA iceberg_namespace
    LIMIT TO (t1) FROM SERVER iceberg_server INTO public;
 
-- Query the Iceberg table
SELECT COUNT(1) FROM t1;
 
-- Write to the Iceberg table
INSERT INTO t1 SELECT ...;