Snowflake agent

Snowflake agent#

Union can be seamlessly integrated with the Snowflake service, allowing you to easily query data in Snowflake.

Installation#

To use the Snowflake agent, run the following command:

pip install flytekitplugins-snowflake

Configuring key-pair authentication#

  1. Generate the private key

  2. Assign the public key to a Snowflake user

Get the metadata for the task#

The user, account, database, warehouse, and schema metadata are required to configure the Snowflake task.

Run the following SQL in the Snowflake console to get the metadata:

SELECT
    CURRENT_USER() AS "User",
    CONCAT(CURRENT_ORGANIZATION_NAME(), '-', CURRENT_ACCOUNT_NAME()) AS "Account",
    CURRENT_DATABASE() AS "Database",
    CURRENT_SCHEMA() AS "Schema",
    CURRENT_WAREHOUSE() AS "Warehouse";

Example usage#

For a usage example, see Snowflake agent example.

Local testing#

Create a secret file that contains the Snowflake private key:

sudo vim /etc/secrets/snowflake

Union cluster deployment#

After you have finished testing the agent locally, contact the Union team to enable it in your cluster.

Contents