Execution time settings#

class flytekit.configuration.PlatformConfig(endpoint='localhost:30080', insecure=False, insecure_skip_verify=False, ca_cert_file_path=None, console_endpoint=None, command=None, proxy_command=None, client_id=None, client_credentials_secret=None, scopes=<factory>, auth_mode=AuthType.STANDARD, audience=None, rpc_retries=3, http_proxy_url=None)#

This object contains the settings to talk to a Union backend (the DNS location of your Admin server basically).

Parameters:

  • endpoint (str) – DNS for Union backend

  • insecure (bool) – Whether or not to use SSL

  • insecure_skip_verify (bool) – Whether to skip SSL certificate verification

  • console_endpoint (str | None) – endpoint for console if different from Union backend

  • command (List[str] | None) – This command is executed to return a token using an external process

  • proxy_command (List[str] | None) – This command is executed to return a token for proxy authorization using an external process

  • client_id (str | None) – This is the public identifier for the app which handles authorization for a Flyte deployment. More details here: https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/.

  • client_credentials_secret (str | None) – Used for service auth, which is automatically called during pyflyte. This will allow the Flyte engine to read the password directly from the environment variable. Note that this is less secure! Please only use this if mounting the secret as a file is impossible

  • scopes (List[str]) – List of scopes to request. This is only applicable to the client credentials flow

  • auth_mode (AuthType) – The OAuth mode to use. Defaults to pkce flow

  • ca_cert_file_path (str | None) – [optional] str Root Cert to be loaded and used to verify admin

  • http_proxy_url (str | None) – [optional] HTTP Proxy to be used for OAuth requests

  • audience (str | None)

  • rpc_retries (int)

classmethod auto(config_file=None)#

Reads from Config file, and overrides from Environment variables. Refer to ConfigEntry for details :param config_file: :return:

Parameters:

config_file (str | ConfigFile | None)

Return type:

PlatformConfig

class flytekit.configuration.StatsConfig(host='localhost', port=8125, disabled=False, disabled_tags=False)#

Configuration for sending statsd.

Parameters:

  • host (str) – The statsd host

  • port (int) – statsd port

  • disabled (bool) – Whether or not to send

  • disabled_tags (bool) – Turn on to reduce cardinality.

classmethod auto(config_file=None)#

Reads from environment variable, followed by ConfigFile provided :param config_file: :return:

Parameters:

config_file (str | ConfigFile)

Return type:

StatsConfig

class flytekit.configuration.SecretsConfig(env_prefix='_FSEC_', default_dir='/etc/secrets', file_prefix='')#

Configuration for secrets.

Parameters:

  • env_prefix (str) – This is the prefix that will be used to lookup for injected secrets at runtime.

  • default_dir (str) – This is the default directory that will be used to find secrets as individual files under.

  • file_prefix (str) – This is the prefix for the file in the default dir.

classmethod auto(config_file=None)#

Reads from environment variable or from config file :param config_file: :return:

Parameters:

config_file (str | ConfigFile)

Return type:

SecretsConfig

class flytekit.configuration.S3Config(enable_debug=False, endpoint=None, retries=3, backoff=datetime.timedelta(seconds=5), access_key_id=None, secret_access_key=None)#

S3 specific configuration

Parameters:

  • enable_debug (bool)

  • endpoint (str | None)

  • retries (int)

  • backoff (timedelta)

  • access_key_id (str | None)

  • secret_access_key (str | None)

classmethod auto(config_file=None)#

Automatically configure :param config_file: :return: Config

Parameters:

config_file (str | ConfigFile)

Return type:

S3Config

class flytekit.configuration.GCSConfig(gsutil_parallelism=False)#

Any GCS specific configuration.

Parameters:

gsutil_parallelism (bool)

class flytekit.configuration.DataConfig(s3=S3Config(enable_debug=False, endpoint=None, retries=3, backoff=datetime.timedelta(seconds=5), access_key_id=None, secret_access_key=None), gcs=GCSConfig(gsutil_parallelism=False), azure=AzureBlobStorageConfig(account_name=None, account_key=None, tenant_id=None, client_id=None, client_secret=None), generic=GenericPersistenceConfig(attach_execution_metadata=True))#

Any data storage specific configuration. Please do not use this to store secrets, in S3 case, as it is used in Flyte sandbox environment we store the access key id and secret. All DataPersistence plugins are passed all DataConfig and the plugin should correctly use the right config

Parameters:

  • s3 (S3Config)

  • gcs (GCSConfig)

  • azure (AzureBlobStorageConfig)

  • generic (GenericPersistenceConfig)