S3

Package: flyte.storage

S3 specific configuration.

Authentication resolution used by Flyte + obstore:

1. If explicit static credentials are provided via Flyte S3 inputs/environment (access_key_id/secret_access_key), those are used.

2. If static credentials are not provided, and both AWS_PROFILE and AWS_CONFIG_FILE are available, Flyte configures a boto3-backed obstore credential provider so profile-based auth can be used. This requires that the boto3 library is installed.

3. If neither of the above applies, obstore uses the default AWS credential chain (for remote runs this commonly resolves via workload identity / IAM attached to the service account and then IMDS fallbacks where applicable).

Parameters

class S3(
    retries: int,
    backoff: datetime.timedelta,
    enable_debug: bool,
    attach_execution_metadata: bool,
    endpoint: typing.Optional[str],
    access_key_id: typing.Optional[str],
    secret_access_key: typing.Optional[str],
    region: typing.Optional[str],
    addressing_style: typing.Optional[str],
)

Methods

auto()

def auto(
    region: str | None,
) -> S3

Returns: Config

for_sandbox()

def for_sandbox()

get_fsspec_kwargs()

def get_fsspec_kwargs(
    anonymous: bool,
    kwargs,
) -> typing.Dict[str, typing.Any]

Returns the configuration as kwargs for constructing an fsspec filesystem.