2.0.0b53

flyteplugins.polars.df_transformer

Directory

Classes

Class Description
ParquetToPolarsDecodingHandler Helper class that provides a standard way to create an ABC using.
ParquetToPolarsLazyFrameDecodingHandler Helper class that provides a standard way to create an ABC using.
PolarsLazyFrameToParquetEncodingHandler Helper class that provides a standard way to create an ABC using.
PolarsToParquetEncodingHandler Helper class that provides a standard way to create an ABC using.

Methods

Method Description
get_polars_storage_options() Get storage options in a format compatible with Polars.

Variables

Property Type Description
PARQUET str

Methods

get_polars_storage_options()

def get_polars_storage_options(
    protocol: typing.Optional[str],
    anonymous: bool,
) -> typing.Dict[str, str]

Get storage options in a format compatible with Polars.

Polars requires storage_options to be a flat dict with string keys and values, unlike fsspec which accepts nested dicts and complex objects.

Parameter Type Description
protocol typing.Optional[str]
anonymous bool