Entrypoint#

class union.remote.UnionRemote(config=None, default_project='flytesnacks', default_domain='development', data_upload_location='flyte://my-s3-bucket/', **kwargs)#

Parameters:

config (Config | None)
default_project (str | None)
default_domain (str | None)
data_upload_location (str)

activate_launchplan(ident)#

Given a launchplan, activate it, all previous versions are deactivated.

Parameters:: ident (Identifier)

create_artifact(artifact)#

Create an artifact in FlyteAdmin.

Parameters:: artifact (Artifact) – The artifact to create.
Returns:: The artifact as persisted in the service.

download(data, download_to, recursive=True)#

Download the data to the specified location. If the data is a LiteralsResolver, LiteralMap and if recursive is specified, then all file like objects will be recursively downloaded (e.g. FlyteFile/Dir (blob),

StructuredDataset etc).

Note: That it will use your sessions credentials to access the remote location. For sandbox, this should be automatically configured, assuming you are running sandbox locally. For other environments, you will need to configure your credentials appropriately.

Parameters:

data (LiteralsResolver | Literal | LiteralMap) – data to be downloaded
download_to (str) – location to download to (str) that should be a valid path
recursive (bool) – if the data is a LiteralsResolver or LiteralMap, then this flag will recursively download

execute(entity, inputs, project=None, domain=None, name=None, version=None, execution_name=None, execution_name_prefix=None, image_config=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a task, workflow, or launchplan, either something that’s been declared locally, or a fetched entity.

This method supports: - Flyte{Task, Workflow, LaunchPlan} remote module objects. - @task-decorated functions and TaskTemplate tasks. - @workflow-decorated functions. - LaunchPlan objects.

For local entities, this code will attempt to find the entity first, and if missing, will compile and register the object.

Not all arguments are relevant in all circumstances. For example, there’s no reason to use the serialization settings for entities that have already been registered on Admin.

Parameters:

options (Options | None)
entity (FlyteTask | FlyteLaunchPlan | FlyteWorkflow | PythonTask | WorkflowBase | LaunchPlan) – entity to execute
inputs (Dict[str, Any]) – dictionary mapping argument names to values
project (str) – execute entity in this project. If entity doesn’t exist in the project, register the entity first before executing.
domain (str) – execute entity in this domain. If entity doesn’t exist in the domain, register the entity first before executing.
name (str) – execute entity using this name. If not None, use this value instead of entity.name
version (str) – execute entity using this version. If None, uses auto-generated value.
execution_name (str | None) – name of the execution. If None, uses auto-generated value.
image_config (ImageConfig | None)
wait (bool) – if True, waits for execution to complete
type_hints (Dict[str, Type] | None) – Python types to be passed to the TypeEngine so that it knows how to properly convert the input values for the execution into Flyte literals. If missing, will default to first guessing the type using the type engine, and then to type(v). Providing the correct Python types is particularly important if the inputs are containers like lists or maps, or if the Python type is one of the more complex Flyte provided classes (like a StructuredDataset that’s annotated with columns).
overwrite_cache (bool | None) – Allows for all cached values of a workflow and its tasks to be overwritten for a single execution. If enabled, all calculations are performed even if cached results would be available, overwriting the stored data once execution finishes successfully.
envs (Dict[str, str] | None) – Environment variables to be set for the execution.
tags (List[str] | None) – Tags to be set for the execution.
cluster_pool (str | None) – Specify cluster pool on which newly created execution should be placed.
execution_cluster_label (str | None) – Specify label of cluster(s) on which newly created execution should be placed.
execution_name_prefix (str | None)

Return type:

FlyteWorkflowExecution

execute_local_launch_plan(entity, inputs, version, project=None, domain=None, name=None, execution_name=None, execution_name_prefix=None, options=None, wait=False, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Parameters:

entity (LaunchPlan) – The locally defined launch plan object
inputs (Dict[str, Any]) – Inputs to be passed into the execution as a dict with Python native values.
version (str) – The version to look up/register the launch plan (if not already exists)
project (str | None) – The same as version, but will default to the Remote object’s project
domain (str | None) – The same as version, but will default to the Remote object’s domain
name (str | None) – The same as version, but will default to the entity’s name
execution_name (str | None) – If specified, will be used as the execution name instead of randomly generating.
options (Options | None) – Options to be passed into the execution.
wait (bool) – If True, will wait for the execution to complete before returning.
overwrite_cache (bool | None) – If True, will overwrite the cache.
envs (Dict[str, str] | None) – Environment variables to be passed into the execution.
tags (List[str] | None) – Tags to be passed into the execution.
cluster_pool (str | None) – Specify cluster pool on which newly created execution should be placed.
execution_cluster_label (str | None) – Specify label of cluster(s) on which newly created execution should be placed.
execution_name_prefix (str | None)

Returns:

FlyteWorkflowExecution object

Return type:

FlyteWorkflowExecution

execute_local_task(entity, inputs, project=None, domain=None, name=None, version=None, execution_name=None, execution_name_prefix=None, image_config=None, wait=False, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a @task-decorated function or TaskTemplate task.

Parameters:

entity (PythonTask) – local task entity.
inputs (Dict[str, Any]) – register the task, which requires compiling the task, before running it.
project (str) – The execution project, will default to the Remote’s default project.
domain (str) – The execution domain, will default to the Remote’s default domain.
name (str) – specific name of the task to run.
version (str) – specific version of the task to run.
execution_name (str | None) – If provided, will use this name for the execution.
image_config (ImageConfig | None) – If provided, will use this image config in the pod.
wait (bool) – If True, will wait for the execution to complete before returning.
overwrite_cache (bool | None) – If True, will overwrite the cache.
envs (Dict[str, str] | None) – Environment variables to set for the execution.
tags (List[str] | None) – Tags to set for the execution.
cluster_pool (str | None) – Specify cluster pool on which newly created execution should be placed.
execution_cluster_label (str | None) – Specify label of cluster(s) on which newly created execution should be placed.
execution_name_prefix (str | None)

Returns:

FlyteWorkflowExecution object.

Return type:

FlyteWorkflowExecution

execute_local_workflow(entity, inputs, project=None, domain=None, name=None, version=None, execution_name=None, execution_name_prefix=None, image_config=None, options=None, wait=False, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute an @workflow decorated function. :param entity: :param inputs: :param project: :param domain: :param name: :param version: :param execution_name: :param image_config: :param options: :param wait: :param overwrite_cache: :param envs: :param tags: :param cluster_pool: :param execution_cluster_label: :return:

Parameters:

entity (WorkflowBase)
inputs (Dict[str, Any])
project (str)
domain (str)
name (str)
version (str)
execution_name (str | None)
execution_name_prefix (str | None)
image_config (ImageConfig | None)
options (Options | None)
wait (bool)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

execute_reference_launch_plan(entity, inputs, execution_name=None, execution_name_prefix=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a ReferenceLaunchPlan.

Parameters:

entity (ReferenceLaunchPlan)
inputs (Dict[str, Any])
execution_name (str | None)
execution_name_prefix (str | None)
options (Options | None)
wait (bool)
type_hints (Dict[str, Type] | None)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

execute_reference_task(entity, inputs, execution_name=None, execution_name_prefix=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a ReferenceTask.

Parameters:

entity (ReferenceTask)
inputs (Dict[str, Any])
execution_name (str | None)
execution_name_prefix (str | None)
options (Options | None)
wait (bool)
type_hints (Dict[str, Type] | None)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

execute_reference_workflow(entity, inputs, execution_name=None, execution_name_prefix=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a ReferenceWorkflow.

Parameters:

entity (ReferenceWorkflow)
inputs (Dict[str, Any])
execution_name (str | None)
execution_name_prefix (str | None)
options (Options | None)
wait (bool)
type_hints (Dict[str, Type] | None)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

execute_remote_task_lp(entity, inputs, project=None, domain=None, execution_name=None, execution_name_prefix=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a FlyteTask, or FlyteLaunchplan.

NOTE: the name and version arguments are currently not used and only there consistency in the function signature

Parameters:

entity (FlyteTask | FlyteLaunchPlan)
inputs (Dict[str, Any])
project (str)
domain (str)
execution_name (str | None)
execution_name_prefix (str | None)
options (Options | None)
wait (bool)
type_hints (Dict[str, Type] | None)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

execute_remote_wf(entity, inputs, project=None, domain=None, execution_name=None, execution_name_prefix=None, options=None, wait=False, type_hints=None, overwrite_cache=None, envs=None, tags=None, cluster_pool=None, execution_cluster_label=None)#

Execute a FlyteWorkflow.

NOTE: the name and version arguments are currently not used and only there consistency in the function signature

Parameters:

entity (FlyteWorkflow)
inputs (Dict[str, Any])
project (str)
domain (str)
execution_name (str | None)
execution_name_prefix (str | None)
options (Options | None)
wait (bool)
type_hints (Dict[str, Type] | None)
overwrite_cache (bool | None)
envs (Dict[str, str] | None)
tags (List[str] | None)
cluster_pool (str | None)
execution_cluster_label (str | None)

Return type:

FlyteWorkflowExecution

fast_package(root, deref_symlinks=True, output=None, options=None)#

Packages the given paths into an installable zip and returns the md5_bytes and the URL of the uploaded location :param root: path to the root of the package system that should be uploaded :param output: output path. Optional, will default to a tempdir :param deref_symlinks: if symlinks should be dereferenced. Defaults to True :param options: additional options to customize fast_package behavior :return: md5_bytes, url

Parameters:

root (PathLike)
deref_symlinks (bool)
output (str)
options (FastPackageOptions | None)

Return type:

Tuple[bytes, str]

fetch_execution(project=None, domain=None, name=None)#

Fetch a workflow execution entity from flyte admin.

Parameters:

project (str) – fetch entity from this project. If None, uses the default_project attribute.
domain (str) – fetch entity from this domain. If None, uses the default_domain attribute.
name (str) – fetch entity with matching name.

Returns:

FlyteWorkflowExecution

Raises:

FlyteAssertion if name is None

Return type:

FlyteWorkflowExecution

fetch_launch_plan(project=None, domain=None, name=None, version=None)#

Fetch a launchplan entity from flyte admin.

Parameters:

project (str) – fetch entity from this project. If None, uses the default_project attribute.
domain (str) – fetch entity from this domain. If None, uses the default_domain attribute.
name (str) – fetch entity with matching name.
version (str) – fetch entity with matching version. If None, gets the latest version of the entity.

Returns:

FlyteLaunchPlan

Raises:

FlyteAssertion if name is None

Return type:

FlyteLaunchPlan

fetch_task(project=None, domain=None, name=None, version=None)#

Fetch a task entity from flyte admin.

Parameters:

project (str) – fetch entity from this project. If None, uses the default_project attribute.
domain (str) – fetch entity from this domain. If None, uses the default_domain attribute.
name (str) – fetch entity with matching name.
version (str) – fetch entity with matching version. If None, gets the latest version of the entity.

Returns:

FlyteTask

Raises:

FlyteAssertion if name is None

Return type:

FlyteTask

fetch_task_lazy(project=None, domain=None, name=None, version=None)#

Similar to fetch_task, just that it returns a LazyEntity, which will fetch the workflow lazily.

Parameters:

project (str)
domain (str)
name (str)
version (str)

Return type:

LazyEntity

fetch_workflow(project=None, domain=None, name=None, version=None)#

Fetch a workflow entity from flyte admin. :param project: fetch entity from this project. If None, uses the default_project attribute. :param domain: fetch entity from this domain. If None, uses the default_domain attribute. :param name: fetch entity with matching name. :param version: fetch entity with matching version. If None, gets the latest version of the entity. :raises: FlyteAssertion if name is None

Parameters:

project (str)
domain (str)
name (str)
version (str)

Return type:

FlyteWorkflow

fetch_workflow_lazy(project=None, domain=None, name=None, version=None)#

Similar to fetch_workflow, just that it returns a LazyEntity, which will fetch the workflow lazily.

Parameters:

project (str)
domain (str)
name (str)
version (str)

Return type:

LazyEntity[FlyteWorkflow]

generate_console_http_domain()#

This should generate the domain where console is hosted.

Returns:
Return type:: str

generate_console_url(entity)#

Generate a UnionAI console URL for the given Union remote endpoint.

This will automatically determine if this is an execution or an entity and change the type automatically.

Parameters:: entity (FlyteWorkflowExecution | FlyteNodeExecution | FlyteTaskExecution | FlyteWorkflow | FlyteTask | FlyteLaunchPlan)

get(uri=None)#

General function that works with flyte tiny urls. This can return outputs (in the form of LiteralsResolver, or individual Literals for singular requests), or HTML if passed a deck link, or bytes containing HTML, if ipython is not available locally.

Parameters:: uri (str | None)
Return type:: LiteralsResolver | Literal | bytes | None

get_artifact(uri=None, artifact_key=None, artifact_id=None, query=None, get_details=False)#

Get the specified artifact.

Parameters:

uri (str | None) – An artifact URI.
artifact_key (ArtifactKey | None) – An artifact key.
artifact_id (ArtifactID | None) – The artifact ID.
query (ArtifactQuery | None) – An artifact query.
get_details (bool) – A bool to indicate whether or not to return artifact details.

Returns:

The artifact as persisted in the service.

Return type:

Artifact | None

static get_extra_headers_for_protocol(native_url)#

launch_backfill(project, domain, from_date, to_date, launchplan, launchplan_version=None, execution_name=None, version=None, dry_run=False, execute=True, parallel=False, failure_policy=None, overwrite_cache=None)#

Creates and launches a backfill workflow for the given launchplan. If launchplan version is not specified, then the latest launchplan is retrieved. The from_date is exclusive and end_date is inclusive and backfill run for all instances in between.

-> (start_date - exclusive, end_date inclusive)

If dry_run is specified, the workflow is created and returned. If execute==False is specified then the workflow is created and registered. In the last case, the workflow is created, registered and executed.

The parallel flag can be used to generate a workflow where all launchplans can be run in parallel. Default is that execute backfill is run sequentially

Parameters:

project (str) – str project name
domain (str) – str domain name
from_date (datetime) – datetime generate a backfill starting at this datetime (exclusive)
to_date (datetime) – datetime generate a backfill ending at this datetime (inclusive)
launchplan (str) – str launchplan name in the union backend
launchplan_version (str) – str (optional) version for the launchplan. If not specified the most recent will be retrieved
execution_name (str) – str (optional) the generated execution will be named so. this can help in ensuring idempotency
version (str) – str (optional) version to be used for the newly created workflow.
dry_run (bool) – bool do not register or execute the workflow
execute (bool) – bool Register and execute the wwkflow.
parallel (bool) – if the backfill should be run in parallel. False (default) will run each bacfill sequentially.
failure_policy (Optional[WorkflowFailurePolicy]) – WorkflowFailurePolicy (optional) to be used for the newly created workflow. This can control failure behavior - whether to continue on failure or stop immediately on failure
overwrite_cache (Optional[bool]) – if True, will overwrite the cache.

Returns:

In case of dry-run, return WorkflowBase, else if no_execute return FlyteWorkflow else in the default case return a FlyteWorkflowExecution

Return type:

Optional[FlyteWorkflowExecution, FlyteWorkflow, WorkflowBase]

list_signals(execution_name, project=None, domain=None, limit=100, filters=None)#

Parameters:

execution_name (str) – The name of the execution. This is the tailend of the URL when looking at the workflow execution.
project (str | None) – The execution project, will default to the Remote’s default project.
domain (str | None) – The execution domain, will default to the Remote’s default domain.
limit (int) – The number of signals to fetch
filters (List[Filter] | None) – Optional list of filters

Return type:

List[Signal]

list_tasks_by_version(version, project=None, domain=None, limit=100)#

Parameters:

version (str)
project (str | None)
domain (str | None)
limit (int | None)

Return type:

List[FlyteTask]

raw_register(cp_entity, settings, version, create_default_launchplan=True, options=None, og_entity=None)#

Raw register method, can be used to register control plane entities. Usually if you have a Flyte Entity like a WorkflowBase, Task, LaunchPlan then use other methods. This should be used only if you have already serialized entities

Parameters:

cp_entity (TaskSpec | LaunchPlan | WorkflowSpec | Node | BranchNode | ArrayNode) – The controlplane “serializable” version of a flyte entity. This is in the form that FlyteAdmin understands.
settings (SerializationSettings) – SerializationSettings to be used for registration - especially to identify the id
version (str) – Version to be registered
create_default_launchplan (bool) – boolean that indicates if a default launch plan should be created
options (Options) – Options to be used if registering a default launch plan
og_entity (PythonTask | BranchNode | Node | LaunchPlan | WorkflowBase | ReferenceWorkflow | ReferenceTask | ReferenceLaunchPlan | ReferenceEntity | ArrayNode) – Pass in the original workflow (flytekit type) if create_default_launchplan is true

Returns:

Identifier of the created entity

Return type:

Identifier | None

recent_executions(project=None, domain=None, limit=100, filters=None)#

Parameters:

project (str | None)
domain (str | None)
limit (int | None)
filters (List[Filter] | None)

Return type:

List[FlyteWorkflowExecution]

register_launch_plan(entity, version, project=None, domain=None, options=None)#

Register a given launchplan, possibly applying overrides from the provided options. :param entity: Launchplan to be registered :param version: :param project: Optionally provide a project, if not already provided in UnionRemote constructor or a separate one :param domain: Optionally provide a domain, if not already provided in UnionRemote constructor or a separate one :param options: :return:

Parameters:

entity (LaunchPlan)
version (str)
project (str | None)
domain (str | None)
options (Options | None)

Return type:

FlyteLaunchPlan

register_script(entity, image_config=None, version=None, project=None, domain=None, destination_dir='.', copy_all=False, default_launch_plan=True, options=None, source_path=None, module_name=None, envs=None, fast_package_options=None)#

Use this method to register a workflow via script mode. :param destination_dir: The destination directory where the workflow will be copied to. :param copy_all: [deprecated] Please use the copy_style field in fast_package_options instead. :param domain: The domain to register the workflow in. :param project: The project to register the workflow in. :param image_config: The image config to use for the workflow. :param version: version for the entity to be registered as :param entity: The workflow to be registered or the task to be registered :param default_launch_plan: This should be true if a default launch plan should be created for the workflow :param options: Additional execution options that can be configured for the default launchplan :param source_path: The root of the project path :param module_name: the name of the module :param envs: Environment variables to be passed to the serialization :param fast_package_options: Options to customize copy_all behavior, ignored when copy_all is False. :return:

Parameters:

entity (WorkflowBase | PythonTask)
image_config (ImageConfig | None)
version (str | None)
project (str | None)
domain (str | None)
destination_dir (str)
copy_all (bool)
default_launch_plan (bool)
options (Options | None)
source_path (str | None)
module_name (str | None)
envs (Dict[str, str] | None)
fast_package_options (FastPackageOptions | None)

Return type:

FlyteWorkflow | FlyteTask

register_task(entity, serialization_settings=None, version=None)#

Register a qualified task (PythonTask) with Remote For any conflicting parameters method arguments are regarded as overrides

Parameters:

entity (PythonTask) – PythonTask can be either @task or a instance of a Task class
serialization_settings (SerializationSettings | None) – Settings that will be used to override various serialization parameters.
version (str | None) – version that will be used to register. If not specified will default to using the serialization settings default

Returns:

Return type:

FlyteTask

register_workflow(entity, serialization_settings=None, version=None, default_launch_plan=True, options=None)#

Use this method to register a workflow. :param version: version for the entity to be registered as :param entity: The workflow to be registered :param serialization_settings: The serialization settings to be used :param default_launch_plan: This should be true if a default launch plan should be created for the workflow :param options: Additional execution options that can be configured for the default launchplan :return:

Parameters:

entity (WorkflowBase)
serialization_settings (SerializationSettings | None)
version (str | None)
default_launch_plan (bool | None)
options (Options | None)

Return type:

FlyteWorkflow

remote_context()#: Context manager with remote-specific configuration.

set_signal(signal_id, execution_name, value, project=None, domain=None, python_type=None, literal_type=None)#

Parameters:

signal_id (str) – The name of the signal, this is the key used in the approve() or wait_for_input() call.
execution_name (str) – The name of the execution. This is the tail-end of the URL when looking at the workflow execution.
value (Literal | Any) – This is either a Literal or a Python value which UnionRemote will invoke the TypeEngine to convert into a Literal. This argument is only value for wait_for_input type signals.
project (str | None) – The execution project, will default to the Remote’s default project.
domain (str | None) – The execution domain, will default to the Remote’s default domain.
python_type (Type | None) – Provide a python type to help with conversion if the value you provided is not a Literal.
literal_type (LiteralType | None) – Provide a Flyte literal type to help with conversion if the value you provided is not a Literal

sync(execution, entity_definition=None, sync_nodes=False)#

This function was previously a singledispatchmethod. We’ve removed that but this function remains so that we don’t break people.

Parameters:

execution (FlyteWorkflowExecution)
entity_definition (FlyteWorkflow | FlyteTask)
sync_nodes (bool) – By default sync will fetch data on all underlying node executions (recursively, so subworkflows will also get picked up). Set this to False in order to prevent that (which will make this call faster).

Returns:

Returns the same execution object, but with additional information pulled in.

Return type:

FlyteWorkflowExecution

sync_execution(execution, entity_definition=None, sync_nodes=False)#

Sync a FlyteWorkflowExecution object with its corresponding remote state.

Parameters:

execution (FlyteWorkflowExecution)
entity_definition (FlyteWorkflow | FlyteTask)
sync_nodes (bool)

Return type:

FlyteWorkflowExecution

sync_node_execution(execution, node_mapping)#

Get data backing a node execution. These FlyteNodeExecution objects should’ve come from Admin with the model fields already populated correctly. For purposes of the remote experience, we’d like to supplement the object with some additional fields: - inputs/outputs - task/workflow executions, and/or underlying node executions in the case of parent nodes - TypedInterface (remote wrapper type)

A node can have several different types of executions behind it. That is, the node could’ve run (perhaps multiple times because of retries): - A task - A static subworkflow - A dynamic subworkflow (which in turn may have run additional tasks, subwfs, and/or launch plans) - A launch plan

The data model is complicated, so ascertaining which of these happened is a bit tricky. That logic is encapsulated in this function.

Parameters:

execution (FlyteNodeExecution)
node_mapping (Dict[str, FlyteNode])

Return type:

FlyteNodeExecution

sync_task_execution(execution, entity_definition=None)#

Sync a FlyteTaskExecution object with its corresponding remote state.

Parameters:

execution (FlyteTaskExecution)
entity_definition (FlyteTask | None)

Return type:

FlyteTaskExecution

terminate(execution, cause)#

Terminate a workflow execution.

Parameters:

execution (FlyteWorkflowExecution) – workflow execution to terminate
cause (str) – reason for termination

upload_file(to_upload, project=None, domain=None, filename_root=None)#

Function will use remote’s client to hash and then upload the file using Admin’s data proxy service.

Parameters:

to_upload (Path) – Must be a single file
project (str | None) – Project to upload under, if not supplied will use the remote’s default
domain (str | None) – Domain to upload under, if not specified will use the remote’s default
filename_root (str | None) – If provided will be used as the root of the filename. If not, Admin will use a hash

Returns:

The uploaded location.

Return type:

Tuple[bytes, str]

wait(execution, timeout=None, poll_interval=None, sync_nodes=True)#

Wait for an execution to finish.

Parameters:

execution (FlyteWorkflowExecution) – execution object to wait on
timeout (timedelta | None) – maximum amount of time to wait
poll_interval (timedelta | None) – sync workflow execution at this interval
sync_nodes (bool) – passed along to the sync call for the workflow execution

Return type:

FlyteWorkflowExecution

class flytekit.remote.remote.Options(labels=None, annotations=None, raw_output_data_config=None, security_context=None, max_parallelism=None, notifications=None, disable_notifications=None, overwrite_cache=None)#

These are options that can be configured for a launchplan during registration or overridden during an execution. For instance two people may want to run the same workflow but have the offloaded data stored in two different buckets. Or you may want labels or annotations to be different. This object is used when launching an execution in a Union backend, and also when registering launch plans.

Args:

labels: Custom labels to be applied to the execution resource annotations: Custom annotations to be applied to the execution resource security_context: Indicates security context for permissions triggered with this launch plan raw_output_data_config: Optional location of offloaded data for things like S3, etc.

remote prefix for storage location of the form s3://<bucket>/key... or gcs://... or file://.... If not specified will use the platform configured default. This is where the data for offloaded types is stored.

max_parallelism: Controls the maximum number of tasknodes that can be run in parallel for the entire workflow. notifications: List of notifications for this execution. disable_notifications: This should be set to true if all notifications are intended to be disabled for this execution.

Parameters:

labels (Labels | None)
annotations (Annotations | None)
raw_output_data_config (RawOutputDataConfig | None)
security_context (SecurityContext | None)
max_parallelism (int | None)
notifications (List[Notification] | None)
disable_notifications (bool | None)
overwrite_cache (bool | None)