Core flytekit#

This package contains the most common abstractions you’ll need to write Flyte workflows and extend flytekit.

Basic authoring#

These are the essentials needed to get started writing tasks and workflows.

task

This is the core decorator to use for any task type in flytekit.

workflow

This decorator declares a function to be a Flyte workflow.

kwtypes

This is a small helper function to convert the keyword arguments to an OrderedDict of types.

current_context

Use this method to get a handle of specific parameters available in a flyte task.

ExecutionParameters

This is a run-time user-centric context object that is accessible to every @task method.

FlyteContext

This is an internal-facing context object, that most users will not have to deal with.

map_task

Wrapper that creates a map task utilizing either the existing ArrayNodeMapTask or the drop in replacement ArrayNode implementation

ImperativeWorkflow

An imperative workflow is a programmatic analogue to the typical @workflow function-based workflow and is better suited to programmatic applications.

create_node

This is the function you want to call if you need to specify dependencies between tasks that don't consume and/or don't produce outputs. For example, if you have t1() and t2(), both of which do not take in nor produce any outputs, how do you specify that t2 should run before t1? ::.

NodeOutput

FlyteContextManager

FlyteContextManager manages the execution context within Flytekit.

Note

Tasks and workflows can both be locally run, assuming the relevant tasks are capable of local execution. This is useful for unit testing.

Branching and conditionals#

Branches and conditionals can be expressed explicitly in Flyte. These conditions are evaluated in the flyte engine and hence should be used for control flow. Dynamic workflows can be used to perform custom conditional logic not supported by flytekit.

conditional

Use a conditional section to control the flow of a workflow.

Customizing Tasks & Workflows#

TaskMetadata

Metadata for a Task.

Resources

This class is used to specify both resource requests and resource limits.

WorkflowFailurePolicy

Defines the behavior for a workflow execution in the case of an observed node execution failure.

PodTemplate

Custom PodTemplate specification for a Task.

Dynamic and nested workflows#

dynamic

Please first see the comments for flytekit.task() and flytekit.workflow().

Signaling#

approve

Create a Gate object for binary approval.

sleep

Create a sleep Gate object.

wait_for_input

Create a Gate object that waits for user input of the specified type.

Scheduling#

CronSchedule

Use this when you have a launch plan that you want to run on a cron expression.

FixedRate

Use this class to schedule a fixed-rate interval for a launch plan.

Notifications#

Email

This notification should be used when sending regular emails to people.

PagerDuty

This notification should be used when sending emails to the PagerDuty service.

Slack

This notification should be used when sending emails to the Slack.

Reference entities#

get_reference_entity

See the documentation for flytekit.reference_task and flytekit.reference_workflow as well.

LaunchPlanReference

A reference object containing metadata that points to a remote launch plan.

TaskReference

A reference object containing metadata that points to a remote task.

WorkflowReference

A reference object containing metadata that points to a remote workflow.

reference_task

A reference task is a pointer to a task that already exists on your Flyte installation.

reference_workflow

A reference workflow is a pointer to a workflow that already exists on your Flyte installation.

reference_launch_plan

A reference launch plan is a pointer to a launch plan that already exists on your Flyte installation.

Core task types#

SQLTask

Base task types for all SQL tasks.

ContainerTask

This is an intermediate class that represents Flyte Tasks that run a container at execution time.

PythonFunctionTask

A Python Function task should be used as the base for all extensions that have a python function.

PythonInstanceTask

This class should be used as the base class for all Tasks that do not have a user defined function body, but have a platform defined execute method.

LaunchPlan

Launch Plans are one of the core constructs of Flyte.

Secrets and SecurityContext#

Secret

See cookbook:secrets for usage examples.

SecurityContext

This is a higher level wrapper object that for the most part users shouldn't have to worry about.

Common Flyte IDL Objects#

AuthRole

Labels

Annotations

WorkflowExecutionPhase

This class holds enum values used for setting notifications.

Blob

BlobMetadata

This is metadata for the Blob literal.

Literal

Scalar

LiteralType

BlobType

This type represents offloaded data and is typically used for things like files.

Task utilities#

HashMethod

Flyte-specific object used to wrap the hash function for a specific type

Artifacts#

Artifact

An Artifact is effectively just a metadata layer on top of data that exists in Flyte.

Documentation#

Description

Full user description with formatting preserved.

Documentation

DescriptionEntity contains detailed description for the task/workflow/launch plan. Documentation could provide insight into the algorithms, business use case, etc. Args: short_description (str): One-liner overview of the entity. long_description (Optional[Description]): Full user description with formatting preserved. source_code (Optional[SourceCode]): link to source code used to define this entity.

SourceCode

Link to source code used to define this task or workflow.